I have code that opens a URL, whose page looks like this...


The code parses this web page and writes it to a CSV file. Here is a snipet of the code,

def __req(stat):
    url = ... % (stat)
    return urllib.urlopen(url).read().strip().strip('"')

#Manipulate data from web and write to a csv file 
def get_values(symbol):
    values = __req('l1m3m4p2va2j1j4').split(',')                  
        for x in values:data.append(x)

This works with a one line web-page. But what if the web-page has more than 1 line? For example,

How would I have to modify the code to capture all 5 lines from the site instead of just the first one? Thanks again.

Edited by oaktrees: n/a

8 Years
Discussion Span
Last Post by oaktrees

You can use generators to iterate over your web page's lines


def gen_lines(symbol, stat):
    url = URL_TEMPLATE % (symbol, stat)
    for line in urllib.urlopen(url):
        yield line.strip()

def gen_rows(symbol, stat):
    for line in gen_lines(symbol, stat):
        row = line.split(",")
        row = [ x.strip('"') for x in row ]
        yield row

def test_me(symbol):
    for row in gen_rows(symbol, 'l1m3m4p2va2j1j4'):
        print row

This makes sense. Could you tell me what this line of could is doing, it's syntax is different than I've seen before:

row = [ x.strip('"') for x in row ]

Yes. This solution works perfectly. Thank you for teaching me about generators.

This question has already been answered. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.