I have code that opens a URL, whose page looks like this...

"A",31.49,"5/26/2010","4:00pm",+0.25,31.51,32.25,31.37,3811721

The code parses this web page and writes it to a CSV file. Here is a snipet of the code,

def __req(stat):
    url = ... % (stat)
    return urllib.urlopen(url).read().strip().strip('"')

#Manipulate data from web and write to a csv file 
def get_values(symbol):
    values = __req('l1m3m4p2va2j1j4').split(',')                  
        data=[]
        for x in values:data.append(x)

This works with a one line web-page. But what if the web-page has more than 1 line? For example,
"A",31.49,"5/26/2010","4:00pm",+0.25,31.51,32.25,31.37,3811721
"AA",11.25,"5/26/2010","4:00pm",-0.05,11.64,11.68,11.20,34764564
"AACC",5.94,"5/26/2010","4:00pm",+0.40,5.64,6.00,5.56,76466
"AAI",5.50,"5/26/2010","4:00pm",+0.17,5.38,5.60,5.35,5000869
"AAON",23.28,"5/26/2010","4:00pm",+0.01,23.56,23.70,23.15,83939

How would I have to modify the code to capture all 5 lines from the site instead of just the first one? Thanks again.

Recommended Answers

All 4 Replies

You can use generators to iterate over your web page's lines

URL_TEMPLATE = ...

def gen_lines(symbol, stat):
    url = URL_TEMPLATE % (symbol, stat)
    for line in urllib.urlopen(url):
        yield line.strip()

def gen_rows(symbol, stat):
    for line in gen_lines(symbol, stat):
        row = line.split(",")
        row = [ x.strip('"') for x in row ]
        yield row

def test_me(symbol):
    for row in gen_rows(symbol, 'l1m3m4p2va2j1j4'):
        print row

This makes sense. Could you tell me what this line of could is doing, it's syntax is different than I've seen before:

row = [ x.strip('"') for x in row ]

This makes sense. Could you tell me what this line of could is doing, it's syntax is different than I've seen before:

row = [ x.strip('"') for x in row ]

It's called "list comprehension syntax". See the doc here http://docs.python.org/tutorial/datastructures.html#list-comprehensions.
In the above code, the line removes the " surrounding words in your html file (is it really what you want ?).

Yes. This solution works perfectly. Thank you for teaching me about generators.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.