954,525 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Have something to say? Contribute New Article Reply to this Article

Web-Scrape Loop via CSV list of URLs ???

HI,
I've got a list of 10 websites in CSV. All of the sites have the same general format, including a large table. I only want the the data in the 7th columns. I am able to extract the html and filter the 7th column data (via RegEx) on an individual basis but I can't figure out how to loop through the CSV. I think I'm close but my script won't run. I would really appreciate it if someone could help me figure-out how to do this. Here's what i've got:

[Code]
#Python v2.6.2

import csv
import urllib2
import re

urls = csv.reader(open('list.csv'))
n =0
while n <=10:
for url in urls:
response = urllib2.urlopen(url[n])
html = response.read()
print re.findall('td7.*?td',html)
n +=1
[iCode]

PYGUY
Newbie Poster
2 posts since Sep 2009
Reputation Points: 10
Solved Threads: 0
 

Here is the answer. My CSV format was off.

#Python v2.6.2

import csv 
import urllib2
import re

urls = csv.reader(open('list.csv'))
for url in urls:
    response = urllib2.urlopen(url[0])
    html = response.read()
    print re.findall('td7.*?td',html)
PYGUY
Newbie Poster
2 posts since Sep 2009
Reputation Points: 10
Solved Threads: 0
 

This question has already been solved

Post: Markdown Syntax: Formatting Help
You
View similar articles that have also been tagged: