I would like to scrape http://www.cpso.on.ca/publicregister/default.aspx?id=2048. by city. Specifically I have list of city names I would enter on the "form", hit submit and then collect/ parse all the results (across all pages), and then repeat for each city. I have never had to do a "post" before and this one is unlike any thing I've ever seen. Not even sure where to begin. Not sure how mechanize would work (or could work) in such a situation.

Thanks for any help.


Recommended Answers

I would recommend starting with BeautifulSoup, which is made for just such a purpose.

Jump to Post

All 2 Replies

Sorry for the delay in response I was expecting an email notification...
My problem is not so much with the parsing of the data (beautiful soup) as the actual gathering of the data. I am unable to send the "post" back to the server to get the pages for parsing.


Be a part of the DaniWeb community

We're a friendly, industry-focused community of 1.21 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.