Hey guys, I'm a little confused on a web scraping topic. I understand that you can use urllib2 to webscrape, but my project requires a little more advanced scraping. I want to use python to inpute data into a search box, then return the results of that information. As an example, I want to be able to input a doctors name into healthgrades.com, then retrieve the results, as well as the url links to those results. Can anyone help me out? I've been reading a lot about something called mechanize, but I don't really know where to begin with all of this. Thank you.
de1337ed
0
Newbie Poster
Recommended Answers
Jump to PostTry BeautifulSoup or try Python standard library parsers http://docs.python.org/library/markup.html.
Jump to PostA HTML form submits data using HTTP GET or HTTP POST. See what your form does and write code accordingly. Since you are using urllib, I assumed you know HTTP GET or HTTP POST. If not search google. THis HOWTO tells you how to use urbllib2 to submit data
All 6 Replies
GreenDay2001
de1337ed
0
Newbie Poster
GreenDay2001
woooee
814
Nearly a Posting Maven
de1337ed
0
Newbie Poster
griswolf
304
Veteran Poster
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.