943,762 Members | Top Members by Rank

Ad:
  • Python Discussion Thread
  • Unsolved
  • Views: 634
  • Python RSS
Nov 14th, 2008
0

Screen Scraping - part deux

Expand Post »
I have written a script that scrapes a particular website and returns the date of an article, description and url. I want to display this information on the console with clickable text so that when I click on text/link called "site" it will open the url of the article in my default browser.

I have been searching google for the best way to do this but I have not found too much. Should I use curses, if so, I am not sure which curses module I should use. Or is there a better/easier way?

Any ideas?
Reputation Points: 10
Solved Threads: 0
Newbie Poster
ccandillo is offline Offline
16 posts
since May 2008
Nov 14th, 2008
0

Re: Screen Scraping - part deux

You want to create a hyperlink within the terminal window? I don't know if that would be possible through standard means, and using curses would be overkill in my opinion.

Does this script require to be used within the terminal or would you be willing to give it a nice GUI with wxPython or similar? That way you could easily implement hyperlinks.

If you wanted to keep it within the terminal, you could list a letter before each link, a la:
A) www.google.com
B) www.etc.com
C) www.foo.bar

And then have a raw_input('Enter a letter to go to that website, exit to quit: ') or something similar, which would then call the default browser via the webbrowser module.
Last edited by jlm699; Nov 14th, 2008 at 1:02 pm.
Reputation Points: 355
Solved Threads: 292
Veteran Poster
jlm699 is offline Offline
1,102 posts
since Jul 2008
Nov 14th, 2008
0

Re: Screen Scraping - part deux

Using raw_input() would be way easy but my script updates every 5 minuites. And I am not sure that the script would be able to update if it's waiting on user input.
Reputation Points: 10
Solved Threads: 0
Newbie Poster
ccandillo is offline Offline
16 posts
since May 2008
Nov 14th, 2008
0

Re: Screen Scraping - part deux

Ah i have had this problem before. I found it really really useful to use Threading.
So maybe you could make a function or class that uses Threading and then you can have it wait for user input while the rest of the program goes its own way.
Reputation Points: 264
Solved Threads: 183
Veteran Poster
Paul Thompson is offline Offline
1,095 posts
since May 2008
Nov 14th, 2008
0

Re: Screen Scraping - part deux

Why not use GUI as Jlm699 suggested. Wxpython have nice HTML window which will make your program look nice!

+1 for jlm699 - GUI makes it easier
Reputation Points: 462
Solved Threads: 392
Senior Poster
evstevemd is offline Offline
3,681 posts
since Jun 2007
Nov 14th, 2008
0

Re: Screen Scraping - part deux

Well, I don't think I'm ready for Wxpython so I'm gonna try to use Threading. I am also looking at the code for "hacker_top.py" which is along the lines of what I want to do. If someone out there is curious too, it's a simple google search away.

Thanks for the info guys!
Reputation Points: 10
Solved Threads: 0
Newbie Poster
ccandillo is offline Offline
16 posts
since May 2008

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Python Forum Timeline: How to detect a click in a grid cell?
Next Thread in Python Forum Timeline: wxPython and opencv





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC