Screen Scraping - part deux

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site
Reply

Join Date: May 2008
Posts: 11
Reputation: ccandillo is an unknown quantity at this point 
Solved Threads: 0
ccandillo ccandillo is offline Offline
Newbie Poster

Screen Scraping - part deux

 
0
  #1
Nov 14th, 2008
I have written a script that scrapes a particular website and returns the date of an article, description and url. I want to display this information on the console with clickable text so that when I click on text/link called "site" it will open the url of the article in my default browser.

I have been searching google for the best way to do this but I have not found too much. Should I use curses, if so, I am not sure which curses module I should use. Or is there a better/easier way?

Any ideas?
Reply With Quote Quick reply to this message  
Join Date: Jul 2008
Posts: 1,067
Reputation: jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough 
Solved Threads: 267
Sponsor
jlm699's Avatar
jlm699 jlm699 is offline Offline
Knows where his Towel is

Re: Screen Scraping - part deux

 
0
  #2
Nov 14th, 2008
You want to create a hyperlink within the terminal window? I don't know if that would be possible through standard means, and using curses would be overkill in my opinion.

Does this script require to be used within the terminal or would you be willing to give it a nice GUI with wxPython or similar? That way you could easily implement hyperlinks.

If you wanted to keep it within the terminal, you could list a letter before each link, a la:
A) www.google.com
B) www.etc.com
C) www.foo.bar

And then have a raw_input('Enter a letter to go to that website, exit to quit: ') or something similar, which would then call the default browser via the webbrowser module.
Last edited by jlm699; Nov 14th, 2008 at 1:02 pm.
1. Use Code Tags.
2. Homework? Show Effort.
3. Keep discussions on the forum: no PMs
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 11
Reputation: ccandillo is an unknown quantity at this point 
Solved Threads: 0
ccandillo ccandillo is offline Offline
Newbie Poster

Re: Screen Scraping - part deux

 
0
  #3
Nov 14th, 2008
Using raw_input() would be way easy but my script updates every 5 minuites. And I am not sure that the script would be able to update if it's waiting on user input.
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 949
Reputation: Paul Thompson has a spectacular aura about Paul Thompson has a spectacular aura about 
Solved Threads: 146
Sponsor
Paul Thompson's Avatar
Paul Thompson Paul Thompson is offline Offline
previously paulthom12345

Re: Screen Scraping - part deux

 
0
  #4
Nov 14th, 2008
Ah i have had this problem before. I found it really really useful to use Threading.
So maybe you could make a function or class that uses Threading and then you can have it wait for user input while the rest of the program goes its own way.
Make it idiot proof and someone will make a better idiot.
Check out my Site | and join us on IRC | Python Specific IRC
Reply With Quote Quick reply to this message  
Join Date: Jun 2007
Posts: 1,467
Reputation: evstevemd has a spectacular aura about evstevemd has a spectacular aura about evstevemd has a spectacular aura about 
Solved Threads: 128
evstevemd's Avatar
evstevemd evstevemd is online now Online
Nearly a Posting Virtuoso

Re: Screen Scraping - part deux

 
0
  #5
Nov 14th, 2008
Why not use GUI as Jlm699 suggested. Wxpython have nice HTML window which will make your program look nice!

+1 for jlm699 - GUI makes it easier
Atheist: God is man made imagination, he doesn't exist!
Theist: It's okay, can you imagine anything else that doesn't exist?
---- Python, C++ PHP and Java ----
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 11
Reputation: ccandillo is an unknown quantity at this point 
Solved Threads: 0
ccandillo ccandillo is offline Offline
Newbie Poster

Re: Screen Scraping - part deux

 
0
  #6
Nov 14th, 2008
Well, I don't think I'm ready for Wxpython so I'm gonna try to use Threading. I am also looking at the code for "hacker_top.py" which is along the lines of what I want to do. If someone out there is curious too, it's a simple google search away.

Thanks for the info guys!
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Other Threads in the Python Forum
Thread Tools Search this Thread



Tag cloud for Python
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC