943,999 Members | Top Members by Rank

Ad:
  • Python Discussion Thread
  • Unsolved
  • Views: 1194
  • Python RSS
Aug 8th, 2007
0

Python doubt

Expand Post »
Hi friends:
First of all, thanks for your time.
I was wondering if it´s possible to develop (using Python, of course) some kind of application which can interact whith an internet page to which I'am a registered user (it´s not a matter of hacking) and to work whith the results retrieved (mostly article prices).
The thing is, in a first page, I have to log in, once logged, and then the second screen gives me an input where to make the consults. All of this I can do it with any web browser but I want to change the interfase to add more functionallity.
Can you give some guide about it?

Thanks, from Argentina,

Albini, Gabriel E.-
Similar Threads
Reputation Points: 10
Solved Threads: 0
Newbie Poster
galbi is offline Offline
2 posts
since Aug 2007
Aug 8th, 2007
0

Re: Python doubt

This question intrigued me and sent me into the details of Foundations of Python Network Programming.

Think about it like this:

(1) You need to authenticate.
(2) You need to read the data.
(3) You need to find some way to display and manipulate the data.

I assume that your desire is for read-only functionality; i.e., you aren't trying to send the manipulated data back to the server?

The thing is, (1) and (2) are trivial. (3) is hard(er).

Here's simple code (tweaked from FoPNP) that spits out the contents of a web page:

Python Syntax (Toggle Plain Text)
  1. import sys, urllib2
  2.  
  3.  
  4. URL = raw_input("Where shall we go today? ") # ex.: http://www.google.com
  5. req = urllib2.Request(URL)
  6. fd = urllib2.urlopen(req)
  7.  
  8. while True:
  9. data = fd.read(1024)
  10. if not len(data):
  11. break
  12. sys.stdout.write(data)

Note that this is a really mindless program; it won't even think to attach the "http://" for you. If needed, authentication is handled by urllib2.HTTPBasicAuthHandler.

If you run it with, say, http://www.google.com, you'll get the text version of the HTML.

This is where part (3) comes in. To get a Firefox-like rendering of the web-page, your program has to replicate browser functionality. Congratulations! You get to re-write Netscape! YUCK.

Another possibility is that you could read in the HTML, tweak it (by parsing it, and then inserting stuff of your own), and then write it as a local .htm file that you could use Firefox (or IE if you must) to display. That's a do-able project, and it only requires learning to parse HTML.

Fortunately, the modules HTMLParser and htmllib are available to help you. I've never done this, so my help must stop here.

Jeff

P.S.
The other possibility would be to create your own FireFox extension to modify your browser to create the tools you need. That's probably the right solution for your problem, but I know nothing about that and can't even say whether it would be easy or hard. Here is a tutorial on writing FF extensions:

http://www.rietta.com/firefox/index.html
Last edited by jrcagle; Aug 8th, 2007 at 10:05 pm.
Reputation Points: 92
Solved Threads: 156
Practically a Master Poster
jrcagle is offline Offline
608 posts
since Jul 2006
Aug 9th, 2007
0

Re: Python doubt

Did You consider writing a Greasemonkey script? That is a rather simple way so enhance a webpages functionality.
Reputation Points: 20
Solved Threads: 3
Newbie Poster
N317V is offline Offline
23 posts
since Jul 2006
Aug 10th, 2007
0

Re: Python doubt

I was just thinking about this myself and came up with ClientForm, but haven't tried it yet.
http://wwwsearch.sourceforge.net/ClientForm/
Reputation Points: 741
Solved Threads: 692
Nearly a Posting Maven
woooee is offline Offline
2,307 posts
since Dec 2006
Aug 10th, 2007
0

Re: Python doubt

I was just thinking about this myself and came up with ClientForm, but haven't tried it yet.
http://wwwsearch.sourceforge.net/ClientForm/
Reputation Points: 741
Solved Threads: 692
Nearly a Posting Maven
woooee is offline Offline
2,307 posts
since Dec 2006
Aug 12th, 2007
0

Re: Python doubt

Thanks everybody for your answers.
Jeff: Those are really good news, :-) I'm not crazy for for something like this. I´ve tried your code and worked fine.
N317V: Great alternative!!! I like the idea, instead of python I've to learn javascript... pitty, I liked Python.
Woooee: A bit too much for me yet.

Thanks again,

G.E.A.
Reputation Points: 10
Solved Threads: 0
Newbie Poster
galbi is offline Offline
2 posts
since Aug 2007
Aug 13th, 2007
0

Re: Python doubt

Click to Expand / Collapse  Quote originally posted by galbi ...
N317V: Great alternative!!! I like the idea, instead of python I've to learn javascript... pitty, I liked Python.
I suggest you learn both. :-)
Reputation Points: 20
Solved Threads: 3
Newbie Poster
N317V is offline Offline
23 posts
since Jul 2006

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Python Forum Timeline: simple regex..
Next Thread in Python Forum Timeline: interp2d





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC