Python

Python RSS

Python doubt

•

Join Date: Aug 2007

Posts: 2

Reputation: galbi is an unknown quantity at this point

Solved Threads: 0

galbi

Offline

Newbie Poster

Python doubt

Aug 8th, 2007

Hi friends:
First of all, thanks for your time.
I was wondering if it´s possible to develop (using Python, of course) some kind of application which can interact whith an internet page to which I'am a registered user (it´s not a matter of hacking) and to work whith the results retrieved (mostly article prices).
The thing is, in a first page, I have to log in, once logged, and then the second screen gives me an input where to make the consults. All of this I can do it with any web browser but I want to change the interfase to add more functionallity.
Can you give some guide about it?

Thanks, from Argentina,

Albini, Gabriel E.-

•

Join Date: Jul 2006

Posts: 608

Reputation: jrcagle is on a distinguished road

Solved Threads: 150

jrcagle

Offline

Practically a Master Poster

Re: Python doubt

Aug 8th, 2007

This question intrigued me and sent me into the details of Foundations of Python Network Programming.

Think about it like this:

(1) You need to authenticate.
(2) You need to read the data.
(3) You need to find some way to display and manipulate the data.

I assume that your desire is for read-only functionality; i.e., you aren't trying to send the manipulated data back to the server?

The thing is, (1) and (2) are trivial. (3) is hard(er).

Here's simple code (tweaked from FoPNP) that spits out the contents of a web page:

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
import sys, urllib2
 
 
URL = raw_input("Where shall we go today? ") # ex.:  http://www.google.com
req = urllib2.Request(URL)
fd = urllib2.urlopen(req)
 
while True:
    data = fd.read(1024)
    if not len(data):
        break
    sys.stdout.write(data)
import sys, urllib2


URL = raw_input("Where shall we go today? ") # ex.:  http://www.google.com
req = urllib2.Request(URL)
fd = urllib2.urlopen(req)

while True:
    data = fd.read(1024)
    if not len(data):
        break
    sys.stdout.write(data)

Note that this is a really mindless program; it won't even think to attach the "http://" for you. If needed, authentication is handled by urllib2.HTTPBasicAuthHandler.

If you run it with, say, http://www.google.com, you'll get the text version of the HTML.

This is where part (3) comes in. To get a Firefox-like rendering of the web-page, your program has to replicate browser functionality. Congratulations! You get to re-write Netscape! YUCK.

Another possibility is that you could read in the HTML, tweak it (by parsing it, and then inserting stuff of your own), and then write it as a local .htm file that you could use Firefox (or IE if you must) to display. That's a do-able project, and it only requires learning to parse HTML.

Fortunately, the modules HTMLParser and htmllib are available to help you. I've never done this, so my help must stop here.

Jeff

P.S.
The other possibility would be to create your own FireFox extension to modify your browser to create the tools you need. That's probably the right solution for your problem, but I know nothing about that and can't even say whether it would be easy or hard. Here is a tutorial on writing FF extensions:

http://www.rietta.com/firefox/index.html

Last edited by jrcagle; Aug 8th, 2007 at 10:05 pm.

•

Join Date: Jul 2006

Posts: 23

Reputation: N317V is an unknown quantity at this point

Solved Threads: 3

N317V

Offline

Newbie Poster

Re: Python doubt

Aug 9th, 2007

Did You consider writing a Greasemonkey script? That is a rather simple way so enhance a webpages functionality.

•

Join Date: Dec 2006

Posts: 1,008

Reputation: woooee is a jewel in the rough

Solved Threads: 285

woooee

Offline

Veteran Poster

Re: Python doubt

Aug 10th, 2007

I was just thinking about this myself and came up with ClientForm, but haven't tried it yet.
http://wwwsearch.sourceforge.net/ClientForm/

•

Join Date: Dec 2006

Posts: 1,008

Reputation: woooee is a jewel in the rough

Solved Threads: 285

woooee

Offline

Veteran Poster

Re: Python doubt

Aug 10th, 2007

I was just thinking about this myself and came up with ClientForm, but haven't tried it yet.
http://wwwsearch.sourceforge.net/ClientForm/

•

Join Date: Aug 2007

Posts: 2

Reputation: galbi is an unknown quantity at this point

Solved Threads: 0

galbi

Offline

Newbie Poster

Re: Python doubt

Aug 12th, 2007

Thanks everybody for your answers.
Jeff: Those are really good news, :-) I'm not crazy for for something like this. I´ve tried your code and worked fine.
N317V: Great alternative!!! I like the idea, instead of python I've to learn javascript... pitty, I liked Python.
Woooee: A bit too much for me yet.

Thanks again,

G.E.A.