I need to log in to a website using python. The two fields are username and password. This is what I have tried:

import urllib
import urllib2
import httplib

username = "username"
password = "password"

url = "https://webconnect.bloomfield.org/Zangle/StudentConnect/logincheck.aspx"

password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_mgr.add_password(None, "https://webconnect.bloomfield.org/Zangle/StudentConnect/", username, password)
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(handler)
req = urllib2.Request(url)
f = opener.open(req)
print f.read()

This only returns the html code of the page with the login form, meaning the login wasn't successful. Of course when I tried, I used a valid username and password. While you can't test it out on this site, is there anything you see that is obviously the problem. I'm not very good with programming with websites, I figured out that much from tutorials...

Re: Submitting data to a form 80 80

Mechanize is an easier way IMHO to do this. This link explains how to log in with a user name and password to a remote site

Re: Submitting data to a form 80 80

Thanks! That is very useful. I've spent so long trying to get other random code to work, but that was by far the easiest solution. I have one more question though. Now that I'm logged in, I need to navigate to a different page that I can only access once I'm logged in. How do I go to that page? br.open('http://websitename.com/page.aspx')? When I tried that it said my session had expired and that I had to log back in. How can I get to that page without it kicking me out? Typing that page name in the browser after I'm logged in works fine, btw. Here's the code I'm using:

import mechanize
import cookielib

# Browser
br = mechanize.Browser()

# Cookie Jar
cj = cookielib.LWPCookieJar()

# Browser options

# Follows refresh 0 but not hangs on refresh > 0
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)

# User-Agent (this is cheating, ok?)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]

# Open some site, let's pick a random one, the first that pops in mind:
r = br.open('https://webconnect.bloomfield.org/Zangle/StudentConnect/')
#r = br.open('http://google.com')

# Show the available forms
#for f in br.forms():
#    print f

br.open('https://webconnect.bloomfield.org/Zangle/StudentConnect/stuassignments.aspx') #This is where I try to open that page... It doesn't work.
print br.response().read()
Re: Submitting data to a form 80 80

Here's an update. I tried the code on facebook, and it logged in successfuly, but it wasn't able to go to another page when I used br.open(). When I switched it to br.follow_link() it worked fine. The problem is I'm not always going to have a link on the page I want to go to, like in my original example in the code posted above. I need to be able to directly go to that url...

Re: Submitting data to a form 80 80

I think the problem in the original code is you didn't handle any cookies, or change your useragent. Take a look at this.

Facebook really doesn't like datamining on there servers.
I'm not saying that's what your doing; just giving a heads up.

commented: good point +10
Re: Submitting data to a form 80 80

It does seem like a cookie problem, but according to http://wwwsearch.sourceforge.net/mechanize/ cookies are automatically handled. And the code you posted doesn't work for me, I tried it a while ago. The mechanize code at least logs me in, its just going to another page that's the problem.

Re: Submitting data to a form 80 80


Re: Submitting data to a form 80 80

You must be doing something wrong then. I can log into all of the sites I'm a member at via python; including here at daniweb.

Post the source of the page your trying to log in to.

I tried to visit https://webconnect.bloomfield.org/Zangle/StudentConnect/logincheck.aspx
they both aren't coming up.

Re: Submitting data to a form 80 80

Could you post the exact code you used to log in to those websites? And after you log in, could you switch to a different page on that website and print the source? I hate asking for code, but this just isn't working for me...

Re: Submitting data to a form 80 80

I don't use mechinize, this is done with urllib, urllib2, cookielib.
This is a very basic example and should get you started. I used this exact code
to log into www.hellboundhackers.org and complete the programming challenges, so I
know for a fact this code works.
You'll have to change to url's to the site you want to log into, and find the login_data
in the source of the page your trying to log into. More detailed example in the code.

If your still having trouble, post the source of the page so I can look at it.

import urllib, urllib2, cookielib

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.addheaders.append(('User-agent', 'Mozilla/4.0'))
opener.addheaders.append( ('Referer', 'http://www.urltosite.com/index.html') )

#You will need to look at the source of the page your
#trying to log into and find the name="" for the username and password
#<input type='text' name='user_name' class='textbox' style='width:100px' /><br />
#<input type='password' name='user_pass' class='textbox' style='width:100px' /><br />
#The login button looks like this:
#<input type='submit' name='login' value='Login' class='button' /><br />
login_data = urllib.urlencode({'user_name' : 'Name',
'user_pass' : 'Pass',
'login' : 'Login'

#This is where you are logged in
resp = opener.open('http://www.urltosite.com/index.html', login_data)

#Now you can go to member only pages
resp = opener.open('http://www.urltosite.com/Member_only_index.html')
d = resp.read() #The source to the member only page

#Create a html page to save the source to
#so we can see if it worked
f = open("test.html","w")

#close file and page handles
Re: Submitting data to a form 80 80

I tried this with facebook and demonoid, and it when I open up test.html it tells me to login Here's the facebook login source:
<input type="text" class="inputtext" name="email" id="email" value="" tabindex="1" /></td>
<td><input type="password" class="inputtext" name="pass" id="pass" tabindex="2" /></td>
<td><label class="uiButton uiButtonConfirm uiButtonMedium"><input value="Login" tabindex="4" type="submit" />

I'm hoping you have a facebook account so you can try it yourself... Thanks.

Re: Submitting data to a form 80 80

I do have a face book account and have been trying to log in with python with no success yet. I did discover that the url we need to be trying to log into is http://www.facebook.com/login.php

Re: Submitting data to a form 80 80

You need a diferent aproach because the login page it's on php with a javascript login.

Re: Submitting data to a form 80 80

It being a php page has nothing to do with it, the javascript is whats holding it back.

I got the login name working by using url arguments i.e. facebook.com/login.php?name=my_login_name&pass=my_pass

The pass didn't work though.

Re: Submitting data to a form 80 80

It's probably not working because you are not supplying all the required arguments. Sadly, it looks like Facebook wants quite a few arguments to log in such as authentication token and api key, but the Facebook devs have provided some information on all of it: http://wiki.developers.facebook.com/index.php/Authorization_and_Authentication_for_Desktop_Applications.

Good luck.

Re: Submitting data to a form 80 80

Why do they have to make everything so difficult.....

Re: Submitting data to a form 80 80

Why do they have to make everything so difficult.....

Well, I have some marginally good news. Some users with the same problem as you created a library to hopefully make the connection process easy. Pyfacebook's home: link. This is the current home: do not use the Google code page, as it is supposedly outdated. You can find examples in the "examples" directory on github (link) or from the facebook developer's area (link).

Hopefully that will make things easier.

Re: Submitting data to a form 80 80

I will take a look at Pyfacebook and see if I can get it too work. Also, while facebook was my original intention for my program, I want to extend it to other sites. The other site I was trying was in my first few posts. It's a school site, so you wont be able to test anything with it, but just like facebook I wasn't able to log in. What are some reasons this could be happening? Tech B, you mentioned javascript earlier....

Re: Submitting data to a form 80 80

I wasn't able to figure it out yet. I'd rather figure out how to do a simple connection without external libraries than waste time learning how to use this library anyway... thanks for the suggestion though.

Tech B, did you read my last question?

Re: Submitting data to a form 80 80

Sorry, my internet has been having some issues.

You will need to figure out what; and how many arguments to send in the url to submit the data. Go over the source of the log in page and see what else can be passed.

Re: Submitting data to a form 80 80

I was just about to post here. I just found some code online that I modified a little and it works now, but only for facebook mobile. That will do for now, but it still bugs me not knowing why I can't login to the normal facebook. And I still have to figure out how to log in to those other sites, but at least this worked. Here's the code if you want to see it. Thanks for everything.

import urllib2                                                   

email = "email here"                                                            
password = "password here"     

opener = urllib2.build_opener (urllib2.HTTPCookieProcessor ())                 
urllib2.install_opener (opener)                                                
loginRequest = urllib2.Request ('https://login.facebook.com/login.php?m&amp;nex\t=http%3A%2F%2Fm.facebook.com%2Fhome.php','email=%s&pass=%s&login=Login' % (email, password))                        

urllib2.urlopen (loginRequest)                                                 
connection = urllib2.urlopen ('http://m.facebook.com/')                        

f = open("test.html","w")

BTW, does anyone know how this person figured out that url? https://login.facebook.com/login.php?m&amp;nex\t=http%3A%2F%2Fm.facebook.com%2Fhome.php? Knowing how they got it will help me find the correct url for other sites.

Re: Submitting data to a form 80 80

He is usinf the mobile facebook address because it asks less information to login, like authentication token and api key.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of 1.19 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.