Can we crawl password protected website?

[gunbuster363]

I know the password.

I want to crawl all the post of my gf's xanga down to my comp, for better reading.
But my gf have set "require friend login" to see her post.
So I need to login my id of xanga before reading her post.

What I want to ask is,
can we write a program that know how to get pass the password and crawl the webpage?
I know the password, because that is my id.

Thanks All.

[gunbuster363]

the login webpage, for example, is like this:
http://gunbuster363.xanga.com/XangaL...chives/2004/12

after entering the password and login, the page is :
http://gunbuster363.xanga.com/archives/2004/12/

[masijade]

HttpUrlConnection or HttpsUrlConnection. It is possible to use Cookies and post requests and authentication with both of them. Search for some tutorials concerning them.

[gunbuster363]

I think the sign in button is in javascript
I am stucked in here

<a id="signin" href="javascript: SigninSubmit();" tabindex="3">Sign In</a>

[masijade]

So look at what that function does.

It's probably no more than a post request. And, if you don't know what I mean by that, then you need to study HTML and Http in general before you continue here.