DaniWeb Forum Index > Software Development > C#

C# RSS

hot to get html from url requiring login

Please support our C# advertiser: $4.95 a Month - ASP.NET Web Hosting – Click Here!

•

Join Date: Nov 2007

Posts: 9

Reputation: Krimi is an unknown quantity at this point

Solved Threads: 0

Krimi

Offline

Newbie Poster

hot to get html from url requiring login

Mar 17th, 2008

hi
i need to read html from a url requiring that i log in to the site.
how is that done? i think it depends on how the site is build?
its for a torrent site, and they often use the same template as far as i know.
the site im trying to read through c# is http://nordic-t.org/browse.php

this code works fine for getting any page, that doesnt require login:

 Help with Code Tags 
 C# Syntax (Toggle Plain Text) 
            // Address of URL
            string URL = textBox1.Text;
            try
            {
                // Get HTML data
                WebClient client = new WebClient();
                Stream data = client.OpenRead(URL);
                StreamReader reader = new StreamReader(data);
                string str = "";
                str = reader.ReadLine();
 
                while (str != null)
                {
                    richTextBox1.AppendText(str);
                    str = reader.ReadLine();
                }
                data.Close();
            }
            catch (WebException exp)
            {
                MessageBox.Show(exp.Message, "Exception");
            }
            // Address of URL
            string URL = textBox1.Text;
            try
            {
                // Get HTML data
                WebClient client = new WebClient();
                Stream data = client.OpenRead(URL);
                StreamReader reader = new StreamReader(data);
                string str = "";
                str = reader.ReadLine();

                while (str != null)
                {
                    richTextBox1.AppendText(str);
                    str = reader.ReadLine();
                }
                data.Close();
            }
            catch (WebException exp)
            {
                MessageBox.Show(exp.Message, "Exception");
            }

how do i merge username and password into all this?
i searched, but didnt find anything for doing it with a page using php. maybe i searched for the wrong things.. still kinda newbie with programming

•

Join Date: Feb 2007

Posts: 1,598

Reputation: scru has a spectacular aura about

Solved Threads: 130

scru

Offline

Posting Virtuoso

Re: hot to get html from url requiring login

Mar 18th, 2008

Without even bothering to question you about your motives, I shall get right to it:

The measures you will need to take may vary from site to site, and I for one do not know which methods this site uses to verify users. But it is important to note that it is quite possible that you can't access the "normal" page without logging in, or at least convincing the server that you have. This is because the server probably doesn't bother to send you the normal page (and it shouldn't) until you are verified as a user.

I would suggest that before running the script on that page, you use the web client to log in to the site. This will set any variables that the server may use to log/verify your authentication.

Next, if that web client you are using supports cookies, I would suggest turning them on. If it doesn't, get one that does. Whenever the server asks to write to a cookie, let it. Whenever it asks for the data in a cookie it wrote to, let it (don't let it read cookies it didn't write unless your web client is encapsulated, essentially meaning separated from IE, Firefox or whatever browsers you use). The reasons cookies are important, is that many sites use them to help handle user log ins (including the ones I get to work on)

Good luck.