Hi,

I am trying to read first 6000 bytes from a webpage, using StreamReader.Read(buffer, offset, no_of_bytes_to_read) method and trying to close the connection. Because, the useful data I need is present in the first 6000 bytes of the webpage and webpage size is atleast 100Kb. And, I need to repeat this operation for around 100s of such pages. Thats why I would close the connection after I am done reading initial 6000 bytes.

Now the problem is, I noticed that when code fires streamRead.Close(), it actually reads those remaining 94Kb of Data and then closes the connection !! This consumes extra 8 to 10 seconds just for closing the reader.(Verified via network transfer logs and the debug-time timestamps before and after this streamRead.Close() statement.)

Isn't this strange that why the reader should read the remaining bytes when I am actually telling it to close the connection and quit. Maybe I am doing it incorrectly, I tried different methods but none working. Is there other to achieve what I just need?

byte[] bytes = new byte[6000];

                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(query);
                HttpWebResponse resp = (HttpWebResponse)request.GetResponse();
                Stream reader = resp.GetResponseStream(); 

                int bytes_read = 0;
                int count;
                int bytes_to_read = 6000;

                while (bytes_read < 6000)
                {
                    count = reader.Read(bytes, bytes_read, bytes_to_read);
                    MessageBox.Show(System.Text.ASCIIEncoding.ASCII.GetString(bytes));
                    bytes_read += count;
                    bytes_to_read -= count;
                }

                reader.Close(); /// <-- This Line takes 8 to 10 seconds to finish its work because it reads remaining page from server and then closes
                resp.Close();
                string response = System.Text.ASCIIEncoding.ASCII.GetString(bytes);

Hmm, it didnt do that for me, and the file I picked (like a moron) was huge.

Only thing I could think of which may help would be

reader.Seek(resp.ContentLength, 0);
reader.Close();

if (!reader.close())
{
reader.Close();
}
else
{}

application.DoEvents();

Which version of .net are you working in? mine didnt complain, as well as mine didnt suffer the same as yours.

And no, the close was immediate for me even with your URL

System.Diagnostics.Debug.Print("Before close: " + DateTime.Now.ToString());
reader.Close();
System.Diagnostics.Debug.Print("After close: " + DateTime.Now.ToString());

---------------------------------------------
Before close: 19/01/2009 4:32:41 PM
After close: 19/01/2009 4:32:53 PM

are you using exact same code which I posted in first post?

Now I am wondering what ghost has got into my pc. Are you using VS2005 or 2008? Mine is 2008.

I used your exact code, replacing query with the URL string.
Im using 2008 as well.

Ok, I tried other servers and above code works fine. But problem persists with the server url pasted above.

So, its something to do with response headers that application receives from the server.

I changed the MessageBox output to a System.Diagnostics.Debug.Print so that it wouldn't wait for me to click anything (possibly still reading in the background).

VC# Express claimed it was making a project for .Net 3.5 (I made a console application, but did not output to the console.)

This load line from the debug seems to agree with the 3.5:

GAC_MSIL\System.Core\3.5.0.0__

This is my output:

Before close: 1/19/2009 8:44:57 AM
After close: 1/19/2009 8:44:57 AM
The thread 0x1574 has exited with code 0 (0x0).
The thread 0x1908 has exited with code 0 (0x0).

LizR, its windows xp SP2.

Just curious,.. how come this matters? Isn't .Net 2.0 library same for each windows OS?

Ah, thanks Murtan, I get a similar output with delay of 1 second for reader.Close() after I read 200 bytes from www.google.com and it works fine(closes without reading further).

I will try testing with some different machine or maybe Useragents tomorrow with the problematic "query" value and post update.

I just thought you might like another sample. Those results were from the 'problem' link you posted in message 5

LizR, its windows xp SP2.

Just curious,.. how come this matters? Isn't .Net 2.0 library same for each windows OS?

.net is, but I was wondering if you had vista more to the point, as Im using XP SP2 too.. I was just wondering if something else was causing a difference.

No, but it is good that it appears to be at least in some part environmental for tovishal2001

This article has been dead for over six months. Start a new discussion instead.