HTTP response problem

[unclepauly]

im developing an http proxy in C#. my problem is reproducable outside of this language so if you can tell me of a better forum then that would be good, as i am stumped.

to cut a long story short:

sometimes, the response i get back from the server is g-zipped - that is, one header is "Content-Encoding: gzip". but the actual bytes that i get back are not correct. as an example, take the following. this can be reproduced using telnet, so its not my code thats wrong.

here is the header :

GET /www.new/app_full_proxy.php?app=2462715666&v=1&size=p&cksum=df4b829ed8cd6f0b6ee9cdea66632753&src=http%3A%2F%2Fsol.ltserv.net%2Ffbsms%2Ftheme%2Fdefault%2F HTTP/1.1
Accept: */*
Referer: http://www.facebook.com/home.php
Accept-Language: en-gb
UA-CPU: x86
Accept-Encoding: gzip
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618)
Proxy-Connection: Keep-Alive
Host: platform.ak.facebook.com
Cookie: [cookie details go here !]

and here is the response:

HTTP/1.1 200 OK
Server: Apache/1.3.41.fb2
X-Cnection: close
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Content-Length: 20
Vary: Accept-Encoding
Date: Tue, 19 May 2009 10:34:43 GMT
Connection: keep-alive

? [the 20 bytes listed above]

as you can see this is just my facebook home-page requesting something. i've taken out the cookie details. the response i get back says the content length is 20 bytes long, and here they are:

0x1f
0x8b
0x08
0x00
0x00
0x00
0x00
0x00
0x00
0x03
0x03
0x00
0x00
0x00
0x00
0x00
0x00
0x00
0x00
0x00

bytes 0, 1 and 2 are correct - 0x1f and 0x8b are the 'magic numbers' for GZip, so this is definitely 'supposed to be' GZip data. then the 0x08 byte is to say use the 'deflate' compression, which is also correct. then the remaining bytes are just wrong - all null bytes with a couple of 3's thrown in.

if you create a sample C# Gzip application, and try to unzip the above bytes, you won't get an error, or an execption, you will just see the GZipStream.Read function return 0 bytes forever - and here lies my problem.

all i am doing is unzipping the data that the server is sending back. so why is the server sending back this response if it blatently cant be unzipped ?

[celmor]

Hi:

I have reproduce your request. And I could see that the problem is not the Gzip stuff. The problem is the Response Headers. It seems a hack trick. They're trying to held open the tcp connection.

Can you see the field Connection and X-Cnection? There is the problem.

I suposse, that they're trying to get the account or the password using the cookies, sending them to their servers.

You can read more information at:

http://www.nextthing.org/archives/2005/08/07/fun-with-http-headers

[unclepauly]

thank you very much for your reply.

ive read the bit on x-cnection and i have to be honest, i still dont fully understand whats going on. if they are trying to keep the connection open then why also send a load of unzippable gzip bytes ? why go to the bother of sending any body at all ?

thanks anyway.

[celmor]

You're right. But I've no idea.

I've looking for more information, and I've just seen that these responses are made from a facebook application which sends sms. Facebook hasn't made this app. And it's possible that developers had had some problems or they continue having problems sometimes.

As I've read, some facebook app developers use X-Cnection header because they have to intercept the responses from facebook servers. And they need some information, as obvious.

Uncle, can you help me, or try it, with ssl proxy connections? I send you a private message some days ago, I'm having problems when I've to navigate through https pages. I recieve CONNECT package, then I send HTTP 1.0 200 OK \r\n\r\n. I create a sslsocket and the handshake listener is taken (I'm using Java), when I accept the exception and get my own certificate by browser. When a Sniffer I see that after I do this (only in that moment) the browser send me a request for a https page (if I'm trying in gmail, then this https request is for /accounts....) but my program (who is waiting for a request, don't recieve it). Do you know what I've to do between I recieve the connect and the moment I could navigate through the https page?

We could speak slowly after this if you can.

See you.

[unclepauly]

ive replied to your private message - thanks for your help.