Custom Proxy Server

[dmaclam]

So I am currently writing a custom proxy server to handle http requests on a local computer. I am able to get the headers but am not sure how to then send them to the browser. here is my current source code:

[

Imports System.Net
Imports System.Net.Sockets
Imports System.Text
Imports System.Threading
Imports System.Net.HttpWebRequest

Public Class ProxyServer
    Private tcpListener As New TcpListener(IPAddress.Any, 8080)
    Private listener As New Thread(AddressOf Listen)

    Public Sub Start()
        tcpListener.Start()
        listener.IsBackground = True
        listener.Start()
    End Sub

    Private Sub Listen()
        Dim tcpClient As TcpClient

        While True 'Loop And Wait For Connections
            Try
                'Accept Incomming Connection
                tcpClient = tcpListener.AcceptTcpClient()
                Dim a As New Thread(AddressOf DoWork)
                a.Start(tcpClient)
            Catch ex As SocketException
                Exit While
            End Try
        End While
    End Sub

    Private Sub DoWork(ByVal client As TcpClient)
        Dim netStream As NetworkStream = client.GetStream
        While True
            Try
                If netStream.CanRead And netStream.CanWrite Then
                    Dim bytesToRead(client.ReceiveBufferSize) As Byte
                    Dim numBytesRead As Integer = netStream.Read(bytesToRead, 0, CInt(client.ReceiveBufferSize))
                    If numBytesRead < 1 Then Exit While
                    Dim data As String = Encoding.ASCII.GetString(bytesToRead)

                    If data.IndexOf(vbNewLine) > -1 Then 'Check To See If Full Command Recieved
                        'data = this is the whole header
                    End If
                Else
                    Exit While
                End If
            Catch ex As IO.IOException
                Exit While
            End Try

        End While
    End Sub
End Class

If i configure firefox or IE to the proxy server I do receive the header no problem.

Sample Header
www.google.com
GET http://google.ca/ HTTP/1.1 Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/x-gsarcade-launch, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-silverlight, */* Accept-Language: en-us User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; InfoPath.2; .NET CLR 3.5.30729) Accept-Encoding: gzip, deflate Proxy-Connection: Keep-Alive Host: google.ca Cookie: PREF=ID=2cc48986d0a30551:U=4583f9a4276fb2df:FF=0:TM=1297940975:LM=1297941095:S=k0i-eurczYyMJF2q; NID=44=t7EdDRwCJHYI-iYWLo4NeRJ3DtFmv7wZ4yeUmEdemRP7X83I-kRJd6Ah3uf2rw-et_c_H5zYDd7vJuApaX0vs2K_jz30l7uQZPfmTdOeTvKpi67lXrbkiTP2DIOKeYew

My question is how do I send this back to the browser and display the page?

Thanks In Advance.
-Dan

[dmaclam]

So I started over just to see what I could figure out and it turns out I still cannot get the page whole page display in the browser. (Browser just shows text based items for a couple sites. No images. Other sites just hang and display nothing.)

I found some of the code online and some is my own. Its console application if you want to try it out.

Imports System
Imports System.IO
Imports System.Net
Imports System.Net.Sockets
Imports System.Threading

Class WebProxy
    Private clientSocket As Socket
    Private read() As [Byte] = New Byte(1023) {}
    Private Buffer As [Byte]() = Nothing
    Private ASCII As Text.Encoding = Text.Encoding.ASCII
    Private Const HTTP_VERSION As String = "HTTP/1.0"
    Private Const CRLF As String = vbCr + vbLf
    Private RecvBytes(4095) As [Byte]
    Const port As Integer = 8080

    Shared Sub Main(ByVal args() As String)
        Dim tcplistener As New TcpListener(port)
        tcplistener.Start()
        Console.WriteLine("Listening on port {0}", +port)

        WriteToLog("log.txt", "Listening on port " & port)

        While True
            Dim socket As Socket = tcplistener.AcceptSocket()
            Dim webproxy As New WebProxy(socket)
            Dim thread As New Thread(New ThreadStart(AddressOf webproxy.run))
            thread.Start()
        End While
    End Sub 'Main

    Public Sub New(ByVal socket As Socket)
        Me.clientSocket = socket
    End Sub 'New

    Public Sub run()
        Dim clientmessage As String = " "
        Dim sURL As String = " "
        Dim bytes As Integer = readmessage(read, clientSocket, clientmessage)

        If bytes = 0 Then
            Return
        End If

        Dim index1 As Integer = clientmessage.IndexOf(" "c)
        Dim index2 As Integer = clientmessage.IndexOf(" "c, index1 + 1)
        If index1 = -1 OrElse index2 = -1 Then
            Throw New IOException()
        End If

        Console.WriteLine("Connecting to Site: {0}", clientmessage.Substring(index1 + 1, index2 - index1))
        Console.WriteLine("Connection from {0}", clientSocket.RemoteEndPoint)
        WriteToLog("log.txt", DirectCast("Connecting to Site: {0}" & clientmessage.Substring(index1 + 1, index2 - index1), String))
        WriteToLog("log.txt", "Connection from : {0}")


        Dim part1 As String = clientmessage.Substring(index1 + 1, index2 - index1)
        Dim index3 As Integer = part1.IndexOf("/"c, index1 + 8)
        Dim index4 As Integer = part1.IndexOf(" "c, index1 + 8)
        Dim index5 As Integer = index4 - index3
        sURL = part1.Substring(index1 + 4, part1.Length - index5 - 8)

        Try
            Dim IPHost As IPHostEntry = Dns.Resolve(sURL)
            Console.WriteLine("Request resolved: ", IPHost.HostName)
            WriteToLog("log.txt", "Request resolved: " & IPHost.HostName.ToString)

            Dim aliases As String() = IPHost.Aliases
            Dim address As IPAddress() = IPHost.AddressList
            Console.WriteLine(address(0))
            WriteToLog("log.txt", address(0).ToString)

            Dim sEndpoint As New IPEndPoint(address(0), 80)
            Dim IPsocket As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
            IPsocket.Connect(sEndpoint)
            If IPsocket.Connected Then
                Console.WriteLine("Socket connect OK")
                WriteToLog("log.txt", "Socket connect OK")
            End If

            Dim [GET] As String = clientmessage
            Dim ByteGet As [Byte]() = ASCII.GetBytes([GET])
            IPsocket.Send(ByteGet, ByteGet.Length, 0)
            Dim rBytes As Int32 = IPsocket.Receive(RecvBytes, RecvBytes.Length, 0)
            Console.WriteLine("Recieved {0}", +rBytes)
            WriteToLog("log.txt", "Recieved {0}" & CStr(+rBytes))

            'Buffer = RecvBytes;
            Dim strRetPage As String = Nothing
            strRetPage = strRetPage + ASCII.GetString(RecvBytes, 0, rBytes)
            While rBytes > 0
                rBytes = IPsocket.Receive(RecvBytes, RecvBytes.Length, 0)
                strRetPage = strRetPage + ASCII.GetString(RecvBytes, 0, rBytes) '****THIS SHOULD RETURN THE PAGE IF IM NOT MISTAKEN*********
            End While

            IPsocket.Shutdown(SocketShutdown.Both)
            IPsocket.Close()
            sendmessage(clientSocket, strRetPage) 'Then we actualy send it...?

        Catch exc2 As Exception
            Console.WriteLine(exc2.ToString())
            WriteToLog("log.txt", exc2.ToString)
        End Try
    End Sub 'run

    Private Function readmessage(ByVal ByteArray() As Byte, ByRef s As Socket, ByRef clientmessage As String) As Integer
        Dim bytes As Integer = s.Receive(ByteArray, 1024, 0)
        Dim messagefromclient As String = Text.Encoding.ASCII.GetString(ByteArray)
        clientmessage = CStr(messagefromclient)
        Return bytes
    End Function 'readmessage

    Private Sub sendmessage(ByVal s As Socket, ByVal message As String)
        Buffer = New [Byte](message.Length + 1) {}
        Dim length As Integer = ASCII.GetBytes(message, 0, message.Length, Buffer, 0)
        s.Send(Buffer, length, 0)
    End Sub 'sendmessage

End Class 'WebProxy

[dmaclam]

UPDATE:

Firefox is now loading basic pages fine (Pages without any passed variables.)
IE is displaying just the text portion of the sites no images.

Having problems still with most sites that load things dynamically. No search engines load, nor this site.

Content Encoding Error
The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.