Asynchronous transactions with xml https post

[lazynewt]

Hi guys i am wondering if anyone can help.

I basically have a few thousand transactions 7,000 - 10,000 transactions to process over a short time. The flow goes like this.

<Select single record> ---> <convert to xml doc> ----> <transmit document as string to https:// location> ---> <receive a response as xml string> ---> <save response to sql)

Now i understand as i will be looking to process a few thousand at a time i need to communicate with the site asynchronously (send 1000) wait... > receive (1000) as each transaction takes approx 2 seconds so if i were to process synchronously it would take all night......

Now first i would like to ask if i am going about this the right way ?
the service i am communicating with is not .net but is built to handle large amounts of requests.

I have obtained the following code that suggests it will do what i am after

dim strxmlreq as string = '(this is where i input my xml string)
 Dim request As HttpWebRequest

        ' Create the request
        request = CType(WebRequest.Create(My.Settings.MPIURL), HttpWebRequest)
        request.Method = "POST"
        ' Convert the xml doc string into a byte array
        Dim bytes As Byte()
        bytes = System.Text.Encoding.Unicode.GetBytes(strxmlreq)

        ' Assign the content length
        request.ContentLength = bytes.Length

        ' Write the xml doc bytes to request stream
        request.GetRequestStream.Write(bytes, 0, bytes.Length)

        Dim result As IAsyncResult
        Dim state As WebRequestState
        Dim timeout As Integer

        ' Create the state object used to access the web request
        state = New WebRequestState(request)

        ' Begin the async request
  result = request.BeginGetResponse(New AsyncCallback(AddressOf RequestComplete), state)
        ' Set timeout at 1 minute
        timeout = 1000 * 60

        ' Register a timeout for the async request
        ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle, New WaitOrTimerCallback(AddressOf TimeoutCallback), state, timeout, True)

The problem i have is if the above will work for me where exactly do i catch the returned xml string ? I assumed "result" would be the xml output string.

If anyone has any ideas or can reccomend a better way to process these transactions i would really appreciate it.

Thanks.

[GeekByChoiCe]

think that might help --> http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse.aspx

[lazynewt]

Thanks for the pointer but im still very confused. Is this solution going to work fast enough and can anyone dum this down a bit for me :).

Thanks.

[lazynewt]

So i now have a procedure that will post asynchronously using threadpool. The module is below.

Imports System.Net
Imports System.IO
Imports System.Threading
Imports System.Collections

Module Module1
   

    Public Class RequestState
        Public Request As HttpWebRequest
        Public URL As String
        Public Sub New()
            Request = Nothing
        End Sub
    End Class

    Public Sub CreateAsyncWebRequest(ByVal MyURL As String, ByVal XMLInputString As String)
        Dim oHttp As HttpWebRequest = DirectCast(WebRequest.Create(MyURL), HttpWebRequest)
        oHttp.Method = "POST"
        Dim postBuffer As Byte() = System.Text.Encoding.ASCII.GetBytes(XMLInputString)
        oHttp.ContentLength = postBuffer.Length
        Dim postData As Stream = oHttp.GetRequestStream()
        postData.Write(postBuffer, 0, postBuffer.Length)
        postData.Close()
        Dim objState As New RequestState
        objState.Request = oHttp
        objState.URL = MyURL
        Dim result As IAsyncResult = DirectCast(oHttp.BeginGetResponse(New AsyncCallback(AddressOf ProcessAsyncWebResponse), objState), IAsyncResult)
        ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle, _
                 New WaitOrTimerCallback(AddressOf ScanTimeoutCallback), _
                 objState, (30 * 1000), True)
    End Sub

    Private Sub ScanTimeoutCallback(ByVal Request As Object, ByVal TimedOut As Boolean)
        If TimedOut Then
            Dim objState As RequestState = DirectCast(Request, RequestState)
            If Not (Request Is Nothing) Then
                objState.Request.Abort()
                Console.WriteLine(objState.URL + " is processed")
            End If
        End If
    End Sub

    Private Sub ProcessAsyncWebResponse(ByVal Result As IAsyncResult)
        Dim objState As RequestState = DirectCast(Result.AsyncState, RequestState)
        Dim oHttp As HttpWebRequest = DirectCast(objState.Request, HttpWebRequest)
        Dim myResponse As HttpWebResponse
        Dim loResponseStream As StreamReader
        Dim strxmlout As String
        Try
            myResponse = DirectCast(oHttp.EndGetResponse(Result), HttpWebResponse)
            loResponseStream = New StreamReader(myResponse.GetResponseStream(), System.Text.Encoding.ASCII)
            strxmlout = loResponseStream.ReadToEnd()
            'rtbSendNotes.Text = m_xmlOut
            'frmMainProcess.TextBox2.Text += strxmlout
            writelog.writelogfile(objState.URL + " is processed-i win -------" + strxmlout)
        Catch ex As Exception
            'Exception handling code
            writelog.writelogfile(objState.URL + " is processed with exception: " + ex.Message)
        Finally
            If Not (myResponse Is Nothing) Then
                myResponse.Close()
            End If

            If Not (loResponseStream Is Nothing) Then
                loResponseStream.Close()
            End If
        End Try
        frmMainProcess.TextBox2.Refresh()


    End Sub


End Module

The big problem im having is that when i call the procedure using

CreateAsyncWebRequest(My.Settings.MYURL, strxmlreq)

the program freezes and waits until a response has been processed from the url. I am trying to use the threadpool to fire off a few hundred/thousand transactions at a time. Can anyone please help?

Thanks.