web crawler

[amvx86]

Hello everyone,

Maybe someone out there can help me. I have this code that I am using and for the life of me it keeps throwing an error:

        If Len(ComboBox1.Text) < 1 Then
            MsgBox("You need to specify a target for this to operate properly.", vbExclamation, "Error")
            Exit Sub
        End If

        GetCleanURL()

        WebBrowser1.Navigate(Server_Addy_RAW)



        Dim links As HtmlElementCollection = WebBrowser1.Document.Links
        For Each link As HtmlElement In links
            Linkbox.Items.Add(link.GetAttribute("href"))
        Next

the point of the code is to navigate to a web site and then pull the links and add them to a list box. When that is done it keeps crawling the web location. However, the error I keep getting is this:

Object reference not set to an instance of an object. and to use the new keyword. When I remove the webbrowser1.navigate it will bomb unless the web browser control navigates to the page befre and then the crawler is invoked. Any ideas?

./x86

[tinstaafl]

How are you declaring Webbrowser1?

[amvx86]

Tinstaafl,

I'm using it as is as one normally would. webbrowser1.navigate(website) and then once it navigates then we call the crawl. I should actually attempt to wait for the page to load and try that again. Let me see how this works and I will get back to you. I may have answered my own question.

[amvx86]

OKay I got it working, you have to add this:

                Do Until WebBrowser1.ReadyState = WebBrowserReadyState.Complete
                    Application.DoEvents()
                Loop

before you can crawl the site. Sorry for the post.