I have recently made a program that gets the links from a webpage, then checks to see if they are valid or not. I used the HtmlElement to get the "HREF" attribute and stored all the links into a list. I also tried using the HttpWebRequest and stored the stream into a string and parsed it out to get the links. These ways seem to work but doesn't give me all the links. I guess those are scripted.
The only way I could get all of the links was to open the page in FireFox, then view Page Source, and save it to a txt file, then use my program to read the file and save it to a string and parse the links out that way.
What I need to know is how or if this is possible to do automatically. I'm just using a webBrowser control. I know there are several built in controls that allow you to refresh a page and things, but is there a way to view and grab the page source?