954,176 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Have something to say? Contribute New Article Reply to this Article

Import html docs from other server

I've made a web-crawler in VB6. I'd like to port it to .NET so I can make it accesible to the public over the web.

Basically my own version of Poodle.

Porting most of the code will be very easy, BUT, I haven't found any way via .NET to read a file 'text,html,etc..' over the web on another server.

Is this possible?

Thx

jerMAn
Light Poster
28 posts since Jun 2005
Reputation Points: 10
Solved Threads: 0
 

You cannot direclty read a file on another server. You can browser to it, though. What you need to do is called "screen scraping", I think. You might start a thread or do some Google searches on that topic.

tgreer
Made Her Cry
Team Colleague
2,118 posts since Dec 2004
Reputation Points: 227
Solved Threads: 37
 

Found a component AspTear ( works great )

Thx

jerMAn
Light Poster
28 posts since Jun 2005
Reputation Points: 10
Solved Threads: 0
 

Hmmm.. Looks like ASP.net has a built in component for scrapping pages.

Anyone interested...

jerMAn
Light Poster
28 posts since Jun 2005
Reputation Points: 10
Solved Threads: 0
 

My crawler works pretty good
http://www.proaliusweb.com/WebForm1.aspx

Thanks for the help.

jerMAn
Light Poster
28 posts since Jun 2005
Reputation Points: 10
Solved Threads: 0
 

This article has been dead for over three months

Post: Markdown Syntax: Formatting Help
You