Member Avatar for dstoltz

I have a webpage in ASP, that scraps the content from another website using:

Set http = Server.CreateObject("MSXML2.ServerXMLHTTP.3.0")
http.open "GET", "http://www.thesite.com/thepage.aspx", true
http.send
getURL=http.responsetext

I then examine the content of "getURL" for certain text....

The problem is, that website has changed from static HTML, to using jQuery - which probably triggers another file to produce some content, etc....so if you do a VIEW SOURCE CODE in the browser on that site, you only see the jQuery, not the actual text/html you see in the browser, so this renders my program useless.

I understand this is because when you download a webpage, the browser display the source before the browser contructs the DOM. You can view the DOM with plugins like firebug (firefox), web inspector (safari), developer tools (ie), etc...

But can you do this programmatically?????

Thanks for any comments.

Edited by dstoltz
  • 2 Contributors
  • 1 Reply
  • 231 Views
  • 12 Hours Discussion Span
  • Latest Post Latest Post by LastMitch
Member Avatar for LastMitch

The problem is, that website has changed from static HTML, to using jQuery - which probably triggers another file to produce some content, etc....so if you do a VIEW SOURCE CODE in the browser on that site, you only see the jQuery, not the actual text/html you see in the browser, so this renders my program useless.

Can you find this:

Set http = Server.CreateObject("MSXML2.ServerXMLHTTP.3.0")
http.open "GET", "http://www.thesite.com/thepage.aspx", true
http.send
getURL=http.responsetext

on this website:

http://juiceui.com/

Let know if you can.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.