Page source to string (dll)

[Suzie999]

Hi

I have a function I use in a win32 dll which downloads the source of a web page to file.

The primary windows function used in it is "URLDownloadToFile" in "urlmon.h"

But I would like to have a function that downloads directly to a string or char array.

I don't have much of an idea as to where to start though, and hoping someone might
direct me to a starting point.

Appreciate any advice.

[Suzie999]

Sorry to late to edit.

maybe a better word would be to 'Read' the websource to a variable.

It is passed then to my application for parsing.

[thines01]

Do you want ALL of the source in a string?
Is there any criteria on what is kept or ignored?

[thines01]

Here is a technique:

#include "stdafx.h"

using namespace System;
using namespace System::IO;
using namespace System::Net;

int main(array<System::String ^> ^args)
{
   WebClient^ wc = gcnew WebClient();

   try
   {
      StreamReader^ fileIn = gcnew StreamReader(wc->OpenRead("http://www.yahoo.com"));
      String^ strData = fileIn->ReadToEnd(); //entire page in one string
      fileIn->Close();
   }
   catch(Exception^ exc)
   {
      Console::WriteLine("Could not process page: " + exc->Message);
   }

   return 0;
}

[Suzie999]

@thines01

Yes, I'm after the whole page, my app will later do the parsing of it.

[Suzie999]

I'm having trouble understanding what is going on there.

Please excuse my noobness, I have only ever used C++ procedural
Is that OO syntax? as it looks similar to a C# method I have for
same purpose.

If I try your code in a win32 console app, I get all kinds of errors as
though I'm missing includes or something.

[BobS0327]

Why not open a socket to the web page , send a HTTP GET and use the recv function to receive the data from the web page and load the received data into a string?

[thines01]

It's a CLR console app.

[Suzie999]

Thank you thines01

I really appreciate your help.
I had a few issues to get that code to compile in VS 2010
but it works great, although I'm still trying to work my way through
understanding it.

Am I correct thinking it is using .Net?

Anyone know io a way I can get this to compile to a Dll library?

@BobS0327 Thanks for suggestion I will look into that.

[thines01]

Yes, it's using .Net.
To make it a class library (dll), you really should create the project as a "class library" project.

Other than that, the next easiest thing to do is to ADD another project to your workspace that is a "class library" and then Add a Reference from the original project to the new project.

THEN
Create a class in the new project's .h file that looks like this:

// WebAsString.h

#pragma once
using namespace System;
namespace WebAsString {
public ref class CWebAsString
{
public:
   static bool GetPageAsString(String^ strPage, String^% strPageContent, String^% strError);
};
}

...and a .CPP file that looks like this:

// This is the main DLL file.
#include "stdafx.h"
#include "WebAsString.h"

using namespace System;
using namespace System::IO;
using namespace System::Net;
namespace WebAsString {
bool CWebAsString::GetPageAsString(String^ strPage, String^% strPageContent, String^% strError)
{
   bool blnRetVal = true;
   WebClient^ wc = gcnew WebClient();

   try
   {
      StreamReader^ fileIn = gcnew StreamReader(wc->OpenRead(strPage));
      strPageContent = fileIn->ReadToEnd(); //entire page in one string
      fileIn->Close();
   }
   catch(Exception^ exc)
   {
      blnRetVal = false;
      strError = exc->Message;
   }

   return blnRetVal;
}
}

And it now can be called from main() (in your ORIGINAL project) like this:

#include "stdafx.h"
using namespace System;
using namespace System::IO;
using namespace System::Net;
using namespace WebAsString;
int main(array<System::String ^> ^args)
{
   String^ strError = "";
   String^ strPageContent = "";
   if(!CWebAsString::GetPageAsString("http://www.yahoo.com", strPageContent, strError))
   {
      Console::WriteLine("Could not load page: " + strError);
      return -1;
   }
  
   Console::WriteLine(strPageContent);
   return 0;
}

[Suzie999]

thines01

That is much more than I would have expected
I am very grateful for your time and thank you
so much.

[thines01]

Did it work?