Online HTML to EXCEL: BIG Problem

[ayesha789]

Dears
This is an electricity billing Site link http://www.mepco.com.pk/ls/defftree.asp
I click on each link and SELECT and COPY/PASTE , It takes 3 hours minimum each day to save all pages in Excel.
I want to get all pages automatically using PHP or Query.
Because higher management demand reports each day.

[ayesha789]

Please give me any idea....

[urtrivedi]

you need to learn curl.

[ayesha789]

THANKS

[ayesha789]

Its confusing

[ayesha789]

How to download Curl and use?

[urtrivedi]

I will write steps to follow for completing the job. You may have to learn certain things before starting from manauls and tutorials

1) You must have substation database with id and name at your server where u are going to write a script
2) write a script fetch subsation list from database and go through them one by one using loop.
3) in loop you have write open url using curl and you have to pass substation id. curl will return you the page soucre in your variable.
4) you have to extract table data and save it in some text file with appropriate file name.

[ayesha789]

any good site to learn Curl

[ayesha789]

I got this code , but how to add curl in PHP?

<?php
/**
* Initialize the cURL session
*/
$ch = curl_init();
/**
* Set the URL of the page or file to download.
*/
curl_setopt($ch, CURLOPT_URL,
'http://news.google.com/news?hl=en&topic=t&output=rss');
/**
* Create a new file
*/
$fp = fopen('rss.xml', 'w');
/**
* Ask cURL to write the contents to a file
*/
curl_setopt($ch, CURLOPT_FILE, $fp);
/**
* Execute the cURL session
*/
curl_exec ($ch);
/**
* Close cURL session and file
*/
curl_close ($ch);
fclose($fp);
?>

[ayesha789]

How I install Curl in XAMPP

Installation

To use PHP's cURL support you must also compile PHP --with-curl[=DIR] where DIR is the location of the directory containing the lib and include directories. In the include directory there should be a folder named curl which should contain the easy.h and curl.h files. There should be a file named libcurl.a located in the lib directory. Beginning with PHP 4.3.0 you can configure PHP to use cURL for URL streams --with-curlwrappers.

[urtrivedi]

you need to enable php_curl.dll extention

[urtrivedi]

WITHOUT CURL

$handle = fopen("http://www.mepco.com.pk/ls/ConsumerLoadShedding.asp?SDiv=5124&off=Shah Rukn-e-Alam Sub Division", "rb");
$contents = stream_get_contents($handle);
fclose($handle);
echo $contents;

or WITH CURL

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, "http://www.mepco.com.pk/ls/ConsumerLoadShedding.asp?SDiv=5124&off=Shah Rukn-e-Alam Sub Division");
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);	
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,  2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

curl_setopt($ch, CURLOPT_MAXREDIRS, 10);	

$res=curl_exec($ch);
echo $res;

Actually I was trying above code for your site. They have prohibited such access. its giving me following message.

Your page is blocked due to a security policy that prohibits access to Category default

[ayesha789]

you need to enable php_curl.dll extention

But I dont know how to enable, I am using XAMPP and in PHP Manual they said put Curl files in Libcurl....confused

[urtrivedi]

I have no clue about xampp. what i used to do was
I opened my php.ini file enabled curl by adding following line with other extension, save the file

extension=php_curl.dll

Then i ensured that this file is there in php/extensions/ folder
restarted apache
and thats it

But as i told u in my previous post, even after writing a code you will not able to access that website, because they are not allowing.

[chrishea]

If you want to dump an online (html) report to excel it's pretty easy. My little utility module is set up to save to various formats. You can see the details and do a download here. It won't be totally automated as you will need to click on a link for each report to create the files but we're talking about minutes versus hours.

[ayesha789]

Thanks dear, this tool is cool. I will try this for getting result.