Member Avatar for donaldw

Hi all. Here's my situation and how I'm solving it. Please let me know if my methods look good or if there is a better way of doing it.

A local business around here has a drawing each day where they pick some customers' names to win something. The winners' names are posted on a web page each day. I wanted to automate getting the results, specifically if my name is on the list.

I wrote up a PHP script that does the following: First it fetches the content of the page using curl_exec. Then it searches those results using stristr for the name(s) that I'm interested in. If it finds them, it appends the name(s) to a string of winners and sets a flag to true. After checking for the names, if the flag is true, I use mail to send the results, including the names, to my email. I have a cron job set up to wget the page with the script every morning.


Is that pretty much a reasonable way of doing it, or would there be a better way?

  • 2 Contributors
  • 2 Replies
  • 107 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by donaldw

Dani AI

Generated

Good, simple approach by and a useful scheduling tip from . A few focused improvements will make the job more reliable, less fragile, and kinder to the site you’re polling.

Prefer structured parsing over blind substring checks: load the page into PHP’s DOM and use XPath to pick the elements that contain winners. That survives small layout changes and avoids false matches.

$doc = new DOMDocument();
@$doc->loadHTML($html);
$xpath = new DOMXPath($doc);
$nodes = $xpath->query('//ul[@id="winners"]/li | //table[@id="winners"]/tr/td[1]');
foreach ($nodes as $n) {
    $name = trim($n->textContent);
    // normalize and collect
}

Persist the last-seen list and only notify on new entries to prevent duplicate mails. A tiny JSON file or SQLite DB works fine; compare arrays with array_diff and send only the delta. Use a proper mail library (SMTP with auth) for better deliverability and headers.

file_put_contents('last.json', json_encode($current));
$prev = json_decode(@file_get_contents('last.json'), true) ?: [];
$new = array_values(array_diff($current, $prev));
if ($new) {
    // send notification using a mailer library
}

Operational tips: use an HTTP client that supports timeouts and retries (for example, Guzzle), check HTTP status and headers (Last-Modified/ETag) to skip unchanged pages, set a clear User-Agent, and implement backoff to avoid hammering the site. Respect robots.txt and the site’s terms. When running from cron, use absolute paths, redirect stdout/stderr to a log, and keep error handling and logging so failures are visible.

References: DOMDocument and DOMXPath docs (https://www.php.net/manual/en/class.domdocument.php, https://www.php.net/manual/en/class.domxpath.php), PHPMailer (https://github.com/PHPMailer/PHPMailer), Guzzle (https://docs.guzzlephp.org/en/stable/), robots.txt guidance (https://www.robotstxt.org/), PHP CLI notes (https://www.php.net/manual/en/features.commandline.php).

Recommended Answers

All 2 Replies

Member Avatar for edwinhermann

Sounds good - I use a similar system to send me a message when a particular radio station plays the same song twice in one day (first caller gets $1,000). The songs are posted on their web site.

Anyway, back to what you're doing: You could improve the cron side of things. Instead of using cron to call wget, which calls the PHP page (via a Web server), why not modify your PHP file like so:

#!/usr/bin/php
<?php

// your PHP code here

?>

Ensure the file has execute permissions (chmod 700 will work becase that gives you permissions to read, write and execute).

Then just have cron call that file instead of wget. (That file is effectively an executable script).

Note: You may need to change /usr/bin/php to match the path of wherever PHP happens to be installed on your server.

Member Avatar for donaldw

That's a good idea. Plus, then I could eliminate my other cron job of rm-ing the file that I fetched.

Ever won that thousand bucks?

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.