Member Avatar for showman13

I would like to be able to have a script that could run as a CRON job that would check the price of an item on a site like eBay or Amazon to keep a watch on it for fluctuations.

Does anyone have a suggestion as to how to implement something like that?

  • 2 Contributors
  • 2 Replies
  • 220 Views
  • 3 Hours Discussion Span
  • Latest Post Latest Post by showman13

Dani AI

Generated

— good, practical question. is right that the first step is retrieving the product page, but for anything you care about long-term you’ll want a more robust, polite, and maintainable approach than blind HTML scraping.

Prefer official APIs when possible (Amazon’s product API and eBay’s public APIs) because they return structured data, are less likely to break, and avoid ToS problems. If an API isn’t available for the exact data you need, use a server HTTP client (cURL, Guzzle) or a headless browser (Puppeteer/Selenium) for JS-rendered pages. Always look first for embedded structured data (JSON‑LD or microdata) — sites often publish price as machine-readable JSON and parsing that is far more stable than brittle CSS selectors.

A simple cron workflow: 1) fetch with a descriptive User-Agent and rate limits, 2) parse structured data or DOM to extract price and currency, 3) normalize to a numeric value, 4) compare with your stored last price and threshold, 5) persist the new value and send a notification if it crossed your threshold. Add retry/backoff logic, detailed logging, and alerts for bot-detection pages (CAPTCHAs or 503s). Respect robots.txt and the marketplace ToS; frequent, aggressive scraping risks IP blocks and possible account action.

Troubleshooting tips: prefer JSON‑LD when present, strip commas/currency symbols before casting, treat shipping/taxes separately, and test against multiple listings (mobile vs desktop HTML can differ). For reliability, store the raw HTML or JSON for failed parses so you can debug later.

Example PHP skeleton (JSON‑LD extraction):

<?php
$html = curl_exec($ch); // fetch with curl/Guzzle
if (preg_match('/<script[^>]+type=["\']application\/ld\+json["\'][^>]*>(.*?)<\/script>/is', $html, $m)) {
  $ld = json_decode(trim($m[1]), true);
  $price = $ld['offers']['price'] ?? null; // normalize and store
}
?>

Run this from cron, log everything, and prefer APIs for production use.

Recommended Answers

All 2 Replies

Member Avatar for showman13

Thank you rproffitt for that very direct and precise answer.

That is why I like this forum. When someone asks a question the answers are generally very helpful.

That looks like exactly what I needed, and gives me a direction to go for finding the same type of information from other platforms.

I appreciate it
Douglas

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.