Member Avatar for bigwhiteegg

Hi there
what I'm trying to do here is
[subject]
building a website that updates several other sites' content
such as some images and URL
hopefully it is not in the illegal arena
I have some idea in mind
just not sure if it is going to work
so i decided to post it here, see if anyone can give me some advice

[question?]
write a PHP to extract data I need from the sites, save it to a XML -> set a cron job letting the PHP auto-run -> in HTML, read the XML file

Q1. is it illegal??
Q2. how should I set the tag in the XML for HTML using?
Q3. is my idea going to work? can someone give me some hints if it doesn't?

thanks a lot for helping me out :)

  • 3 Contributors
  • 5 Replies
  • 125 Views
  • 2 Days Discussion Span
  • Latest Post Latest Post by Alish735

Dani AI

Generated

The high-level plan you sketched (server-side PHP to fetch remote content, save structured data, schedule with cron, have the front end read the cached file) is workable. 's iframe idea can work as a quick hack, but it is brittle: many sites set X-Frame-Options or CSP to block framing, hotlink images, or include JS that breaks when served from another origin. A cron-driven server fetch plus caching is more robust, predictable, and easier to control.

Legality and etiquette are not automatic yes/no answers. Check the target site's Terms of Service and any published API/RSS first and prefer those over scraping. Republishing full copyrighted content or hotlinking images can expose you to complaints or takedowns; when in doubt, ask permission, attribute, or cache only metadata (title, summary, link). robots.txt guides crawlers but does not remove legal risk.

Suggested XML shape (keep timestamps, provenance and caching metadata):

<feed>
  <item id="123" source="https://example.com/page" lastUpdated="2025-11-29T12:34:56Z">
    <title>Example title</title>
    <summary>Short summary</summary>
    <content><![CDATA[ ... sanitized HTML ... ]]></content>
    <image url="https://example.com/img.jpg" cached="/cache/img-123.jpg" />
    <checksum>sha256:...</checksum>
    <ttl>3600</ttl>
  </item>
</feed>

Practical implementation notes: schedule the fetcher with cron, for example:

*/15 * * * * /usr/bin/php /var/www/html/fetch.php >/dev/null 2>&1

Use conditional GETs (If-Modified-Since / ETag) to avoid needless downloads. Parse HTML with a real parser (DOMDocument, DomCrawler, etc.), rewrite relative URLs to absolute, sanitize or strip remote scripts, and store images locally only with permission. Handle HTTP errors, redirects, rate limits and exponential backoff; log activity. If pages rely on client-side JS to build content, use a headless browser (Puppeteer) rather than naive HTML fetches. These steps keep the system reliable, respectful of remote servers, and easier to maintain.

Recommended Answers

All 5 Replies

Member Avatar for RoyalElite96

I think the most simple solution would be to get the content in an <iframe> or use PHP's file_get_contents. If you want to save it take a look at fwrite (http://www.tizag.com/phpT/filewrite.php).

And for the refresh: <meta http-equiv="refresh" content="30">.

I don't know if it is illegal.

I hope the rest was helpful though!

Member Avatar for bigwhiteegg

can you explain more about the iframe part?
I'm not really getting it :(

Member Avatar for RoyalElite96

OK, use an iframe to link to a PHP file that saves the current site.

........
<iframe src="run.php" width="0" height="0" frameborder="0" scrolling="no"></iframe>
........

The PHP file should look something like this:

........
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="refresh" content="30">.
<title>Title</title>
</head>

<body>

<?php

$file = file_get_contents('http://external.com/index.html');
$saves = fopen('saves.txt', 'w+');
fwrite($saves, $file);
fclose($saves);

?>

</body>
</html>
........

Then add a second iframe:

........
<iframe src="http://external.com/index.html" width="100%" height="100%" frameborder="0" scrolling="no"></iframe>
........

The first iframe handles the saving part (it creates a simple txt file with the contents of the website you want to display, it is invisible). The second one shows the site in a visible form.

Member Avatar for bigwhiteegg

wow that was clear and helpful!
thanks a lot!!
I'll work on it and see if any more problems come out

Member Avatar for Alish735

I never use iframe tag but now i will try it.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.