Member Avatar for Vladimirr

Hello, everybody! I am a newbie, so my question will probably sound
silly to you. I have one task that I don’t know how to carry out – I need
to check one web page from time to time (preferably every 20
minutes) and record (=copy) its HTML code every time (the HTML
code of the page is allowed for copying) .

The problem is I can’t keep my computer on all the time, plus, I do need
to leave for work everyday. So, I guess, it should be some kind of script
that I could have run round the clock on some remote windows
server, right?

Do you have any idea how I could do it or what I should start with?

  • 2 Contributors
  • 4 Replies
  • 497 Views
  • 20 Hours Discussion Span
  • Latest Post Latest Post by Vladimirr

Dani AI

Generated

Short answer for : you can automate periodic HTML snapshots while your PC is off, but the fetch must run on a machine or service that stays up. ’s approach (run a small fetch job on a host) is the right pattern — the choices today are basically (a) a hosted cron/HTTP scheduler, (b) a CI runner like GitHub Actions, or (c) a serverless scheduled function or small VPS running a cron job. Each has trade-offs in simplicity, cost, and where you store snapshots. (cron-job.org)

Practical options and when to pick them:

  • Use a hosted cron service (very low setup; it issues HTTP requests on your schedule and can hit a webhook or API that saves the HTML). Good if you want “set-and-forget.” (cron-job.org)
  • Use GitHub Actions (schedule a workflow that fetches the page and commits or stores snapshots). Good when you want built-in versioning and cheap/no-cost runs for small workloads. Note scheduled workflows are configured with cron syntax. (docs.github.com)
  • Use serverless (EventBridge/Cloud Scheduler → Lambda / function) to fetch and save into object storage if you want scale, reliability and direct cloud storage. This is more setup but robust. (docs.aws.amazon.com)

Quick tips to make snapshots reliable and polite:

  • Fetch with a robust client (curl is a common choice on Linux runners) and save with a UTC timestamp in the filename so each snapshot is unique. Rotate or expire old files to avoid storage bloat. (curl.se)
  • Respect the target site’s robots.txt and terms of service, don’t poll too fast (20 minutes is usually fine), and set a clear User-Agent so site owners can contact you if needed. (developers.google.com)

Example: a minimal GitHub Actions workflow (runs every 20 minutes) that fetches and commits a timestamped HTML file:

name: snapshot-site
on:
  schedule:
    - cron: '*/20 * * * *'

jobs:
  snapshot:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Fetch page
        run: |
          ts=$(date -u +'%Y%m%d%H%M')
          curl -sSL 'https://example.com' -o "snap-${ts}.html"
      - name: Commit & push
        run: |
          git config user.name "github-actions"
          git config user.email "actions@users.noreply.github.com"
          git add snap-*.html
          git commit -m "snapshot $ts" || echo "no changes"
          git push

If you want, pick one option and I can give a short checklist for setup (storage, rotation, authentication for pushes, or a serverless example).

Recommended Answers

All 4 Replies

Member Avatar for Vladimirr

Hello, DimaYasny!!!

Thank you very much for your help. I still have some questions:

1)
“get this:
”

I have downloaded this zip file from your link and unzipped it. To
tell you the truth, I don’t know how I should proceed next. I thought
there would be something like “install” file, but there isn’t one. So,
I guess, nothing needs to be installed here, right?

2)
“create a batch file: testsite.bat”

How do I create this batch file? Is it like I have to create a common
Notepad file and then change its extension from .txt to .bat? And
where do I place it after creating it? Into the same folder where all
the other unzipped files are?

3)
“put it in windows scheduler to run every 20 minutes”

Please explain to me how to do that?

4) Lastly, the most important question: after I have fully followed on
your instructions, will I be able to turn off my computer and be sure
that the desired task is still being carried out? I am afraid what you
are suggesting is simply creating a task on my computer that will
be carried out by itself every time I turn on my computer. Which
means that I will have to keep my computer on all the time, otherwise,
if I turn it off, the task process will also stop. Am I correct?
If yes, then this is not what I really want. Perhaps, what I
need is a web-page-html-periodically-copying script that I could
somehow place on some remote windows server that would run my
script all the time regardless of whether my computer is on or off.

Member Avatar for DimaYasny

1. all you need the is wget.exe file.

2. you can use notepad and simply save the file as .bat
it has to be in the same directory where wget.exe is

3. start > control panel > scheduled tasks

4. you place this script on a server you have access to, and it will check the url and place the output in logfile.txt in the same directory.
SOME MACHINE has to be up all the time for the task to be carried out, there is no way around that.

if my instructions are not clear enough for you, I suggest you find a local IT professional, explain your situation, and pay fir his services.

Member Avatar for Vladimirr

Your instructions are more than clear. Thank you very much! (I have just added to your reputation)

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.