Member Avatar for whizkidash

Team,

Hope you all are doing fine

I have one admin server which is being used dedicately to run cron jobs on hourly basis, fetching the details from Database which is in a different server.These cronjob are run on every hourly/5 minutes basis depending as per end user requirement.
The cron jobs are basically reports which end users need for their records

OS version is Red Hat Enterprise Linux Server release 6.6 where the cronjob is set

Now the tricky point:
1)Since last week, everything was getting run once, but when we query the database we are seeing duplicate entries for each cronjobs and the reports are published twice

2) Also, even when I have disabled/removed/changed the timestamp from crontab, we still see the duplicate entries on the database.

Need to find out why this is happening and its remedy,Can anyone guide please.

Regards
Whizkid

  • 2 Contributors
  • 3 Replies
  • 1K Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by Dani
Shanghai Fuente commented: have to ask why at listing gear that is over a decade old and here, can only be found here or there, one or two, used, reform and well, not what you +0

Dani AI

Generated

A couple of places can still trigger your job even after you edit the user crontab. is on the right track about duplicate schedules. On RHEL 6, check system crontabs and drop-in files that run outside your user’s crontab, and also confirm nothing else is calling the same script.

Quick triage:

  • Look for duplicate entries in system crons and all user spools.
sudo grep -R "report_script.sh" /etc/crontab /etc/cron.d /etc/cron.hourly /etc/cron.daily /var/spool/cron/*
sudo crontab -l
sudo crontab -l -u root
  • Verify only one crond and see who is launching the job.
pgrep -x crond -a
sudo tail -n 200 /var/log/cron | grep -E "report_script\.sh|CRON"
  • Make sure runs are not overlapping (a long run started at :00 may still be running at :05 and a second copy starts). Add a lock:
flock -n /var/lock/report.lock -c "/path/to/report_script.sh --args"

Given you still see duplicates after removing the crontab, also check for a second scheduler outside the OS:

  • Database-side schedulers:

    • Oracle: SELECT owner, job_name, enabled FROM dba_scheduler_jobs WHERE job_name LIKE 'REPORT%';
    • MySQL: SHOW VARIABLES LIKE 'event_scheduler'; then SELECT EVENT_SCHEMA, EVENT_NAME, STATUS FROM information_schema.EVENTS;
    • PostgreSQL (pg_cron extension): SELECT jobid, schedule, command FROM cron.job;

  • Another host running the same job. Grep for the script path in your config management (Puppet/Chef/Ansible), Jenkins, or any job runner you use.

Finally, make the job idempotent so an accidental second run cannot double-insert. Add a unique key on the report’s natural key and use an upsert:

  • PostgreSQL: INSERT ... ON CONFLICT DO NOTHING
  • MySQL: INSERT IGNORE or ... ON DUPLICATE KEY UPDATE
  • Oracle: MERGE with matching keys

That combination (find the second trigger + single-instance lock + idempotent writes) usually eliminates this exact symptom.

Recommended Answers

All 3 Replies

Member Avatar for Dani

Are you sure you accidentally aren't running the cronjob from multiple users on the server? Two users might have the same crontab file accidentally. Just a thought?

Member Avatar for whizkidash

No Dani, its from a single user only.

Member Avatar for Dani

I see you marked this thread as solved. Were you able to figure out the problem? What was it?

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.