This is a request for information on creating a scheduling system. I have been unsuccessful in finding worthwhile results from Google. Since I have never created one, I thought I might ask those who have and try to avoid common mistakes or pitfalls.

As an example, let's say I am scheduling a web scraper:
Scraper 1 runs every 5mins Monday-Sunday
Scraper 2 runs every 15 mins Monday-Friday
Scraper 1 does not run on Christmas Day

I am not asking for code, but best practices, reading materials or things to consider.

Think crontab, but with a history (when was last scheduled run executed and finished? Is it still executing?). I plan to store the schedule data in a database.

Thanks for any input.

(EDIT: Did not see the "Software Design" forum at first go, if this is in the wrong forum..)

well if you are wanting a database, don't know how deep you want to go with this

schedule
SCHEDULE_ID
SCRAPER_ID
LAST_RUN_TIME

scrapers
SCRAPER_ID
SCRAPER_NAME
INTERVAL_SECONDS
EXCLUDE_DATE

then you can check if LAST_RUN_TIME + INTERVAL_SECONDS >= currenttime
this means it is now scheduled to stated

then check your exclude date
id(date != EXCLUDE_DATE)
{
run the scraper
}

i don't know what you are meaning by best practices or reading materials, for what crons?

this might help
http://www.adminschoice.com/docs/crontab.htm