Recover from an accidentally deleted crontab?
Over the weekend I made a bad mistake by accidentally deleting my crontab file.
I meant to enter:
crontab -e (for edit)
But by accident I entered:
crontab -r (for remove).
(You ever notice that "e" and "r" keys are next to each other!!")
If there was an "Are you sure you want to delete crontab?" message I don't remember seeing it… I was obviously not paying attention to what I was doing… having entered "crontab -e" a million times in the past.
There is no system backup of this important file (that I could find) and I could not find a way to recover it (via Google) except to go through the past few syslogs looking (via grep) for "cron".
I had about 25 entries in the crontab file and it took several hours for me to research and re-enter them all.
Was there a better (easier/faster) way to recover?
Now I run a job in crontab that creates a copy of the crontab file at 1 AM each night.
00 01 * * * crontab -l > /home/me/my-crontab-backup.txt
If you know of a better recovery method, please post.
I hope this helps someone… and prevents the time-sink that killed my Saturday night!
9 Replies
I have a simple program that builds my crontab from a bunch of little pieces…so that even if the "master" crontab gets deleted, all the little pieces are still around for re-assembly. The pieces are in version control (git repositories) for the each of the projects to which they apply.
I never keep a copy of the "master" crontab. I just change the piece I need, push the changes to the git repository for that project and then reassemble the "master" from the (now-modified) pieces.
You could do the same with a shell script (and m4 to process includes). See man m4.
-- sw
Here's the description my program processes:
# crontab
#
##################################################
# Regular maintenance/backups of web server
##################################################
#
include /srv/www/bin/cron/cron.tab
##################################################
# Regular update of net information database
##################################################
#
include /srv/netinfo/bin/cron/cron.tab
##################################################
# Regular system backups
##################################################
#
include /srv/backup/bin/cron/cron.tab
##################################################
# Regular DMARC database backups
##################################################
#
include /srv/dmarc/bin/cron/cron.tab
##################################################
# Regular iCal/iCard database backups
##################################################
#
include /srv/ical/bin/cron/cron.tab
##################################################
# Regular mail system maintenance
##################################################
#
include /srv/mail/bin/cron/cron.tab
##################################################
# Personal tasks
##################################################
#
include /home/stevewi/bin/cron/cron.tab
m4(1) should be able to process this easily as (n.b. the form of an include for m4(1) is include(/the/path/to/cron.tab)…you'll have to change this with 2 vi(1) substitutions):
m4 $HOME/etc/crontab.conf | crontab -
or
sudo mv /etc/crontab /etc/crontab.old
sudo m4 $HOME/etc/crontab.conf > /etc/crontab
Each cron.tab file has the cron jobs for that particular project (this is the one for mail):
# clean up _mailreaver_ caches and old mail in mailboxes.
#
@daily nice -n 19 sudo /srv/mail/bin/cron/cleanup /srv/mail/var/etc/cleanup.conf
# backup the mail server databases
#
@daily nice -n 19 sudo /srv/mail/bin/cron/backupdb
# clean up deferred mail
#
@weekly nice -n 19 sudo /srv/mail/bin/cron/rmdefer
# update spamd (spamassassin daemon)
#
@weekly nice -n 19 sudo /usr/local/bin/sa-update
# update the DKIM signing key (monthly)
#
@monthly nice -n 19 sudo /srv/mail/bin/cron/updatedkim
# trim spamassassin bayesian filter stuff (monthly)
#
@monthly nice -n 19 sudo /srv/mail/bin/cron/trimbayes
# update the list of mail trackers every day
#
@daily nice -n 19 /srv/mail/bin/cron/updtrk
I’d subscribe to Linode’s backup service too if you haven’t already. Just in case something else ever happens.
As Lou said, it'd be backed up in a Linode Backup (you'd have to restore to an alternate linode and get the cron that way).
You could possibly rebuild it based on the cron logs. Grep for the username in them and you should get an idea of what it was running and when. Wouldn't help for a cron job that has frequency beyond your log retention period.
Something like:
cat /var/log/cron* | grep theusernameyoucareabout | cut -d']' -f2 | cut -d'(' -f3|cut -d')' -f1 | sort | uniq -c
You'd have to adjust this for your systems log format and log filename -
(cat all the unzipped cron logs, grep for the username, cut at the first closing square bracket that'd be at the pid give the 2nd value, cut that at the opening parenthesis giving the 3rd value, cut off anything past the first closing parenthesis (assuming your jobs don't have a parenthesis in them), sort that list, show the unique values with a count of them. There's plenty of other ways you could pipe the content around . .or even shorten it to 'grep theusernameyoucareabout /var/log/cron*' and pipe that into whatever you want to use to munge the logs
That'd get you most of the way there on what ran and how often in a single line. From that you could investigate each individual result to get a feel for the schedule.
Steve has a very elegant solution. I may just copy it once I figure out how it works.
I've never used either 'source' or 'include' commands before, and I have no idea what the m4 macro processor would be used for in this instance.
Since I don't change crontab very often, a plain old copy will work just fine for me. I probably don't have to make it each night … perhaps once a week… but since it probably takes about 1/10 of a second to do, it is no big load on the server… which is basically underutilized anyway. All I have on it are some websites … radioqsl.com , newmediacreate.com and a few others… plus my Piwigo photo gallery installation and a ton of Word and Excel files I want to keep in a safe place.)
Thanks for all the responses.
Al --
If you change my crontab.conf file to look like the one below, it will work with m4(1).
# crontab
#
##################################################
# Regular maintenance/backups of web server
##################################################
#
include(/srv/www/bin/cron/cron.tab)
##################################################
# Regular update of net information datab)ase
##################################################
#
include(/srv/netinfo/bin/cron/cron.tab)
##################################################
# Regular system backups
##################################################
#
include(/srv/backup/bin/cron/cron.tab)
##################################################
# Regular DMARC database backups
##################################################
#
include(/srv/dmarc/bin/cron/cron.tab)
##################################################
# Regular iCal/iCard database backups
##################################################
#
include(/srv/ical/bin/cron/cron.tab)
##################################################
# Regular mail system maintenance
##################################################
#
include(/srv/mail/bin/cron/cron.tab)
##################################################
# Personal tasks
##################################################
#
include(/home/stevewi/bin/cron/cron.tab)
## FIN
##
-- sw
Al --
If you change my crontab "assembly" file (I call it $HOME/etc/crontab.conf) from the one above, to the following it should work with m4(1):
# crontab
#
##################################################
# Regular maintenance/backups of web server
##################################################
#
include(/srv/www/bin/cron/cron.tab)
##################################################
# Regular update of net information datab)ase
##################################################
#
include(/srv/netinfo/bin/cron/cron.tab)
##################################################
# Regular system backups
##################################################
#
include(/srv/backup/bin/cron/cron.tab)
##################################################
# Regular DMARC database backups
##################################################
#
include(/srv/dmarc/bin/cron/cron.tab)
##################################################
# Regular iCal/iCard database backups
##################################################
#
include(/srv/ical/bin/cron/cron.tab)
##################################################
# Regular mail system maintenance
##################################################
#
include(/srv/mail/bin/cron/cron.tab)
##################################################
# Personal tasks
##################################################
#
include(/home/stevewi/bin/cron/cron.tab)
## FIN
##
-- sw
Al --
If you change my crontab "assembly" file (I call it $HOME/etc/crontab.conf) from the one above, to the following it should work with m4(1):
# crontab
#
##################################################
# Regular maintenance/backups of web server
##################################################
#
include(/srv/www/bin/cron/cron.tab)
##################################################
# Regular update of net information datab)ase
##################################################
#
include(/srv/netinfo/bin/cron/cron.tab)
##################################################
# Regular system backups
##################################################
#
include(/srv/backup/bin/cron/cron.tab)
##################################################
# Regular DMARC database backups
##################################################
#
include(/srv/dmarc/bin/cron/cron.tab)
##################################################
# Regular iCal/iCard database backups
##################################################
#
include(/srv/ical/bin/cron/cron.tab)
##################################################
# Regular mail system maintenance
##################################################
#
include(/srv/mail/bin/cron/cron.tab)
##################################################
# Personal tasks
##################################################
#
include(/home/stevewi/bin/cron/cron.tab)
## FIN
##
-- sw
Al --
If you change my crontab "assembly" file (I call it $HOME/etc/crontab.conf) from the one above, to the following it should work with m4(1):
# crontab
#
##################################################
# Regular maintenance/backups of web server
##################################################
#
include(/srv/www/bin/cron/cron.tab)
##################################################
# Regular update of net information datab)ase
##################################################
#
include(/srv/netinfo/bin/cron/cron.tab)
##################################################
# Regular system backups
##################################################
#
include(/srv/backup/bin/cron/cron.tab)
##################################################
# Regular DMARC database backups
##################################################
#
include(/srv/dmarc/bin/cron/cron.tab)
##################################################
# Regular iCal/iCard database backups
##################################################
#
include(/srv/ical/bin/cron/cron.tab)
##################################################
# Regular mail system maintenance
##################################################
#
include(/srv/mail/bin/cron/cron.tab)
##################################################
# Personal tasks
##################################################
#
include(/home/stevewi/bin/cron/cron.tab)
## FIN
##
-- sw