Automating Tasks With Unix Crontabs

Web developers often use Unix like operating systems like Ubuntu, centos and others to deal with server installation, configuration, and management. Linux provides many effective tools to make some things easier to maintain, one of these tools is called the Crontab.

- Cron means “time” in Greek language

Intro

Web developers often use Unix like operating systems like Ubuntu, centos and others to deal with server installation, configuration, and management. Linux provides a lot of very effective tools to make some things easier to maintain, one of these tools is called the Crontab.

How can you benefit from Cron jobs, here’s some case scenarios I had to deal with:

1. Task automation, checking email, downloading files over the internet.
2. Maintain system services, rotating logs (done by the OS automatically).
3. Making database backups.
4. Generating and sending reports.
5. Rotating custom logs.
6. Updating search indexes.
7. Checking external APIs.
8. Basically. whatever you need on a recurring basis.

Crontab is basically a service running (in daemon mode) on the operating system, this service does the following:

1. Read the Crontab files, which are just text files containing Cron Tasks (more on this later).
2. Parse all Cron Jobs in that file, and execute the ones that are scheduled for this time as a super user.
3. After one minute, repeat the whole thing.

You can trust the Cron Job service, it’s very reliable. It doesn’t occupy the memory when it’s idle. And if the service exits and fails it will restart at a certain time automatically.

Info: Cron jobs on Unix systems and event scheduler in Microsoft Windows operating system have the same main idea.

Let’s get down to business:

Now there are 2 ways of making Cron Jobs:

Method 1: Using the Crontab command

This method alters the files at /var/spool/cron/crontab directory, each user will have a file named by the username in that directory. we should not alter these files manually however. The Crontab command is responsible for handling these files.

To open the current user Crontab use:

Crontab –e (edit)

this will open the Crontab file like so:

At the bottom of this file, you can start writing your Cron Jobs.

Opening the file for the first time will prompt you for your favorite text editor, at the moment I advise you to use the Nano text editor, option 2 from the list. Alter the file, click F2 to exit then Y to save the changes.

That’s it, nothing else needs to be done, the Cron service will pick up the changes.

Method 2: writing to a pre-configured scheduled Cron directories

These files are located at:

/etc/cron.daily

/etc/cron.hourly

/etc/cron.monthly

/etc/cron.weekly

You just write a file and save it at one of the above directories, say /etc/cron.daily/myCron (leave the file extension), and it will be executed in the background once every day. Very simple huh!

These files are system wide and always run by the root user. Adding Cron Jobs in these files would require you to specify the user that the Cron Job will run as. take a look at /etc/Crontab:

You see how this file is responsible for running all Crontabs in the directories in method 2.

Info: it’s rarely that you would need to add Crontabs in:

• /etc/Crontab
• /etc/cron.d/*

The order of execution of multiple Crontab files in a directory like /etc/cron.daily/. is according the ASCII code. So a file with a name starting with 0 will run first

After writing your Cron Jobs, To list the current user’s Crontabs

Crontab –l (for listing)

Crontab –u someuser –l (view Cron Jobs for the current user)

Now, how to write a Cron Job

Let’s say we have a command to update a search index for a website, don’t care much about the command itself:

The command is python manage.py update_index, and we need to make the system run it once every day.

So, obviously, instead of doing this manually. We use Cron Jobs.

The syntax for a user’s Cron Job is:

Minute hour dayofmonth month dayofweek command

And for system wide Cron Jobs (which we rarely use), the syntax is:

Minute hour dayofmonth month dayofweek user command

For each minute, hour, day of month, month and day of week, there are many options:

• * for any (ex: * in minutes means at any minute, all 60 of them)
• X, where x is a number stating the actual time (ex: 12 * * * * means anytime every 12thminute of an hour, like 6:12, 9:12 …etc.)
• X-y, a time period (ex: * 6-8 * * *, means any minute between 6 am and 8am of any day of the year)
• X,y,z,…, different options for the same placeholder (ex: * * 10,15,25 * *, means any minute of only the 10th, 15thand 25thdays of any month in the year)
• x/4, a division, (ex: * */8 * * *, means every 3 hours. Same as * 0,3,6,9,12,15,18,21 * * *)

there are short hands to user which may help as more readable, like @daily. You can use it instead of using the first part of the Cron Job (* * * * *). For example:

@daily python manage.py update_index

Is the same as:

0 0 * * * python manage.py update_index

options for short hands are:

@hourly, @daily, @weekly, @monthly, @midnight @annually and @yearly which are the same thing.

There’s a special one, @reboot. Crons with @reboot schedule run only once at system start.

With the 5 placeholders (* * * * *), you can pretty much pick any time interval in a year you can imagine, here’s an exercise for you:

Write a Cron Job to echo your name every Sunday in July only.

It’s a good idea to save whatever return from the executed command by appending to some text file, this is a good practice for monitoring and checking the command behavior, I use this to see errors and warnings from my command that may help for problem troubleshooting.

Another good practice is to just run a bash script at a separate file from the command in the Cron Job, this makes it easier to run multiple commands in multiple lines of script instead of using & or && between commands in one line

Example usage:

0 0 * * * source /some/path/myscript.sh >> /some/path/log.txt

you can send the results of the Cron Job command to your email by specifying:

[email protected] (in the same Crontab file).

info: For this to work you need to have your system or server setup and configured to send emails.

There are many options like ssmtp, postfix, mailgun, sendmail and others. For me I recommend using mailgun, because they give you 10000 emails per month on their free plan (at the time of writing this).

I would like to mention https://Crontab-generator.org which is an online helper tool to generate Cron Jobs. It uses a graphical user interface to pick the right time schedule, which may be easier for newbies to start with

Who has access to Cron Jobs

As a system administrator, you may need to limit users who are allowed to make Cron Jobs on the system. This can be done with 2 files located at /etc/cron.deny and /etc/cron.allow, these files simply contain lines of usernames, empty lines are not allowed. The allow and deny files are read each time a user tries to add, modify or delete a Cron Job, so there’ no need to restart the Cron daemon for changes to take effect.

for example if you want to only allow only userA and userB to make Cron Jobs, then in the cron.allow enter:

# /etc/cron.allow userA userB

and if you want all users to make Cron Jobs the you either enter “ALL” in the cron.allow file, or just delete the file (which doesn’t exist initially).