I’ve been working with personal computers for awhile. I’m not quite from the punch card era, but I do fondly remember saving my prime number sieve program, written in basic, to an audio cassette attached to a Commodore Vic-20. Somewhere I still have the 24-pin dot matrix printout of all the prime numbers less than 1,000,000.
Anyways, I’ve also been using cron for quite some time, and so it frustrates me how often I seem to repeat the same mistakes when working with crontab files. So, read on and let me know if you’ve ever caught yourself on any of these:
It’s not a shell script
The first important observation about the crontab file is this: It’s not a shell script. Sure, it looks a bit like a shell script since you can set some variables, but it’s not. It’s best to think of the crontab as an interpreted file that happens to support some variable declarations that look a lot like a shell script. But they are simple
name = value pairs NOT shell variables. The “value” must be a simple value. For example, you can not de-reference a crontab “variable” in a value, so this won’t work:
#Does not work because you can't de-reference PATH in the value. PATH=/my/foo/path:$PATH #Does not work because sub-shells have no meaning in a crontab file. PATH=`path_script`:/bin
In fact, you can not set arbitrary “variables” in crontab, so this won’t work:
#Does not work - you can not set arbitrary "variables" COMMON_CMD=/my/home/bin/doit.sh
In fact, it’s really wrong to think about these as variables at all. Really, they are cron options that derive their default values from your environment variables. More importantly, there are only a limited number of “variables”, er options, that can be defined in a crontab. Here is the list pulled from “man 5 crontab” on Debian Squeeze:
- Works just like the shell PATH, but it does *not* inherit from your environment. Typically set to a very short list of path elements, often just “/usr/bin:/bin”
- A comma delimited list of mail address to which to send the output of cron jobs. Applies to all cron entries after the MAILTO declaration. You can define it multiple times
- The path to the crontab owners’ home directory.
- The shell to use when invoking cron jobs – the default is /bin/sh.
- Set from /etc/passwd
- The content-type to use for cron output emails.
- The charset to use for cron output emails.
And that’s all.
Percent signs need to be escaped
I forget this one more often than I would like to admit. Usually, I’m trying to do something like:
#Does not do what you might expect because of the '%' sign 0 23 * * * (EPOCH=`date '+%s'`; echo $EPOCH >> /tmp/foo)
If you do this, you will get a friendly email error like this:
(EPOCH=`date '+ Auto-Submitted: auto-generated X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: X-Cron-Env: /bin/sh: -c: line 0: unexpected EOF while looking for matching ``' /bin/sh: -c: line 1: syntax error: unexpected end of file
As you can see from the Subject line, the command that actually got executed stopped at the ‘%’ sign. This is because crontab treats the ‘%’ sign as a newline – all content after the ‘%’ will be passed into STDIN of the command. In order to use the ‘%’ sign in a command you have to escape it with a backslash:
0 23 * * * (EPOCH=`date '+\%s'`; echo $EPOCH >> /tmp/foo)
In a shell script, a single line can span multiple lines by using a trailing ‘\’ at the end of each line. It is so tempting to do this in a crontab file, but you can not. Instead, if my crontab command starts getting long enough that I start thinking about backslashes, then I move the command into a shell script and invoke that instead. Much easier to read and maintain.
You can not put comments at the end of a “variable” declaration, nor in a command. They will be interpreted.
#This comment is ok, but the next one will cause you grief PATH=/bin:/usr/bin #Why did I put this here? #This will append to the file "foo#Store" * * * * * echo `date '+\%s'` >> /tmp/foo#Store useless timestamps!
So, those are the big crontab pitfalls that I re-discover on occasion. Hopefully after reading this I’ll save you some crontab grief.
Thanks for this!
I expected to see PATH in the set of cron options (along with MAILTO, for e.g.). Is PATH different / special?
Right – I completely forgot to include PATH in the list, which is amusing since it’s probably the most important variable. That’s fixed now, thanks!
This is one of the best crontab tutorials I’ve seen, and I’ve come back to it a couple of times. Thanks for sharing!
The “variables”! In my opinion the documentation says you can assign any value to any name. I agree with Tom, this is the most important article about crontabs on the Internet.
I’ve been a Unix admin for 15 years, and I still make some of these mistakes. Handy to have them compiled in one place, thanks.
Random variables are definitely supported on Redhat and Debian. Have been for a long, long time (I checked on a Redhat 3 box, but also Redhat 5 and Debian 6).
* * * * * echo $RANDOMVARIABLE >> /tmp/`whoami`.cronout
Will do what it looks like it should do. As would the COMMON_CMD example above.
I was searching google because I needed to declare a variable in cron to be used as “base_path” to use for many cronjobs.
It works like a charm, tested this:
*/1 * * * * root echo $TEST > $TEST/test.txt
So I don’t really understand about the statement you made:
“#Does not work – you can not set arbitrary “variables”
It works for me, or I misunderstood what you mean.
Thanks for the great post though!
I tried to do like this:
* * * * * cd $DIR && ./myscript.pl
And got this “reply” from crontab:
/bin/bash: line 0: cd: $HOME/git/ergobo: No such file or directory
The $HOME variable was not resolved within the $DIR variable – I think that is one of the points of this article. If I had written:
* * * * * cd $HOME/git/ergobo && ./myscript.pl
it would have worked fine.
i am assigning some values from cat command to variables in shell script but the cron does not pick
Can anyone suggest ????
I think this site will explain some more information about crontab.
I don’t know if it’s new functionality, but you CAN assign arbitrary variables values. But, you can’t use them later in RHS of another assignment.
I also find subshell functionality works with the parent syntax (haven’t tried backtick).
replying to myself… correction, subshell assignment does NOT work… sigh.
everything else I said is true.
crazy, replying to my reply… thought I’d seen subshell work in some context in cron.
it does work in commands to be executed.
e.g., * * * * * echo $(date) >> /tmp/date.txt
DOES work, and will echo the current date/time appended to /tmp/date.txt
silly cron. a bit of a dinosaur.
Found this post after writing a cronjob in /etc/crontab that failed to redirect the output because I didn’t escape my % signs. I had figured that that was why it failed, but wasn’t sure. I’m really glad for the confirmation in this article.
Thank you for this!