Bash script runs manually, but fails on crontab

Question

I'm a newbie to shell scripting. I have written a shell script to do incremental backup of MySQL database.The script is in executable format and runs successfully when executed manually but fails when executed through crontab.

Crontab entry is like this :

*/1 * * * * /home/db-backup/mysqlbackup.sh

Below is the shell script code -

#!/bin/sh
MyUSER="root"       # USERNAME
MyPASS="password"         # PASSWORD
MyHOST="localhost"  # Hostname
Password="" #Linux Password

MYSQL="$(which mysql)"
if [ -z "$MYSQL" ]; then
echo "Error: MYSQL not found"
exit 1
fi
MYSQLADMIN="$(which mysqladmin)"
if [ -z "$MYSQLADMIN" ]; then
    echo "Error: MYSQLADMIN not found"
    exit 1
fi
CHOWN="$(which chown)"
if [ -z "$CHOWN" ]; then
    echo "Error: CHOWN not found"
    exit 1
fi
CHMOD="$(which chmod)"
if [ -z "$CHMOD" ]; then
    echo "Error: CHMOD not found"
    exit 1
fi

GZIP="$(which gzip)"
if [ -z "$GZIP" ]; then
    echo "Error: GZIP not found"
    exit 1
fi
CP="$(which cp)"
if [ -z "$CP" ]; then
    echo "Error: CP not found"
    exit 1
fi
MV="$(which mv)"
if [ -z "$MV" ]; then
    echo "Error: MV not found"
    exit 1
fi
RM="$(which rm)"
if [ -z "$RM" ]; then
    echo "Error: RM not found"
    exit 1
fi
RSYNC="$(which rsync)"
if [ -z "$RSYNC" ]; then
    echo "Error: RSYNC not found"
    exit 1
fi

MYSQLBINLOG="$(which mysqlbinlog)"
if [ -z "$MYSQLBINLOG" ]; then
    echo "Error: MYSQLBINLOG not found"
    exit 1
fi
# Get data in dd-mm-yyyy format
NOW="$(date +"%d-%m-%Y-%T")"

DEST="/home/db-backup"
mkdir $DEST/Increment_backup.$NOW
LATEST=$DEST/Increment_backup.$NOW
$MYSQLADMIN -u$MyUSER -p$MyPASS flush-logs
newestlog=`ls -d /usr/local/mysql/data/mysql-bin.?????? | sed 's/^.*\.//' | sort -g | tail -n 1`
echo $newestlog
for file in `ls /usr/local/mysql/data/mysql-bin.??????`
do
        if [ "/usr/local/mysql/data/mysql-bin.$newestlog" != "$file" ]; then
     echo $file             
     $CP "$file" $LATEST         
        fi
done
for file1 in `ls $LATEST/mysql-bin.??????`
do
 $MYSQLBINLOG $file1>$file1.$NOW.sql 
 $GZIP -9 "$file1.$NOW.sql"     
 $RM "$file1"
done
$RSYNC -avz $LATEST /home/rsync-back

First of all, when scheduled on crontab it is not showing any errors. How can I get to know whether the script is running or not?
Secondly, what is the correct way to execute the shell script in a crontab. Some blogs suggest for change in environment variables. What would be the best solution

When I did $echo PATH, I got this

/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/mysql/bin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:~/usr/lib/jvm/jdk-6/bin

This is not to answer the question itself, but to suggest looking at 3rd-party tools for mysql backup scripts and save yourself some headaches. It's a very common thing to do, so there are lots of tools available. One that I've used is called automysqlbackup. I think that uses a dump approach rather than using the data files directly, so it may not suit you, but there are likely others that would. — jfrank
– jfrank, Commented Jan 30, 2013 at 23:23
By the way -- MYSQL="$(which mysql)" is not, generally speaking, a practice you should follow. The shell already caches PATH lookup results, and does so much more efficiently than by starting an external program such as which. — Charles Duffy
– Charles Duffy, Commented Feb 19, 2017 at 2:17

amaurea · Accepted Answer · 2013-01-30 20:08:38Z

27

The problem is probably that your $PATH is different in the manual environment from that under which crontab runs. Hence, which can't find your executables. To fix this, first print your path in the manual environment (echo $PATH), and then manually set up PATH at the top of the script you run in crontab. Or just refer to the programs by their full path.

Edit: Add this near the top of your script, before all the which calls:

export PATH="/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/mysql/bin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:~/usr/lib/jvm/jdk-6/bin"

edited Jan 30, 2013 at 20:08

answered Jan 30, 2013 at 19:54

amaurea

5,07729 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Jay Over a year ago

It's probably better to execute the profile that sets up the environment in bash instead of copying it manually. There's a description of it here: linuxfromscratch.org/blfs/view/6.3/postlfs/profile.html

Stephen P Over a year ago

It's best to execute a cron job with as minimum an environment as possible, and specified exactly to meet the needs of the job. Executing your profile pulls in too much setup, irrelevant settings, and means you can accidentally break your cron job just by changing your interactive environment. For something like this mysql dump it should not be executing under your account anyway, it should be run under the mysql user crontab; you should not be able to login as the mysql user either, so PATH is unknown (system default) for that user.

Chris Baker Over a year ago

@Jay - Stephen P is absolutely correct -- the best route is to use an explicit path, which is lightweight and direct, versus pulling in the whole profile, which is convoluted and excessive. In my opinion, your suggestion is squarely in the "bad practice" camp.

JimR · Accepted Answer · 2013-02-01 21:10:20Z

17

Another more generic way is to have cron run the user's bash logon process. In addition to the PATH, this will also pick up any LD_LIBRARY_PATH, LANG settings, other environment variables, etc. To do this, code your crontab entry like:

34  12  * * *   bash -l /home/db-backup/mysqlbackup.sh

answered Feb 1, 2013 at 21:10

JimR

5132 silver badges11 bronze badges

1 Comment

R. Du Over a year ago

doesn't it trigger the same problem as mentioned before ? That all .bash_profile and .bash_rc will get run even if not needed ? I'm asking cause I'm facing the same problem, I like using BASH_ENV pointing to a minimal env script but that's not working when a shell is run through crontab.

blokeish · Accepted Answer · 2016-07-29 03:56:10Z

7

My Issue was that I set the cron job in /etc/cron.d (Centos 7). It seems that when doing so I need to specify the user who executes the script, unlike when a cronjob is entered at a user level.

All I had to do was

*/1 * * * * root perl /path/to/my/script.sh
*/5 * * * * root php /path/to/my/script.php

Where "root" states that I am running the script as root. Also need to make sure the following are defined at the top of the file. Your paths might be different. If you are not sure try the command "which perl", "which php".

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin

answered Jul 29, 2016 at 3:56

blokeish

6366 silver badges9 bronze badges

Comments

MD004 · Accepted Answer · 2018-08-04 18:07:13Z

0

Just import your user profile in the beginning of the script.

i.e.:

. /home/user/.profile

edited Aug 4, 2018 at 18:07

MD004

6002 gold badges7 silver badges19 bronze badges

answered Jul 16, 2013 at 15:25

Enio Marques Junior

365 bronze badges

2 Comments

djjeck Over a year ago

This isn't enough in my case

Jens Over a year ago

Not a good idea. This creates a dependency to everything that .profile does. And as soon as .profile outputs just one character, you get a mail from cron.

β.εηοιτ.βε · Accepted Answer · 2020-06-29 21:44:54Z

0

In your crontab file your script entry can have
```
* * * * * /bin/bash /path/to/your/script
```
Make sure to use full paths in your script. I have had a look through it and I have not seen any but just in case I missed it.

edited Jun 29, 2020 at 21:44

β.εηοιτ.βε

40.4k14 gold badges81 silver badges104 bronze badges

answered Jun 29, 2020 at 14:33

O. Kasule

211 silver badge2 bronze badges

Comments

gmi rushi · Accepted Answer · 2023-02-14 12:07:07Z

first run command env > env.tmp then run cat env.tmp copy PATH=.................. Full line and paste into crontab -e, line before your cronjobs. as like below

++=============================================================++
(# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
PATH=/root/.nvm/versions/node/v18.12.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/sna>

# m h  dom mon dow   command
0 3 * * * bash /home/ubuntu/test.sh
* * * * * bash /home/ubuntu/test2.sh >> /home/ubuntu/cronlog.log
* 3 * * * bash /home/ubuntu/logscron.sh)
++=====================================================================++

Collectives™ on Stack Overflow

Bash script runs manually, but fails on crontab

6 Answers 6

3 Comments

1 Comment

Comments

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

3 Comments

1 Comment

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related