How to make cronjobs work?

[Shadow]

On my DS718+ I've set up a Ubuntu 18.04.5-live-server-amd64 GUI-less server in Virtual Machine Manager. I use this instance to host a MailCow server.

My Linux knowledge is still very limited. I'm trying to set a cronjob to have this server make backups every night. I've understood from googling that 'cronjob' is the Linux counterpart of Scheduled Tasks in Windows.

So after googling and googling I've done this:

sudo crontab -e

/mnt/nas3bck is a persistant SMB mapping to my NAS, so the data can flow into the usual backup schedules of my NAS .

Content of backupallmc.sh :

If I execute the script backupallmc.sh manually, it works fine. But I would expect this script to be triggered every night. But it's not... Anyone has any hints for me why it may not work?

 

[fredbert]

In the crontab -e, to edit the current user's cron settings, you have set 0 4 * * 1... run at 04:00 every Monday morning. To run every morning at 04:00... 0 4 * * *

cron - Wikipedia

en.wikipedia.org

 

[Shadow]

Since I set this up over a month ago already searching for answers, it didn't ran once. So that's not it..

 

[one-eyed-king]

Hmm. for further inspection, I would pipe the content of env to a file, as well as pipe stdout and stderr of backup_and_restore.sh to a logfile from within backupallmc.sh to get some insights about what actualy happens when crontab calls it.

 

[Shadow]

at the moment I'm googling how to

pipe the content of env to a file

,

pipe stdout

,

stderr of backup_and_restore.sh to a logfile from within backupallmc.sh

...........

 

[one-eyed-king]

Yeah, I was to lazy to type the commands from your screenshots. I wrote pipe, but I ment to write redirect.

To capture the environment variables present during execution in a log file, add this to your script:

env > /path/to/your/logfile

To get stdout and stderr for the backupallmc.sh script just add >/path/to/your/logfile 2>&1 after the script call. (again, I am too lazy to re-type what's on your screenshot).

Additionaly you will want to add the shebang #!/bin/bash in the bash file you call from crontab.

My educated guess: just because a file has a .sh file ending, doesn't make it a bash file from the os perspective. When you run the script from within bash, you are already in bash and it knows how to interpret it. If cron executes the file, it will start it without knowing what it is, and due to the missing shebang it doesn't know it's ment to be executed with bash. Either try to add the shebang, or prefix your cron command with /bin/bash and give it another try.

 

[Shadow]

Additionaly you will want to add the shebang #!/bin/bash in the bash file you call from crontab.

Like so? this also doesn't work...

To get stdout and stderr for the backupallmc.sh script just add >/path/to/your/logfile 2>&1 after the script call. (again, I am too lazy to re-type what's on your screenshot).

shadow771@server4:/mnt/nas3bk/mailcow$ cat envdump.txt
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LANG=C.UTF-8
HOME=/home/shadow771
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
MAIL=/var/mail/root
LOGNAME=root
USER=root
USERNAME=root
SHELL=/bin/bash
SUDO_COMMAND=/usr/bin/env
SUDO_USER=shadow771
SUDO_UID=1000
SUDO_GID=1000
MAILCOW_BACKUP_LOCATION=/mnt/nas3bk/mailcow
shadow771@server4:/mnt/nas3bk/mailcow$

 

[one-eyed-king]

May I ask where you add the env command that redirects to a file?

The idea was to put it in the bash file beeing executed by cron, to a) see that the script actualy was called and b) get details about the execution context.

I assume you did put the command to the backupallmc.sh script and the output you shared was created by it.
Thus, this would indicate that the cronjob itself is called. If so, the question is why the backup_restore.sh script doesn't do it's job.

To get details about that you still need to capture the stdout/stderr output of the script execution:

To get stdout and stderr for the backupallmc.sh script just add >/path/to/your/logfile 2>&1 after the script call. (again, I am too lazy to re-type what's on your screenshot).

The redirect of stdout&stderr to a file stills needs to be added after --delete-days 1 to capute the outputs of the script. This should allow you to see indications why it fails execution.

 

[Shadow]

The idea was to put it in the bash file beeing executed by cron, to a) see that the script actualy was called and b) get details about the execution context.

I assume you did put the command to the backupallmc.sh script and the output you shared was created by it.
Thus, this would indicate that the cronjob itself is called. If so, the question is why the backup_restore.sh script doesn't do it's job.

To get details about that you still need to capture the stdout/stderr output of the script execution:
The redirect of stdout&stderr to a file stills needs to be added after --delete-days 1 to capute the outputs of the script. This should allow you to see indications why it fails execution.

Ooh. So you mean like so?

#!/bin/bash
/opt/mailcow-dockerized/helper-scripts/backup_and_restore.sh backup all --delete-days 1 >/mnt/nas3bck/mailcow/logfile.txt 2>&1

May I ask where you add the env command that redirects to a file?

Not sure I understand your question. Are you asking how I got the MAILCOW_BACKUP_LOCATION environment variable in? I followed these instructions:

https://linuxize.com/post/how-to-set-and-list-environment-variables-in-linux/

 

[one-eyed-king]

Ooh. So you mean like so?

yep.

Not sure I understand your question. Are you asking how I got the MAILCOW_BACKUP_LOCATION environment variable in? I followed these instructions:

The env > redirect should be between the #!/bin/bash and the backup_and_restore.sh command. The idea is to get as much details about the context the script is running in.

 

[Shadow]

It finally works...

I did 2 things, but I'm not sure which one solved it:

1. within the crontab file i changed #!/bin/bash to #!/bin/sh
2. I moved the .sh file from an SMB mounted folder to a folder on the local filesystem ...

0 4 * * * /bin/sh /opt/mailcow-dockerized/helper-scripts/backupallmc.sh

I saw it successfully made a backup last night at 4:00'clock !

Thanks for the help anyway @one-eyed-king

 

[one-eyed-king]

Wow, finaly fixed!

It wouldn't surprise me if the change from #!/bin/bash to #!/bin/sh made the difference.

It kind of remindes me to ssh login via ssh keys: it only works if your user is in the admin group AND has either /bin/sh or /bin/ash as configured shell (in /etc/passwd) - fails silently if it user has /bin/bash configured.