Bash script called with shell_exec in PHP never finish

Question

I have a site where I want the user to be able to download all the content of a filefolder. I have therefor made a php-script that gives the user the option to create a backupfile. As this file can be rather big (several GBs of data) I have made the script that makes the backup as a bash script that I then run through the php shell_exec command.

The bash script runs fine both directly from a putty-terminal and when called from PHP, but if the files to backup are to many the process stops without finishing when called from PHP (not when called directly).

The bash command to call the script is "bash backup_shell_zip" and the shell_exec command in the calling php-file is shell_exec("bash backup_shell_zip 2>/dev/null >/dev/null &");

I can see that the script, when it runs on the larges site for backup I have with around 12 Gb of files, takes about 12 minutes to finish when called directly from a putty-terminal. When same script is called from the php shell_exec it runs for about 2½ minute before terminating without finishing (having backupped around 3 Gb). When I run the same script from a smaller example (less than 3 Gb) it works fine, but also finishes i around 2 minuttes.

So it looks like the process (bash script) terminates even though it is not finished after a period of time (or ressource) use. Perhaps it have something to do with how the server keeps an eye on processes but I am not a "server-guy", so I hope someone can help ;-)

/Malte

PHP-script calling the Bash Script

shell_exec("bash backup_shell_zip 2>/dev/null >/dev/null &");

The Bash Script

#!/bin/bash

BACKUPFILES="files"
DEST="backup"
CHUNKSIZE="500M"

# Filename
DAY=$(date +%F-%T)
ARCHIVE="backup_files"
FILENAME="$ARCHIVE-$DAY.zip"

# Print status
echo $(date +%T)
echo "Creating backup from $BACKUPFILES to $DEST/$FILENAME"

# Create ZIP-files
zip -r -s $CHUNKSIZE $DEST/$FILENAME $BACKUPFILES 

# Print status
echo $(date +%T)
echo "Finished"

maybe try setting time limit to 20 minutes set_time_limit(1200) before calling shell_exec — Sumit Wadhwa
– Sumit Wadhwa, Commented Sep 21, 2020 at 9:51
I send the shell_exec command with 2>/dev/null >/dev/null & so it should run in the background and not be affected by neither time_limit or ini (max_execution_time), but I have tried both without succes... — Malte von Sehested
– Malte von Sehested, Commented Sep 21, 2020 at 10:11
@MaltevonSehested : I don't see, how just running it in the background should keep the script from being aborted by a time limit. I find it a bit unwise that you throw away the stderr. Didn't it occur to you that the script may abort due to some error and, before dying, writes some useful message to stderr? — user1934428
– user1934428, Commented Sep 21, 2020 at 10:16
As the script runs fine from a commandline I do not think that that is the issue. Also it takes several minutes to complete, so the whole idea is that the user initiates the backup and then go back to a download-page at a later time and download the zip-file(s) with the backup. If I did not throw the script to the background it would "hang" the webbrowser until finished and the user would not be able to do other tasks... — Malte von Sehested
– Malte von Sehested, Commented Sep 21, 2020 at 10:31
I suspect that the reason is that the system discovers that the php that started the shell is no longer "active" and the kills the shell. Perhaps it would be possible to have daemon or something run the script and then have php invoke it, but I am not really a serverguy, so here I am kind of way outside my knowledgearea ;-) — Malte von Sehested
– Malte von Sehested, Commented Sep 21, 2020 at 10:52

Malte von Sehested · Accepted Answer · 2020-09-29 16:15:32Z

1

Turns out it was a question of ressources of the apache-user as opposed to the shell-user that gave the timeout. There is no simple solution, but I am currently working with a cron-job that can divide the task into several smaller jobs if the folder being backup'ed is to big...

answered Sep 29, 2020 at 16:15

Malte von Sehested

214 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Bash script called with shell_exec in PHP never finish

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related