11

Currently, I tried to prevent an onlytask.php script from running more than once:

$fp = fopen("/tmp/"."onlyme.lock", "a+");
if (flock($fp, LOCK_EX | LOCK_NB)) {
  echo "task started\n";
  //
    while (true) {
      // do something lengthy
      sleep(10);
    }
  //
  flock($fp, LOCK_UN);
} else {
  echo "task already running\n";
}
fclose($fp);

and there is a cron job to execute the above script every minute:

* * * * * php /usr/local/src/onlytask.php

It works for a while. After a few day, when I do:

ps auxwww | grep onlytask

I found that there are two instances running! Not three or more, not one. I killed one of the instances. After a few days, there are two instances again.

What's wrong in the code? Are there other alternatives to limit only one instance of the onlytask.php is running?

p.s. my /tmp/ folder is not cleaned up. ls -al /tmp/*.lock show the lock file was created in day one:

-rw-r--r--  1 root root    0 Dec  4 04:03 onlyme.lock
4
  • Very interesting, but it seems like this should be correct... Is the skeleton of your code exactly that? I do wonder though what happens if flock is passed false rather than a resource. Surely it returns false. It would be very, very odd if it doesn't, but I've seen PHP make some pretty odd choices. Also, the internals of flock could be flawed (or the system calls used -- though that would've been noticed long before now). Commented Dec 12, 2012 at 9:24
  • The code is almost 100% like the real code. The // do something lengthy actually exec() another .php which is the script which should run periodically, but not more than once. Commented Dec 12, 2012 at 9:34
  • Second process will wait for finish first process. See bug: stackoverflow.com/questions/5524073/lock-nb-ignored Commented Nov 20, 2014 at 9:45
  • Possible duplicate of How to prevent multiples instances of a script? Commented Nov 29, 2016 at 22:39

7 Answers 7

13

You should use x flag when opening the lock file:

<?php

$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
  // Do processing
  while (true) {
    echo "Working\n";
    sleep(2);
  }
  fclose($f);
  unlink($lock);
}

Note from the PHP manual

'x' - Create and open for writing only; place the file pointer at the beginning of the file. If the file already exists, the fopen() call will fail by returning FALSE and generating an error of level E_WARNING. If the file does not exist, attempt to create it. This is equivalent to specifying O_EXCL|O_CREAT flags for the underlying open(2) system call.

And here is O_EXCL explanation from man page:

O_EXCL - If O_CREAT and O_EXCL are set, open() shall fail if the file exists. The check for the existence of the file and the creation of the file if it does not exist shall be atomic with respect to other threads executing open() naming the same filename in the same directory with O_EXCL and O_CREAT set. If O_EXCL and O_CREAT are set, and path names a symbolic link, open() shall fail and set errno to [EEXIST], regardless of the contents of the symbolic link. If O_EXCL is set and O_CREAT is not set, the result is undefined.

UPDATE:

More reliable approach - run main script, which acquires lock, runs worker script and releases the lock.

<?php
// File: main.php

$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
  // Spawn worker which does processing (redirect stderr to stdout)
  $worker = './worker 2>&1';
  $output = array();
  $retval = 0;
  exec($worker, $output, $retval);
  echo "Worker exited with code: $retval\n";
  echo "Output:\n";
  echo implode("\n", $output) . "\n";
  // Cleanup the lock
  fclose($f);
  unlink($lock);
}

Here goes the worker. Let's raise a fake fatal error in it:

#!/usr/bin/env php
<?php
// File: worker (must be executable +x)
for ($i = 0; $i < 3; $i++) {
  echo "Processing $i\n";
  if ($i == 2) {
    // Fake fatal error
    trigger_error("Oh, fatal error!", E_USER_ERROR);
  }
  sleep(1);
}

Here is the output I got:

galymzhan@atom:~$ php main.php 
Worker exited with code: 255
Output:
Processing 0
Processing 1
Processing 2
PHP Fatal error:  Oh, fatal error! in /home/galymzhan/worker on line 8
PHP Stack trace:
PHP   1. {main}() /home/galymzhan/worker:0
PHP   2. trigger_error() /home/galymzhan/worker:8

The main point is that the lock file is cleaned up properly so you can run main.php again without problems.

Sign up to request clarification or add additional context in comments.

8 Comments

if the code dies at location echo "Working\n"; as a result of a bug, the .lock file will remain on disk and not be cleaned up by anyone, thus no more processing will be done.
@ohho That's the case with any tool which relies on lock files. You can acquire lock inside main script, then run worker script as a child process and then remove the lock file in main script
@galymzhan LOCK will be released when the process die, file will not be deleted when the process die. Please check documentation.
what if the main.php is dead before unlink? nothing will be done after that.
for example, server reboots when exec($worker, $output, $retval); is running.
|
11

Now I check whether the process is running by ps and warp the php script by a bash script:

 #!/bin/bash

 PIDS=`ps aux | grep onlytask.php | grep -v grep`
 if [ -z "$PIDS" ]; then
     echo "Starting onlytask.php ..."
     php /usr/local/src/onlytask.php >> /var/log/onlytask.log &
 else
     echo "onlytask.php already running."
 fi

and run the bash script by cron every minute.

1 Comment

This is usually the way I (system admin) do it. But, I came across a situation where a php script executes without errors and correct return values when called from BASH script run by cron. However, the result was not as expected. In my case, php was supposed to create a dir on a remote server with an ssh call. When php scrpit is run directly, it creates the dir. But, through BASH script, it does not create it. @galymzhan answer solved the part where php should run itself without duplicate process. I am yet to test through cron.
1
<?php

$sLock = '/tmp/yourScript.lock';

if( file_exist($sLock) ) {
 die( 'There is a lock file' );
}

file_put_content( $sLock, 1 );

// A lot of code

unlink( $sLock );

You can add an extra check by writing the pid and then check it within file_exist-statement. To secure it even more you can fetch all running applications by "ps fax" end check if this file is in the list.

6 Comments

It's possible for file_exists to be called while a script is just about to create the file, thus it being created/written to twice. This doesn't work. (Also, unsetting the file wouldn't mean anything as far as the actual file goes.)
I suppose this application is not multithreaded as of php so that wont be a problem. Changed the unset to unlink (misspelled)
If you remove the lock by hand (eg rm -rf /tmp/yourScript.lock) you have your self to blame.
The problem is that file_exists and file_put_contents are not one atomic unit. You have two operations: x then y. Assume you have to processes. Ideally they would go as x1 y1 x1 y2 (though y2 wouldn't happen). What can happen though is x1 x2 y1 y2. That means the check just failed.
if you run the script multiple times at the exact moment, yes. But this is a cron job.
|
0

try using the presence of the file and not its flock flag :

$lockFile = "/tmp/"."onlyme.lock";
if (!file_exists($lockFile)) {

  touch($lockFile); 

  echo "task started\n";
  //
  // do something lengthy
  //

  unlink($lockFile); 

} else {
  echo "task already running\n";
}

3 Comments

it works in the context of the question since there will never be a race condition since the script creates its own lockfile. multiple scripts using same lockfile = different story.
Nope, has nothing to do with the script file. Has to do with more than 1 process running at a time. The file_exists/touch/unlink chain isn't guaranteed to happen uninterrupted for each process without due to processor scheduling. (Although I must cede, Niclas just made a very valid point that the two processes will never spawn off quickly enough to matter, so yes, I suppose this would work. It depends the spawning being spaced apart though.)
"check the presence of the file" does not work. if the process die in middle of a bug, the .lock file will remain in disk and prevents the cron job from starting another process.
0

You can use lock files, as some have suggested, but what you are really looking for is the PHP Semaphore functions. These are kind of like file locks, but designed specifically for what you are doing, restricting access to shared resources.

1 Comment

From the docs: "sem_acquire() is blocking, meaning that subsequent calls with the same semaphore will block indefinitely until the semaphore is released. This ensures serialization, but it is not very practical if all you want to do is check if you should proceed or not. Unfortunately, PHP does not yet support any method of querying the state of a semaphore in a non-blocking manner."
0

Never use unlink for lock files or other functions like rename. It's break your LOCK_EX on Linux. For example, after unlink or rename lock file, any other script always get true from flock().

Best way to detect previous valid end - write to lock file few bytes on the end lock, before LOCK_UN to handle. And after LOCK_EX read few bytes from lock files and ftruncate handle.

Important note: All tested on PHP 5.4.17 on Linux and 5.4.22 on Windows 7.

Example code:

set semaphore:

$handle = fopen($lockFile, 'c+');
if (!is_resource($handle) || !flock($handle, LOCK_EX | LOCK_NB)) {
    if (is_resource($handle)) {
        fclose($handle);
    }
    $handle = false;
    echo SEMAPHORE_DENY;
    exit;
} else {
    $data = fread($handle, 2);
    if ($data !== 'OK') {
        $timePreviousEnter = fileatime($lockFile);
        echo SEMAPHORE_ALLOW_AFTER_FAIL;
    } else {
        echo SEMAPHORE_ALLOW;
    }
    fseek($handle, 0);
    ftruncate($handle, 0);
}

leave semaphore (better call in shutdown handler):

if (is_resource($handle)) {
    fwrite($handle, 'OK');
    flock($handle, LOCK_UN);
    fclose($handle);
    $handle = false;
}

4 Comments

I don't quite understand your lead statement - if you're unlinking the lock file, that means you're already done with your locked section in that process. Similarly, if the file descriptor is closed, for any reason, it means the locking is done. The other scripts should at that point be able to proceed. Why would that be broken?
@JosipRodin I thought so too, until I got a problem with that. You can try to test this behavior. Maybe it depends on the system and its implementation. On Gentoo I had a problem. I do not see any reason to look for the cause, if the above code works. With regard to the removal of the file - deleting a file does not cause the closure of the file descriptor. inode is still available.
I suppose you're referring to the race condition described at stackoverflow.com/questions/17708885/… The reason the simpler methods should be better is that they cause less I/O, which can matter in systems that have a large amount of lock contention and a large amount of I/O on the same devices.
@JosipRodin May be you right. But fstat (or stat) on Windows always have inode == 0. I need solution worked in both systems. And I don't like while(true) - it cause of high load sometimes. For example on highloaded systems.
0

Added a check for old stale locks to galimzhan's answer (not enough *s to comment), so that if the process dies, old lock files would be cleared after three minutes and let cron start the process again. That's what I use:

<?php
$lock = '/tmp/myscript.lock';
if(time()-filemtime($lock) > 180){
    // remove stale locks older than 180 seconds
    unlink($lock);
}
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
    // Do processing
    while (true) {
    echo "Working\n";
        sleep(2);
    }
    fclose($f);
    unlink($lock);
}

You can also add a timeout to the cron job so that the php process will be killed after, let's say 60 seconds, with something like:

* * * * * user timeout -s 9 60 php /dir/process.php >/dev/null

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.