Crontab Linux with PHP, MySQL, and Curl

Question

I am trying to run some php with curl inside of it every so often so that it will retrieve information from a website and then insert it into my database. I am able to get the cronjob to work, and the php works because I have it to send me an email every time it runs, and it does. What's not working however is the CURL script, and or the MySQL scripts.

Cronjob:

* * * * * /usr/bin/php -q  /var/www/rstracker/cronjobs.php

cronjobs.php:

<?php 
$to      = '';
$subject = 'RSTracker Cronjob Ran!';
$message = 'Successful cronjob!';
$headers = 'From: ' . "\r\n" .
    'Reply-To: ' . "\r\n" .
    'X-Mailer: PHP/' . phpversion();

mail($to, $subject, $message, $headers);
?>

<?php 
include("connect.php");

$query = mysql_query("SELECT id FROM users");
while($row = mysql_fetch_array($query))
{

    $userID = $row['id'];
    $date = mktime();

    // create a new cURL resource
    $ch = curl_init();

    // set URL options
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    curl_setopt($ch, CURLOPT_URL, "http://services.runescape.com/m=hiscore_oldschool/index_lite.ws?player=".$username);
    curl_setopt($ch, CURLOPT_HEADER, 0);

    // grab HTML
    $data = curl_exec($ch);

    // close cURL resource, and free up system resources
    curl_close($ch);

    //echo "<textarea>$data</textarea><br />";


    $hs = explode("\n",$data);
    $skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction");
    $i = 0;
    for($i = 0; $i<count($skills);$i++) {
     // Explode each skill into 3 values - rank, level, exp
     $stat = explode(',', $hs[$i]);
     $out[$skills[$i]] = Array();
     $out[$skills[$i]]['rank'] = $stat[0];
     $out[$skills[$i]]['level'] = $stat[1];
     $out[$skills[$i]]['xp'] = $stat[2];
    }




    foreach($out as $key => $value)
    {
        if($value['level']>0||$value['xp']>0||$value['rank']>0)
        {
            mysql_query("INSERT INTO skills SET userID='$userID', skill='$key', level='".$value["level"]."', xp='".$value["xp"]."', rank='".$value["rank"]."', date='$date'");
        }
    }

}


?>

Have you tried running the script outside of a cron job? Does it throw any errors? — Daniel Paul Searles
– Daniel Paul Searles, Commented Jul 10, 2013 at 19:36

Marc B · Accepted Answer · 2013-07-10 19:39:00Z

1

Most likely connect.php isn't in the proper directory. When running as a CRON job, a php script's "working directory" is the home directory of the account it's running under. Unless "connect.php" is somewhere in the include_path, or in the same directory, it's almost certainly NOT being found.

Since it's not being loaded, you don't get a connection to the database.

Since all of your database and curl code are simply assuming that life is perfect, they're not set up to handle the failure of having no DB connection.

As well, even if all this other stuff was working ok, nowhere in your code are you setting $username, so the URL you're scraping off the runescape site is merely

http://services.runescape.com/m=hiscore_oldschool/index_lite.ws?player=

so you'd be scraping invalid data anyways.

General tip: You do not need to create a new curl object each time. you can create ONE curl object outside the loop, then reset the URL inside the loop.

answered Jul 10, 2013 at 19:39

Marc B

362k44 gold badges433 silver badges508 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Dylan Cross Over a year ago

oh my, well i guess that's what happens when you try to code on no sleep! thanks. I just wasn't sure if i had to include "Curl"'s path like you have to php. if not then I can get this working.

hek2mgl · Accepted Answer · 2013-07-10 19:38:18Z

1

I expect that the problem is because of this line:

include("connect.php");

Note that the working directory of the php script when it get's executed by crond is the file sytem root: / rather than the directory where the php script is stored.

Change the path to:

include(__DIR__ . '/connect.php');

answered Jul 10, 2013 at 19:38

hek2mgl

159k31 gold badges263 silver badges279 bronze badges

Collectives™ on Stack Overflow

Crontab Linux with PHP, MySQL, and Curl

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related