13

I have array of mobile numbers, around 50,000. I'm trying to process and send bulk SMS to these numbers using third-party API, but the browser will freeze for some minutes. I'm looking for a better option.

Processing of the data involves checking mobile number type (e.g CDMA), assigning unique ids to all the numbers for further referencing, check for network/country unique charges, etc.

I thought of queuing the data in the database and using cron to send about 5k by batch every minute, but that will take time if there are many messages. What are my other options?

I'm using Codeigniter 2 on XAMPP server.

Improve this question

4 Answers 4

Reset to default
38

I would write two scripts:

File index.php:

<iframe src="job.php" frameborder="0" scrolling="no" width="1" height="1"></iframe>
<script type="text/javascript">
    function progress(percent){
        document.getElementById('done').innerHTML=percent+'%';
    }
</script><div id="done">0%</div>

File job.php:

set_time_limit(0);                   // ignore php timeout
ignore_user_abort(true);             // keep on going even if user pulls the plug*
while(ob_get_level())ob_end_clean(); // remove output buffers
ob_implicit_flush(true);             // output stuff directly
// * This absolutely depends on whether you want the user to stop the process
//   or not. For example: You might create a stop button in index.php like so:
//     <a href="javascript:window.frames[0].location='';">Stop!</a>
//     <a href="javascript:window.frames[0].location='job.php';">Start</a>
// But of course, you will need that line of code commented out for this feature to work.

function progress($percent){
    echo '<script type="text/javascript">parent.progress('.$percent.');</script>';
}

$total=count($mobiles);
echo '<!DOCTYPE html><html><head></head><body>'; // webkit hotfix
foreach($mobiles as $i=>$mobile){
    // send sms
    progress($i/$total*100);
}
progress(100);
echo '</body></html>'; // webkit hotfix
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

Wonderful answer, thank you so much. // keep on going even if user pulls the plug (Very funny)
@Christian The solution works better on FireFox & IE, but google Chrome freezed, any other way out?
Safari Webkit (also use din Chrome) has some issues with the PHP progress function. I found a fix though, I'll get back to you with it... Edit: Fixed.
@Christian Sciberras could you please explain the "hotfix"? Does Chrome just require a valid doctype and layout? Great answer though, will be bookmarking this !
@Christian it works like bad, i will use css and make the progress bar look clean. thank you so much
|
1

I'm assuming these numbers are in a database, if so you should add a new column titled isSent (or whatever you fancy).

This next paragraph you typed should be queued and possibly done night/weekly/whenever appropriate. Unless you have a specific reason too, it shouldn't be done in bulk on demand. You can even add a column to the db to see when it was last checked so that if a number hasn't been checked in at least X days then you can perform a check on that number on demand.

Processing of the data involves checking mobile number type (e.g CDMA), assigning unique ids to all the numbers for further referencing, check for network/country unique charges, etc.

But that still leads you back to the same question of how to do this for 50,000 numbers at once. Since you mentioned cron jobs, I'm assuming you have SSH access to your server which means you don't need a browser. These cron jobs can be executed via the command line as such:

/usr/bin/php /home/username/example.com/myscript.php

My recommendation is to process 1,000 numbers at a time every 10 minutes via cron and to time how long this takes, then save it to a DB. Since you're using a cron job, it doesn't seem like these are time-sensitive SMS messages so they can be spread out. Once you know how long it took for this script to run 50 times (50*1000 = 50k) then you can update your cron job to run more/less frequently.

$time_start = microtime(true);
set_time_limit(0);

function doSendSMS($phoneNum, $msg, $blah);

$time_end = microtime(true);
$time = $time_end - $time_start;
saveTimeRequiredToSendMessagesInDB($time);

Also, you might have noticed a set_time_limit(0), this will tell PHP to not timeout after the default 30seconds. If you are able to modify the PHP.ini file then you don't need to enter this line of code. Even if you are able to edit the PHP.ini file, I would still recommend not changing this feature since you might want other pages to time out.

http://php.net/manual/en/function.set-time-limit.php

Improve this answer

Comments

1

If this isn't a one-off type of situation, consider engineering a better solution.

What you basically want is a queue that your browser-bound process can write to, and than 1-N worker processes can read from and update.

Putting work in the queue should be rather inexpensive - perhaps a bunch of simple INSERT statements to a SQL RDBMS.

Then you can have a daemon or two (or 100, distributed across multiple servers) that read from the queue and process stuff. You'll want to be careful here and avoid two workers taking on the same task, but that's not hard to code around.

So your browser-bound workflow is: click some button that causes a bunch of stuff to get added to the queue, then redirect to some "queue status" interface, where the user can watch the system chew through all their work.

A system like this is nice, because it's easy to scale horizontally quite a ways.

EDIT: Christian Sciberras' answer is going in this direction, except the browser ends up driving both sides (it adds to the queue, then drives the worker process)

Improve this answer

Comments

0

Cronjob would be your best bet, I don't see why it would take any longer than doing it in the browser if your only problem at the moment is the browser timing out.

If you insist on doing it via the browser then the other solution would be doing it in batches of say 1000 and redirecting to the same script but with some reference to where it got up to last time in a $_GET variable.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.