PHP MongoDB insert using loop

Question

I am having some trouble working with MongoDB in PHP at the moment.

I am pulling records of financial data from a CSV file almost a gig, I am looping through the file fine and outputting and parsing the array.

During the while loop I am also trying to insert the data in to MongoDB

// Increase timeout on php script
ini_set('max_execution_time', 600);

while (($data = fgetcsv($file, 0, ",")) !==FALSE) {

    $parsedData['name'] = $data['0'];
    $parsedData['email'] = $data['1'];
    $parsedData['phone'] = $data['2'];
    $parsedData['address'] = $data['3'];
    $parsedData['gender'] = $data['4'];

    $collection->insert($parsedData);

}

So the problem is that it inserts only one of the records or a few, I can't really say it seems quite random.

Any help here would be great.

Tests Completed

Running the same function while testing with mysql returned successful.
print_r($parsedData) displays desired values.
Wrapping $collection->insert in an if statement returns true

Are you running this from command line? Because depending on your server there is generally a timeout of 30 seconds. — Livewire
– Livewire, Commented Jul 29, 2015 at 21:46
I am curling this and also have the following at the top of the model ini_set('max_execution_time', 600); — dcdcdc
– dcdcdc, Commented Jul 29, 2015 at 21:49
@Livewire update timeout seconds, even if it was timing out I should be getting more than one record in the db? — dcdcdc
– dcdcdc, Commented Jul 29, 2015 at 21:52
Without seeing anything, I can only guess the data file has bad line breaks or line breaks the PHP function fgetcsv() cannot parse. — kalinma
– kalinma, Commented Jul 29, 2015 at 21:56

dcdcdc · Accepted Answer · 2015-08-01 12:48:19Z

3

Okay so I managed to resolve this issue after reading more on some MongoDB documentation.

I wrapped the procedure with a try and catch adding an exception
Added fsync and safe to the array that was sent to MongoDB

The final piece added was "new MongoId" as MongoDB was returning duplicate _id (as far as I know this was the only necessary step to take)

while (($data = fgetcsv($file, 0, ",")) !==FALSE) {
    try{ 

        // Add MongoId, without this it was returning a duplicate key 
        // error in the catch.
        $parsedData['_id'] =  new MongoId();

        $parsedData['name'] = $data['0'];
        $parsedData['email'] = $data['1'];
        $parsedData['phone'] = $data['2'];
        $parsedData['address'] = $data['3'];
        $parsedData['gender'] = $data['4'];

        // Submitted "safe" and "fsync" with the array, as far as I
        // can see MongoDB waits till data is entered before it sends
        // a true response instead of continuing after the function is 
        // executed.
        $collection->save($parsedData, array('safe' => true, 'fsync' => true));


    }catch(MongoCursorException $e){
        // This is where I caught the duplicate id
        print_r($e->doc['err']);

        // Kill the procedure
        die();
    }
}

If anyone can add to this it would be great as I thought Mongo generated its own id's and that it would only return true when data is entered or maybe I'm just expecting it to run similar to the MySQL drivers.

edited Aug 1, 2015 at 12:48

answered Jul 30, 2015 at 8:34

dcdcdc

2633 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

kalinma Over a year ago

I always appreciate it when someone who finds their own answer posts it for the benefit of others. Good to know about the difference between Mongo and MySQL regarding unique ids.

Collectives™ on Stack Overflow

PHP MongoDB insert using loop

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related