Apply array_map on column of the array

Question

I was trying to get a solution to speed up the process of PHP. We are running into execution time issues on a specific page. We have an array with around 10.000 rows and we need to apply several callback functions on some of the columns of that array.

What is the best way and the fastest execution possible to go over an array and only apply a callback on several columns.

<?php
$records = [
    ['id' => 2135, 'first_name' => 'John', 'price' => 1000, 'unit' => 5, 'discount' => 30],
    ['id' => 3245, 'first_name' => 'Sally', 'price' => 2000, 'unit' => 8, 'discount' => 80],
    ['id' => 5342, 'first_name' => 'Jane', 'price' => 4000, 'unit' => 5, 'discount' => 34],
    ['id' => 5623, 'first_name' => 'Peter', 'price' => 1500, 'unit' => 4, 'discount' => 25]
];
function simpleMultiply($value)
{
    return $value * 2;
}

$applyToColumn = ['price','unit','discount'];

// $expectedRecords = array_map('simpleMultiply', array_column($records, 'id'));


$expectedRecords = [
    ['id' => 2135, 'first_name' => 'John', 'price' => 2000, 'unit' => 10, 'discount' => 60],
    ['id' => 3245, 'first_name' => 'Sally', 'price' => 4000, 'unit' => 16, 'discount' => 160],
    ['id' => 5342, 'first_name' => 'Jane', 'price' => 8000, 'unit' => 10, 'discount' => 68],
    ['id' => 5623, 'first_name' => 'Peter', 'price' => 3000, 'unit' => 8, 'discount' => 50]
];
?>

yes, we retrieve the data from SQL DB, we convert the result set into an HTML table to display, the example provided is simplified, but the simple multiply is actually several lines long with different behaviours. — Jan Van Looveren
– Jan Van Looveren, Commented May 1, 2016 at 11:53
There is more then only calculation going on inside the function call that i cannot apply inside the database query — Jan Van Looveren
– Jan Van Looveren, Commented May 1, 2016 at 12:01
"We have an array" - Do you have to keep the complete data set in memory for those calculations? — VolkerK
– VolkerK, Commented May 1, 2016 at 12:37

Barmar · Accepted Answer · 2016-05-01 12:45:32Z

2

array_column is an expensive operation if the array is large, because it has to create an entirely new array. Calling it repeatedly for different columns will multiply that expense. I suggest you just use a simple foreach loop.

$expectedRecords = array();
foreach ($records as $r) {
    foreach ($applyToColumn as $col) {
        $r[$col] = simpleMultiply($r[$col]);
    }
    $expectedRecords[] = $r;
}

If you can modify the original $records instead of creating a new $expectedRecords, you can use a reference in the first foreach.

foreach ($records as &$r) {
    foreach ($applyToColumn as $col) {
        $r[$col] = simpleMultiply($r[$col]);
    }
}

answered May 1, 2016 at 12:45

Barmar

789k57 gold badges554 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

VolkerK · Accepted Answer · 2016-05-01 22:32:43Z

You can wrap the result set in an iterator (or a generator, which looks rather similar to an iterator) that modifies the current row before returning it as the iteration result, e.g.

<?php
class Foo extends IteratorIterator {
    public function current() {
        $rv = parent::current();
        if ( $rv ) {
            // ... and more checks here ....
            // this example just assumes the fields/elements exist
            $rv['x'] *= 17;
            return $rv;
        }
    }
}

$pdo = new PDO('mysql:host=localhost;dbname=test;charset=utf8', 'localonly', 'localonly', array(
    PDO::ATTR_EMULATE_PREPARES=>false,
    PDO::MYSQL_ATTR_DIRECT_QUERY=>false,
    PDO::ATTR_ERRMODE=>PDO::ERRMODE_EXCEPTION
));
setup($pdo);


// the point is not to have the complete result set in memory
// see: e.g. php.net/mysqlinfo.concepts.buffering (or whatever database you use ;-) )
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$result = new Foo( $pdo->query('SELECT x,y FROM sofoo') );

foreach( $result as $row ) {
    echo join(', ', $row), PHP_EOL;
}


function setup($pdo) {
    $pdo->exec('
        CREATE TEMPORARY TABLE sofoo(
            id int auto_increment, # I just throw this in casually....
            x int,
            y int,
            primary key(id)
        )
    ');
    $stmt = $pdo->prepare('INSERT INTO sofoo (x,y) VALUES (?,?)');
    foreach( range(1,5) as $x ) {
        $stmt->execute( array($x, $x*13) );
    }
}

or (without most of the boilerplate, using a generator - and invoking the anonymous function immediately)

$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$result = (function($iter) {
    foreach( $iter as $row ) {
        $row['x'] *= 17;
        yield $row;
    }
})($pdo->query('SELECT x,y FROM sofoo') );


foreach( $result as $row ) {
    echo join(', ', $row), PHP_EOL;
}

You can build this "system" arbitrarily complex. Maybe passing the field names and functions to modify the elements to the generator like

$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$result = my_generator(
    $pdo->query('SELECT x,y FROM sofoo'),
    array('x'=>function($e) { return $e*17; })
);
foreach( $result as $row ) {
    echo join(', ', $row), PHP_EOL;
}

function my_generator($iter, $modifiers) {
    foreach( $iter as $row ) {
        foreach( $modifiers as $field=>$func) {
            $row[$field] = $func($row[$field]);
        }
        yield $row;
    }
}

or add some chaining mechanism so that multiple functions can modify the same field. Or passing the complete row to the functions so they can change the structure as well. Or ... or ...

Collectives™ on Stack Overflow

Apply array_map on column of the array

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related