Reading multiple columns from large CSV files in PHP

Question

I need to read two columns from a large CSV file. The CSV has multiple columns and can sometimes have following properties:

~25,000 lines
Contain spaces and blank rows
Be uneven (some columns longer than others)

enter image description here

In the example CSV file above, I would be only interested in the codes in the "Buy" and "Sell" columns (columns A and D).

I have written the following code (warning: it's not very elegant) to iterate over all rows and read only the columns I require. I create strings as inputs for 1 large MYSQL query (as opposed to running many small queries).

<?php 
//Increase the allowed execution time 
set_time_limit(0);
ini_set('memory_limit','256M');
ini_set('max_execution_time', 0);     

//Set to detect the ending of CSV files
ini_set('auto_detect_line_endings', true);

$file = "test.csv";

$buy = $sold = ""; //Initialize empty strings

if (($handle = @fopen($file, "r")) !== FALSE) {

while (($pieces = fgetcsv($handle, 100, ",")) !== FALSE) {       

if ( ! empty($pieces[0]) ) {
    $buy .= $pieces[0] ." ";
} 

if ( ! empty($pieces[3]) ) {
   $sold .= $pieces[3] ." ";
} 
}

echo "Buy ". $buy ."<br>"; //Do something with strings...
echo "Sold ". $sold ."<br>";

//Close the file
fclose($handle);  
}

?>

My question is: is this the best way to perform such a task? The code works for smaller test files, but are there short comings I've overlooked in iterating over the CSV file like this?

Community · Accepted Answer · 2023-11-14 22:27:11Z

1

First, reading any large files is memory consuming if you store them in variables. You may check out reading large files(more than 4GB in unix)

Secondly, you can output the $buy & $sold on the while loop which might be more memory efficient in the way that those two variables are not saved on the memory.

Lastly, Use file seek method in php fseek documentation

edited Nov 14, 2023 at 22:27

CommunityBot

11 silver badge

answered Dec 29, 2013 at 20:43

Momin

86812 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reading multiple columns from large CSV files in PHP

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related