php json encode large arrays gets truncated

Question

I am encountering a problem displaying json with json_encode. Problem is it gets truncated after a certain point. I'm guessing around 2500 lines. I've read a lot of q&a's in stackoverflow and others suggesting to increase memory limit. I've increased it already to 32m and it still gets truncated furthermore I don't think it's the problem for i've echoed the memory usage and with true i get 3.5mb and with false i get 1.5mb I also tried encoding it by 1000 but up until a certain point it still truncates.

my json looks like this

{"Id":"1"},{"Words":""},{"Position":"0"},{"Length":"0"},{"Pdf_Id":"1"}

just 2000 more

My code goes something like this:

json file

echo json_encode(array_values($db->getallqueryjsoncountStart("words_table","Allwords",4000,0)));


public function getallqueryjsoncountStart($table,$jsonparentname,$count,$start)
{

    $this->sqlquery = "select * from $table LIMIT $start,$count";
    $this->stmt = $this->conn->query($this->sqlquery);
    for ($i = 0; $i < $this->stmt->columnCount(); $i++) {
        $col = $this->stmt->getColumnMeta($i);
        $columns[] = $col['name'];
    }
    $this->initcnt = 0;
    $getqueryJson = array();

    while($this->row = $this->stmt->fetch()) 
    {
                for($x = 0;$x < sizeof($columns);$x++)
                {
                    $getqueryJson[][$columns[$x]] = $this->row[$columns[$x]];
                }
    }
    return $getqueryJson;
}

Maybe worth changing the order of records in the SQL query just to be sure that json_encode still breaks after 2500 lines, and not because of a problem with data to encode. — M. Page
– M. Page, Commented Nov 2, 2014 at 13:19
What is the output of json_last_error()? php.net/manual/en/function.json-last-error.php — robbmj
– robbmj, Commented Nov 2, 2014 at 13:22
changed the order still breaks at approx 2500 lines. json_last_error() returns 0. thanks for the replies — Andre malupet
– Andre malupet, Commented Nov 2, 2014 at 13:57

robbmj · Accepted Answer · 2014-11-03 23:36:51Z

5

If the "Words" column actually contains some text there maybe some illegal character contained within the value.

Add the following options to the json_encode function:

JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP

For a description of what each does see the manual page

Also. it looks like the way you are structuring the $getqueryJson array is incorrect. Try changing as follows

$getqueryJson = array();
while($this->row = $this->stmt->fetch()) 
{
    $row = array()
    for($x = 0;$x < count($columns);$x++)
    {
        $row[$columns[$x]] = $this->row[$columns[$x]];
    }
    $getqueryJson[] = $row;
}
return $getqueryJson;

Call the method like so.

$table_data = $db->getallqueryjsoncountStart("words_table","Allwords",4000,0);
echo json_encode($table_data, 
                 JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT | JSON_HEX_AMP);

This will produce a JSON string with the following format.

[{"Id":"1", "words":"", "Position":"0","Length":"0","Pdf_Id":"1"},
 {"Id":"2", "words":"", "Position":"0","Length":"0","Pdf_Id":"2"},
 {"Id":"3", "words":"", "Position":"0","Length":"0","Pdf_Id":"3"},
 ...
 {"Id":"4000", "words":"", "Position":"0","Length":"0","Pdf_Id":"4000"}]

If the problem persists, try calling json_last_error() right after calling json_encode.

edited Nov 3, 2014 at 23:36

answered Nov 2, 2014 at 13:45

robbmj

16.6k8 gold badges41 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Andre malupet Over a year ago

Dude this is awesome! I don't know that illegal characters could make json strings truncate. both works and I am having no problem now with even more than 10,000 records. case closed. thank you very much

Collectives™ on Stack Overflow

php json encode large arrays gets truncated

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related