Perl Split Binary Data into Chunks with Unpack

Question

I'm using Transfer-Encoding: chunked to write an HTTP response.

The response is split into pieces via the following:

my $template = "a$buffer_size" x int(length($response)/$buffer_size) . 'a*';

foreach my $buffer (unpack $template, $response){
    ...
}

This works fine when the content type is text/html, but it is corrupting binary data, such as application/pdf.

Can unpack be used to split binary data into equal lengths?

When you say that the content type is causing the corruption, have you confirmed that by trying to access the same data, send with different MIME types? — Dancrumb
– Dancrumb, Commented Jun 1, 2012 at 12:58
@Dancrumb the "corruption" is that the file is incorrectly flagged as UTF-8 and not ANSI. Strange thing is if I simulate unpack with grep {/\S/} split /(.{$buffer_size})/, everything is fine. Also, the unpack logic is OK if done at the command line (i.e., not via mod_perl or ActiveState PerlEx). — xpsd300
– xpsd300, Commented Jun 1, 2012 at 13:43
Edit your question to provide your full code and input data. When you just paraphrase parts of it in English prose, it is impossible to help you because we are not able to reproduce the problem. — daxim
– daxim, Commented Jun 1, 2012 at 14:02
Using the Plack streaming interface for chunked encoding: codepad.org/531qBASm — daxim
– daxim, Commented Jun 1, 2012 at 14:36

xpsd300 · Accepted Answer · 2012-06-01 19:45:52Z

1

Still not sure why unpack is failing in this context, but I stumbled upon a solution.

If I manipulate the response with an in-memory file, unpack works correctly:

my $resp;
open (my $fh, '>', \$resp);
my $fh_old = select($fh);
print $response;
close $fh;
select($fh_old);
$response = $resp;

Any insight into why this works?

answered Jun 1, 2012 at 19:45

xpsd300

1753 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

xpsd300 Over a year ago

Finally tracked down the root of the problem. The PDF file was being created on-the-fly from a unicode database. I was converting wide characters into cp1252 to match WinAnsiEncoding of the font object. Perl switched into utf8 mode when it encountered a wide character. This caused unpack to fail since it was set to operate in ascii mode.

ikegami · Accepted Answer · 2012-06-01 16:32:34Z

0

That works perfectly fine with binary data. The problem is elsewhere. (Did you binmode all relevant handles?)

answered Jun 1, 2012 at 16:32

ikegami

391k17 gold badges291 silver badges555 bronze badges

Collectives™ on Stack Overflow

Perl Split Binary Data into Chunks with Unpack

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related