concat string with binary: binary content got truncated when hit 0x00

Question

I want to concat some character strings and a jpeg binary file content to make a http post request in C under LINUX. If i use the normal string operation, the binary content in the jpeg will be terminated once it hits 0x00. (for example, if I have my binary file in HEX to be FF D8 FF C0 00 11 08...), after the concat, the binary part will end up with FF D8 FF C0.

Does anyone know how to solve this problem? The language is in C. Thanks

 fp = fopen(filename, "rb")
 fseek(fp, 0, SEEK_END);
 fileLen = ftell(fp);
 fseek(fp, 0, SEEK_SET);
 buffer = (char *)malloc(fileLen);
 fread(buffer, fileLen, 1, fp);  

 //now buffer has the content of the JPEG image

 //next combine both string and binary together to make a http post request

 snprintf(poststr, MAXSUB,
             "--%s\r\nContent-Disposition: form-data;"
             "name=\"datafile\"; filename=\"%s\"\r\nContent-Type: image/jpeg\r\n\r\n"
             "%s\r\n"
             "--%s\r\n"
             "Content-Disposition: form-data; name=\"boxkey\"\r\n\r\n%s\r\n"
             "--%s--", boundary, filename, buffer, boundary, key, boundary);

Community · Accepted Answer · 2021-10-07 10:58:19Z

2

Here is the discussion of your issue and a tip from me, let's see them both.

Copying binary data

A C string is terminated by a 0, so if you print with format specifier %s and pass a (binary) string with embedded 0, the first one will be recognized as the string terminator.

Instead, use memcpy, so that you can pass the size explcitly. Assuming that the size of poststr is big enough to accomodate everything, you should do:

int headLength = snprintf(poststr, MAXSUB,
         "--%s\r\nContent-Disposition: form-data;"
         "name=\"datafile\"; filename=\"%s\"\r\nContent-Type: image/jpeg\r\n\r\n"
     boundary, filename
);

// Declarations added for clarity, you can remove them
char *bodystr = poststr + headLength;
char *footerstr = bodystr + bufferLength;

// Copy binary here
memcpy(bodystr, buffer, bufferLength);

snprintf(footerstr, MAXSUB,
         "--%s\r\n"
         "Content-Disposition: form-data; name=\"boxkey\"\r\n\r\n%s\r\n"
         "--%s--", boundary, key, boundary
);

Please consider that the resulting string will have embedded 0, so you will have to calculate the "real" length and keep it somewhere (strlen will stop as well at the first embedded 0).

Encoding binary data in mime/multipart messages

I see that you are building a multipart HTTP request. You can encode binary data, mostly as BASE64 or quoted-printable. There are C libraries for both, but you could also achieve the same in 50-60 lines of code. You will have to specify what encoding you are using in the headers. See RFC 2045 §6.1 for a syntax reference.

In that case, your code will work, because data will be encoding as an ASCII string.

edited Oct 7, 2021 at 10:58

CommunityBot

11 silver badge

answered Jan 21, 2014 at 20:54

Stefano Sanfilippo

33.2k7 gold badges85 silver badges83 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Hugh H Over a year ago

Hi, thanks for your quick and clean reply! Do I have to encode the binary part in order to make it work? Or can I just leave them as what it is.

Stefano Sanfilippo Over a year ago

I suggest encoding, it will make sure that any client/server can handle your message without breaking it because of encoding or other issues. Your message will be plain ASCII. Also, use Base64 for encoding, because it's more compact than quoted-printable.

Hugh H Over a year ago

hmm, if I encoded my binary part using Based64, where do I decode it or the server will decode it for me?

Stefano Sanfilippo Over a year ago

Yes, you have to specify that your content is encoded as such using an header (see the RFC). This is common in e-mails. You could also use the first version (without encoding), which is what web browsers (e.g. Google Chrome) do. Surely you can leave your content as binary and use the code I posted above, the only real issue was the 0 terminator, your way of doing things is ok. Maybe start with that :)

abligh · Accepted Answer · 2014-01-21 20:58:27Z

If you want to append binary data onto a string, find the location of the terminating NULL, and memcpy the binary data onto there. For instance:

void
appendBinaryToString (char *string, void *binary, size_t length)
{
    /* IMPORTANT
     * Ensure the memory allocated for string has sufficient space
     * to permit the binary to be copied in
     */

    /* Offset of NULL is at strlen(string) */
    memcpy (string + strlen(string), binary, length;
}

Note however that the output will be a binary char array of length length + strlen(string) rather than a zero terminated string. Is that really what you want? If so, you might be better thinking of the question as 'how do I copy the binary content of a string, and another binary blob into adjacent memory locations?'. The reason I mention this is that I think you are trying to copy binary data into poststr then treating that as a string; perhaps you really want to encode the data (base64 or similar) and then add that; as encoded data does not include NUL bytes, you won't have the issue.

Depending on what library you are using to do the post, you may find it does the hard work for you.

Collectives™ on Stack Overflow

concat string with binary: binary content got truncated when hit 0x00

2 Answers 2

Copying binary data

Encoding binary data in mime/multipart messages

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Copying binary data

Encoding binary data in mime/multipart messages

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related