Java byte[] to String and UTF-8

Question

I am implementing a Cipher Block Chaining for school work and the question asks for a method taking String and returning another String. At first, I thought it was odd and that byte[] variables would be much more adequate, but implemented a method still. Basically, here's the code :

static public String encode(String message) {
   byte[] dataMessage = message.getBytes();
   ByteArrayOutputStream out = new ByteArrayOutputStream();

   byte last = (byte) (Math.random() * 256);
   byte cur;
   out.write(last);

   for (byte b : data) {
      cur = (byte) (b^last);
      System.out.println("Encode '" + (char) b + "' = " + b + "^" + last + " > " + cur );
      out.write( cur );
      last = cur;
   }

   System.out.println("**ENCODED BYTES = " + Arrays.toString(out.toByteArray()));
   System.out.println("**ENCODED STR   = " + Arrays.toString(out.toString().getBytes()));

   return out.toString();
}

The decode method works similarly. Some times, the method will spit results like

Encode 'H' = 72^109 > 37
Encode 'e' = 101^37 > 64
Encode 'l' = 108^64 > 44
Encode 'l' = 108^44 > 64
Encode 'o' = 111^64 > 47
**ENCODED BYTES = [109, 37, 64, 44, 64, 47]
**ENCODED STR   = [109, 37, 64, 44, 64, 47]

But sometimes will also spit things like

Encode 'H' = 72^-63 > -119
Encode 'e' = 101^-119 > -20
Encode 'l' = 108^-20 > -128
Encode 'l' = 108^-128 > -20
Encode 'o' = 111^-20 > -125
**ENCODED BYTES = [-63, -119, -20, -128, -20, -125]
**ENCODED STR   = [-17, -65, -67, -17, -65, -67, -17, -65, -67, -17, -65, -67]

I presume that this has something to do with UTF-8 (the system's default encoding), but I'm not familiar enough to figure out exactly why such a string would be returned with the given bytes.

JB Nizet · Accepted Answer · 2011-12-02 23:05:38Z

3

You can't take an arbitrary sequence of bytes and assume it's a valid UTF-8 encoded string. So, I suspect that the toString method, as documented, replaces malformed-input and unmappable-character sequences with the default replacement string for the platform's default character set.

You should thus not transform purely binary data into a String like this. Use some encoding like Hex or Base64 to transform your bytes to a printable string, and vice-versa.

Apache commons-codec has a Base64 utility class.

answered Dec 2, 2011 at 23:05

JB Nizet

694k94 gold badges1.3k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ninjalj Over a year ago

Yes, it replaces every character sequence (the four of them, UTF-8 has a self-synchronizing property that makes it skip to what looks like the start of the next multibyte character) is replaced by U+FFFD REPLACEMENT CHARACTER (in UTF8: 0xef 0xbf 0xbd).

Yanick Rochon Over a year ago

Yes, this is what I think happens (about the replacement character). And I will use a Base64 implementation then.

Jesper · Accepted Answer · 2011-12-02 23:07:35Z

0

This:

out.toString().getBytes()

is not doing what you expect. It takes the encrypted bytes and interprets those bytes as if they are an UTF-8 encoded string. Then it converts the characters in that string back to bytes.

You can't just take arbitrary bytes (in this case, the encrypted data) and then handle it as if it is UTF-8 encoded text.

answered Dec 2, 2011 at 23:07

Jesper

208k47 gold badges325 silver badges361 bronze badges

Collectives™ on Stack Overflow

Java byte[] to String and UTF-8

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related