Send utf-8 encoded strings in Android HTTP request

Question

I have made a HTTP-post inside my android application. Values are sent as strings from my app to my webserver. Problem is, the values are not in UTF-8 as I want them to be. My webserver has UTF-8 encoding so I know that there is code inside my app that I need to change. See my snippet below:

private void sendPostRequest(String facebookId, String name, String email) {

        class SendPostReqAsyncTask extends AsyncTask<String, Void, String>{

            @Override
            protected String doInBackground(String... bcs) {

                String bcFacebookId = bcs[0];
                String bcName = bcs[1];
                String bcEmail = bcs[2];



                HttpClient httpClient = new DefaultHttpClient();

                HttpPost httpPost = new HttpPost("URL");

                BasicNameValuePair facebookIdBasicNameValuePair = new BasicNameValuePair("bcFacebookId", bcFacebookId);
                BasicNameValuePair nameBasicNameValuePair = new BasicNameValuePair("bcName", bcName);
                BasicNameValuePair emailBasicNameValiePair = new BasicNameValuePair("bcEmail", bcEmail);

                List<NameValuePair> nameValuePairList = new ArrayList<NameValuePair>();
                nameValuePairList.add(facebookIdBasicNameValuePair);
                nameValuePairList.add(nameBasicNameValuePair);
                nameValuePairList.add(emailBasicNameValiePair);
                try {

                    UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nameValuePairList);

                    httpPost.setEntity(urlEncodedFormEntity);

                    try {
                        HttpResponse httpResponse = httpClient.execute(httpPost);

                        InputStream inputStream = httpResponse.getEntity().getContent();

                        InputStreamReader inputStreamReader = new InputStreamReader(inputStream);

                        BufferedReader bufferedReader = new BufferedReader(inputStreamReader);

                        StringBuilder stringBuilder = new StringBuilder();

                        String bufferedStrChunk = null;

                        while((bufferedStrChunk = bufferedReader.readLine()) != null){
                            stringBuilder.append(bufferedStrChunk);
                        }

                        return stringBuilder.toString();

                    } catch (ClientProtocolException cpe) {

                        cpe.printStackTrace();
                    } catch (IOException ioe) {
                        System.out.println("Second Exception caz of HttpResponse :" + ioe);
                        ioe.printStackTrace();
                    }

                } catch (UnsupportedEncodingException uee) {
                    System.out.println("An Exception given because of UrlEncodedFormEntity argument :" + uee);
                    uee.printStackTrace();
                }

                return null;
            }

For an example, the letter 'ö' becomes a '?'. How do I fix this? Cheers!

AgilePro · Accepted Answer · 2013-05-18 15:06:55Z

7

The biggest single reason that characters get converted into question marks is the conversion of characters to bytes, and then back into characters, not matching.

The code you have supplied has this line:

InputStreamReader inputStreamReader = new InputStreamReader(inputStream);

This is problematic because you are not specifying how to convert the bytes into characters. Instead you probably want this:

InputStreamReader inputStreamReader = new InputStreamReader(inputStream, "UTF-8");

What you specify for the character encoding will depend upon the character encoding that you have specified elsewhere. Without specifying the character encoding, you will get the "default" character encoding, and that depends upon settings in both the client and the server. Java uses Unicode, and UTF-8 is the only encoding that will preserve all the characters that Java allows.

For debugging, you may want to use the InputStream and retrieve bytes from that, and print out the byte values, in order to verify that they are indeed UTF-8 encoded representations of the original character values. The proper encoding of 'ö' (x00F6) is 'Ã¶' (x00C3 x00B6).

You will also need to assure that the original POST request is properly UTF-8 encoded. The UrlEncodedFormEntity class also uses the default character encoding, which might not be UTF-8. Change this:

UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList);

to

UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList, "UTF-8");

edited May 18, 2013 at 15:06

answered May 18, 2013 at 14:38

AgilePro

5,6904 gold badges38 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

AgilePro Over a year ago

sorry that didn't work. I would suggest debug statement that output the byte values, so that you can determine whether the stream you are reading is properly encoded, and to determine what that encoding is.

Klaus Over a year ago

@Jack How did you work around it? I have the same issue here.

Petr · Accepted Answer · 2014-12-22 20:39:36Z

0

if database coding is set properly + table coding is set properly + columns coding set properly, then all data are stored properly. That's the first part. Now the second, important part - make sure you have this command after your mysql connection : SET NAMES utf8

This was my case for the same issue. Hope this this will work for you as well.

answered Dec 22, 2014 at 20:39

Petr

112 bronze badges

Collectives™ on Stack Overflow

Send utf-8 encoded strings in Android HTTP request

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related