How can i change charset encoding in HTTP response in Java

Question

I have to fetch some JSON object from a remote server and for that i am using this function which is working great except that for sometime some weird data is getting fetched which i believe is because it is using ASCII charset to decode.

Please find below thw method that i am using

public HttpResponse call(String serviceURL,String serviceHost,String namespace,String methodName,String payloadKey, String payloadValue) throws ClientProtocolException,IOException,JSONException
    {
            HttpResponse response = null;
            HttpContext HTTP_CONTEXT = new BasicHttpContext();
            HTTP_CONTEXT.setAttribute(CoreProtocolPNames.USER_AGENT, "Mozilla/5.0");
            HttpPost httppost = new HttpPost(serviceURL);
            httppost.setHeader("User-Agent",Constants.USER_AGENT_BROWSER_FIREFOX);
            httppost.setHeader("Accept", "application/json, text/javascript, */*");
            httppost.setHeader("Accept-Language","en-US,en;q=0.8");
            httppost.setHeader("Content-Encoding", "foo-1.0");
            httppost.setHeader("Content-Type", "application/json; charset=UTF-8");
            httppost.setHeader("X-Requested-With","XMLHttpRequest");
            httppost.setHeader("Host",serviceHost);
            httppost.setHeader("X-Foo-Target", String.format("%s.%s", namespace,methodName));
            /*Making Payload*/
            JSONObject objectForPayload = new JSONObject();
            objectForPayload.put(payloadKey, payloadValue);
            StringEntity stringentity = new StringEntity(objectForPayload.toString());
            httppost.setEntity(stringentity);
            response = client.execute(httppost);
            return response;


    }

All these headers that i am passing are correct and i have verified the same via inspect element in Google chrome or Firebug plugin if you are familiar with Mozilla.

Now the problem is that most of the time i am getting the readable data but sometimes i do get unreadable data.

I debugged using eclipse and noticed that the charset under wrappedEntity is showing as "US-ASCII". I am attaching a jpg for reference enter image description here

Can someone please tell me how can i change the charset from ASCII to UTF-8 of the response before i do response = client.execute(httppost); . PS:As you have noticed that i am passing charset=utf-8 in the header and that i have already verified using firebug and google chrome that i am passing the exact headers .

Please zoom in to see the image more clearly

Thanks in advance

bourne · Accepted Answer · 2013-08-04 20:43:45Z

10

i was able to resolve the issue just mentioning it for people that may face similar issue. after getting the response first get the entity by using HttpEntity entity = response.getEntity(); and since my response was a json object convert entity to string but using "UTF-8" something like this responseJsonObject = new JSONObject(EntityUtils.toString(entity,"UTF-8"));

previously i was just doing responseJsonObject = new JSONObject(EntityUtils.toString(entity));

answered Aug 4, 2013 at 20:43

bourne

1,2594 gold badges17 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

divillysausages Over a year ago

I had this same problem as well, where IOUtils.toString(entity.getContent(), "UTF-8") wasn't working either. Changing it to EntityUtils did the job

lscoughlin · Accepted Answer · 2013-08-04 20:14:40Z

2

I don't think it's a problem with your headers, I think it's a problem with your string. Just having the header say it's utf-8 doesn't mean the string you write is utf-8, and that depends a lot on how the string was encoded and what's in the "payloadValue"

That said, you can always re-encode the thing correctly before sending it across the wire, for example:

objectForPayload.put(payloadKey, payloadValue);
StringEntity stringentity = new StringEntity(
   new String(
      objectForPayload.toString().getBytes(),
      "UTF8"));

See if that works for you.

answered Aug 4, 2013 at 20:14

lscoughlin

2,41318 silver badges23 bronze badges

1 Comment

bourne Over a year ago

thanks for the reply but it did not work,but what i am confused because stringentity is just an json object which i am passing in the payload and whether i am encoding it in utf-8 or not should not matter.I have to somehow change the charset encoding of response i think which i am unable to do

geby · Accepted Answer · 2019-01-04 13:30:01Z

2

You may need to add an "Accept-Encoding"-header and set this to "UTF-8"

edited Jan 4, 2019 at 13:30

answered Aug 4, 2013 at 19:43

geby

3742 silver badges15 bronze badges

2 Comments

bourne Over a year ago

i tried that but it did not worked same result httppost.setHeader("Accept-Encoding","UTF-8");

Sanchi Girotra Over a year ago

It worked for me by using headers.put("Accept-Encoding", "UTF-8");

Julian Reschke · Accepted Answer · 2019-01-04 14:19:11Z

1

Just for the record: the "Content-Encoding" header field is incorrect - a correct server would reject the request as it contains an undefined content coding format.

Furthermore, attaching a charset parameter to application/json is meaningless.

answered Jan 4, 2019 at 14:19

Julian Reschke

42.5k8 gold badges103 silver badges102 bronze badges

Comments

Elmar Brauch · Accepted Answer · 2021-01-20 11:05:23Z

0

bourne already answered that in the above comment though.

Changing entity = IOUtils.toString(response.getEntity().getContent())
TO entity = EntityUtils.toString(response.getEntity(),"UTF-8")
did the trick.

edited Jan 20, 2021 at 11:05

Elmar Brauch

1,4169 silver badges22 bronze badges

answered Jan 20, 2021 at 7:37

Tanvir Rahman

1

Collectives™ on Stack Overflow

How can i change charset encoding in HTTP response in Java

5 Answers 5

1 Comment

1 Comment

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

1 Comment

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related