Apache HttpClient in Java, instream.toString = org.apache.http.conn.EofSensorInputStream

Question

I am GETting a page with Apache HttpClient and I want to store the server reply's http body into a string so I can then manipulate this string and print it to the console.

Unfortunately when running this method I get this message back:

17:52:01,862  INFO Driver:53 - fetchPage STARTING
17:52:07,580  INFO Driver:73 - fetchPage ENDING, took 5716
org.apache.http.conn.EofSensorInputStream@5e0eb724

The fetchPage Class:

public String fetchPage(String part){
    log.info("fetchPage STARTING");
    long start = System.currentTimeMillis();

    String reply;

    String searchurl = URL + URL_SEARCH_BASE + part + URL_SEARCH_TAIL;

    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(searchurl);
    HttpResponse response;
    try {
        response = httpclient.execute(httpget);
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            InputStream instream = entity.getContent();
            int l;
            byte[] tmp = new byte[2048];
            while ((l = instream.read(tmp)) != -1) {
            }
            long elapsedTimeMillis = System.currentTimeMillis()-start;
            log.info("fetchPage ENDING, took " + elapsedTimeMillis);
            reply = instream.toString();
            System.out.println(reply);
            return reply;
        }
    } catch (ClientProtocolException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    return null;
}

The key thing to understand here is that toString() on an InputStream is not a method to read its contents as a String, but rather to get a simple string representation of the object itself. Typically (including in this case) an InputStream doesn't have a useful string representation it can provide, so it just uses the default Object.toString(). — ColinD
– ColinD, Commented Jun 2, 2011 at 16:17

Piotr Niewinski · Accepted Answer · 2020-03-19 19:03:59Z

12

You are calling the toString on the InputStream after it has already read through. You need to create your string from the byte arrays. The simpler way to get the String version of the content is to use the EntityUtils.toString(HttpEntity)

The exact implementation would look like:

import org.apache.http.util.EntityUtils;

public String fetchPage(String part){
    log.info("fetchPage STARTING");
    long start = System.currentTimeMillis();

    String reply;

    String searchurl = URL + URL_SEARCH_BASE + part + URL_SEARCH_TAIL;

    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(searchurl);
    HttpResponse response;
    try {
        response = httpclient.execute(httpget);
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            return EntityUtils.toString(entity);
        }
    } catch (ClientProtocolException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    return null;
}

edited Mar 19, 2020 at 19:03

Piotr Niewinski

1,3372 gold badges15 silver badges29 bronze badges

answered Jun 2, 2011 at 16:05

Joshua

26.8k22 gold badges80 silver badges107 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dominik Over a year ago

If I add the long elapsedTimeMillis = System.currentTimeMillis()-start; right before return EntityUtils.toString(entity); the elapsed time for the procedure is about 1300ms, if I stop the time after String result = EntityUtils.toString(entity); and then return result it takes about 5500ms. Do you know why that is?

ColinD Over a year ago

@Jack: Probably because the client doesn't have to have received all the data the server is sending at the point where you get the entity. Creating the string requires actually reading all the content the server is sending.

Joshua Over a year ago

@Jack Murphy: the EntityUtils.toString(entity) is reading the stream. The client hasn't received all the data yet. You will want the latter to capture the amount of time of transfer.

Collectives™ on Stack Overflow

Apache HttpClient in Java, instream.toString = org.apache.http.conn.EofSensorInputStream

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related