Java http - read large file

Question

I am reading a big file in a Java program, using http access. I read the stream, and then I apply some criteria. Would it be possible to apply the criteria on the read stream, so I will have a light result (I'm reading big files)?

Here is my code for reading the file:

public String getMyFileContent(URLConnection uc){
            String myresult = null;
            try {
                InputStream is = uc.getInputStream();
                InputStreamReader isr = new InputStreamReader(is);
                int numCharsRead;
                char[] charArray = new char[1024];
                StringBuffer sb = new StringBuffer();

                while ((numCharsRead = isr.read(charArray)) > 0) {
                    sb.append(charArray, 0, numCharsRead);
                }
                myresult = sb.toString();

            }
            catch (MalformedURLException e) {
                e.printStackTrace();
            }
            catch (IOException e) {
                e.printStackTrace();
            }
            return result;  
}

And in another method, I then apply the criteria (to parse the content). I couldn't achieve to do like this:

public String getMyFileContent(URLConnection uc){
    String myresult = null;
    try {
        InputStream is = uc.getInputStream();
        InputStreamReader isr = new InputStreamReader(is);
        int numCharsRead;
        char[] charArray = new char[1024];
        StringBuffer sb = new StringBuffer();

        while ((numCharsRead = isr.read(charArray)) > 0) {
            sb.append(charArray, 0, numCharsRead);
            //Apply my criteria here on the stream ??? Is it possible ???
        }
        myresult = sb.toString();
    }
    catch (MalformedURLException e) {
        e.printStackTrace();
    }
    catch (IOException e) {
        e.printStackTrace();
    }
    return myresult;
}

What is your criteria? And what kind of content is there in the file you are reading? — Aravind Yarram
– Aravind Yarram, Commented Jan 3, 2012 at 16:11
Well. It was difficult to understand the problem. But what I understood is: you want to process the stream insted of processing the big array. It's a good thing to do but it depends on the process. If the process you want to apply is capable of processing stream input, alright. So, please: what is the process you want to apply to your input? — helios
– helios, Commented Jan 3, 2012 at 16:12
Side comment: when converting bytes into chars it's a good practice to be explicit on the conversion (it is, the conversion). So if you use the InputStreamReader constructor that receives also the encoding (or encoding name, by example "utf-8") it will be better and not platform dependant. — helios
– helios, Commented Jan 3, 2012 at 16:13

Peter Lawrey · Accepted Answer · 2012-01-03 16:20:37Z

2

The template I would use is

InputStreamReader isr = new InputStreamReader(uc.getInputStream());
int numCharsRead;
char[] charArray = new char[1024];

while ((numCharsRead = isr.read(charArray)) > 0) {
    //Apply my criteria here on the stream
}

however since it is text, this might be more useful

InputStream is = uc.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(uc.getInputStream()));
String line;

while ((line = br.readLine()) != null) {
    //Apply my criteria here on each line
}

answered Jan 3, 2012 at 16:20

Peter Lawrey

535k83 gold badges770 silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Java http - read large file

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related