My Java program doesn't iterate through my entire CSV file

Question

Code:

import java.io.BufferedReader;
import java.io.FileReader;
import java.util.Scanner;

public class dataReader {

    @SuppressWarnings("rawtypes")
    public static void main(String[] args) throws Exception {
        String splitBy =",";
        BufferedReader br = new BufferedReader(new FileReader("C:/practice/testData.csv"));
        String line = br.readLine();
        int counter = 0;

        while ((line = br.readLine()) != null){
            counter++;
            String[] b = line.split(splitBy);

            for (int x = 0; x < b.length; x++){
                System.out.println(b[x]);
            }


        }
        System.out.println(counter);
        br.close();
    }   
}

When I run it, it goes through the CSV and displays some output but it starts with product ID 4000+. It basically only outputs results for the last thousand rows in the CSV. I'm wrangling with a really ugly CSV file to try to get some useful data out of it so that I can write all of it into a new database later on. I'd appreciate any tips.

You're using System.out.println to output, and only seeing the last ~1000 rows. Are you sure the issue isn't your console limit? Seems bold to scroll that far back. — Kevin
– Kevin, Commented Nov 10, 2015 at 8:02
This is exactly it. I add a counter variable so that the while loop runs only 1000 times, and I get the output from the beginning of the file too. Thanks! — HandleThatError
– HandleThatError, Commented Nov 10, 2015 at 8:07
Kevin, do you want to write that as an answer so I can accept it as the official answer? — HandleThatError
– HandleThatError, Commented Nov 10, 2015 at 8:09

Kevin · Accepted Answer · 2015-11-10 08:17:10Z

2

As in the comments, the issue is that System.out.println prints to your console. The console will have a limit to the number of lines it can display, which is why you see the last ~1000 lines.

answered Nov 10, 2015 at 8:17

Kevin

1,62816 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

glethien · Accepted Answer · 2015-11-10 07:58:43Z

2

At a frist glance at your code, you skip the first line by assigning a readline() to your string line

String line = br.readLine();

You should change that as you read the first line and than enter the loop which will read the 2nd line before any operations on the first line took place.

Try something like

String line = "";

answered Nov 10, 2015 at 7:58

glethien

2,4712 gold badges30 silver badges39 bronze badges

1 Comment

HandleThatError Over a year ago

This solves an issue with discarding the first row. The real problem was the fact that the console only displays ~1000 rows at a time, like the gentleman in the comments mentioned. Thanks for the input!

Matthew Gunn · Accepted Answer · 2015-11-10 08:25:45Z

2

Your code won't handle more complicated CSVs, eg a row:

"Robert, John", 25, "Chicago, IL"

is not going to get parsed correctly. Just splitting on ',' is going to separate "Robert, John" into multiple cells though it should be just 1 cell.

The point is, you shouldn't be writing a CSV parser. I highly recommend go downloading the Apache Commons CSV library and using that! Reading CSVs is a solved problem; why reinvent the wheel?

answered Nov 10, 2015 at 8:25

Matthew Gunn

4,5291 gold badge15 silver badges31 bronze badges

2 Comments

HandleThatError Over a year ago

I just started having that type of problem. My parser stopped grabbing half of the lines once they started having extra commas. Thanks for the heads up, I'll check out that library.

HandleThatError Over a year ago

Wow, thanks so much. That library is amazing and it's working beautifully. I didn't even know I needed it until I saw firsthand what happens when you rely on just a comma. It's amazing how much more productive I can become when more experienced people share their knowledge. And I look forward to doing the same thing for other newbs down the road. Thanks again! :)

npinti · Accepted Answer · 2015-11-10 07:57:21Z

1

Without seeing the structure of the file I can only speculate, but this line: String line = br.readLine(); needs to be changed to String line = "" or String line = null. As it currently stands, you are discarding your first line.

answered Nov 10, 2015 at 7:57

npinti

52.2k5 gold badges74 silver badges98 bronze badges

Collectives™ on Stack Overflow

My Java program doesn't iterate through my entire CSV file

4 Answers 4

Comments

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related