Unable to Parse Final Column from CSV File using Apache Commons CSV

Question

I have a CSV file with 19 columns in it (the last column is empty). I'm parsing the file via Apache Commons CSV. When I execute:

System.out.println("csvRecord.toString() = " + csvRecord.toString());

I get the following output:

csvRecord.toString() = CSVRecord [comment=null, mapping={Amount=6, ID=2, UID=4, Address=5, Carrier=17, CPL=14, F=12, GC=13, PIT=9, PP=11, PCIT=10, PT=1, SPI=3, SPI1=8, ST=7, Status=16, SOI=0, TN=18, TP=15}, recordNumber=6, values=[FB, S13 Wood Carving, B07, AG, bis, FB, 44.8, PE, AG XXXXXXX47, UNKNOWN, 39.95, 1.6, 4.1, , -2.2068006148899633, -0.46425251802390777, COMPLETED, US]]

As you can see, the last column TN=18 (which is actually column 19 when counting from one) is included in output of toString() method in the mapping section (but not in the values section).

However, when I convert csvRecord to map via the following code:

Map<String, String> csvRecordMap = csvRecord.toMap();

The last column is no longer present. And when I execute:

if (csvRecordMap.containsKey("TN"))

It returns false.

Based on my testing I've noticed that the toMap() method will not include an empty column if there are no non empty columns after it (even if the column is actually present in CSV file). If a column is empty but there are non empty columns after it, then it will be included in return value of toMap() method.

How can I force parser to return all columns -- either via toMap() method or by somehow extracting it from csvRecord directly as it is obviously present as shown in toString() method ?

Thanks!

Andreas · Accepted Answer · 2019-10-06 07:52:25Z

1

Loop through and add the missing names:

Map<String, String> map = csvRecord.toMap();
for (String name : csvParser.getHeaderNames())
    map.putIfAbsent(name, "");

Demo

CSVFormat csvFormat = CSVFormat.DEFAULT.withHeader();
try (CSVParser csvParser = csvFormat.parse(Files.newBufferedReader(Paths.get("test.txt")))) {
    System.out.println(csvParser.getHeaderNames());
    for (CSVRecord csvRecord : csvParser) {
        System.out.println(csvRecord);
        System.out.println("  toMap(): " + csvRecord.toMap());

        Map<String, String> map = csvRecord.toMap();
        for (String name : csvParser.getHeaderNames())
            map.putIfAbsent(name, "");
        System.out.println("  fixed  : " + map);
    }
}

test.txt

A,B,C,D
1,2,3,4
1,2,3
1,2
1
1,
1,,
1,,,
,,,4,,,

Output (with commons-csv-1.7.jar)

[A, B, C, D]
CSVRecord [comment='null', recordNumber=1, values=[1, 2, 3, 4]]
  toMap(): {A=1, B=2, C=3, D=4}
  fixed  : {A=1, B=2, C=3, D=4}
CSVRecord [comment='null', recordNumber=2, values=[1, 2, 3]]
  toMap(): {A=1, B=2, C=3}
  fixed  : {A=1, B=2, C=3, D=}
CSVRecord [comment='null', recordNumber=3, values=[1, 2]]
  toMap(): {A=1, B=2}
  fixed  : {A=1, B=2, C=, D=}
CSVRecord [comment='null', recordNumber=4, values=[1]]
  toMap(): {A=1}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=5, values=[1, ]]
  toMap(): {A=1, B=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=6, values=[1, , ]]
  toMap(): {A=1, B=, C=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=7, values=[1, , , ]]
  toMap(): {A=1, B=, C=, D=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=8, values=[, , , 4, , , ]]
  toMap(): {A=, B=, C=, D=4}
  fixed  : {A=, B=, C=, D=4}

answered Oct 6, 2019 at 7:52

Andreas

160k13 gold badges164 silver badges262 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

S.O.S Over a year ago

Thanks. Is there a way to force toMap() method to include empty columns? I suspect the issue stems from similar issue discussed in this thread: stackoverflow.com/q/29440357 No?

Andreas Over a year ago

@S.O.S "Is there a way ... ?" Yes, modify the code, it is open-source, so you have full access to it.

Collectives™ on Stack Overflow

Unable to Parse Final Column from CSV File using Apache Commons CSV

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related