how to fetch and validate csv header in open csv?

Question

I want to fetch header from csv file . If I am not use this skipLines then I will get header at 0 index array . But I want to fetch header directly using HeaderColumnNameMappingStrategy but it will not work with my code.

I also want to validate header column list ( like csv had not allowed to contain extra column)

I had also check this How to validate the csv headers using opencsv but it was not helpful to me.

@SuppressWarnings({ "unchecked", "rawtypes" })
public Map<String, Object> handleStockFileUpload(MultipartFile file, Long customerId) {
    Map<String, Object> responseMap = new HashMap<>();
    responseMap.put("datamap", "");
    responseMap.put("errormap", "");
    responseMap.put("errorkeys", "");

    List<Map<String, Integer>> list = new ArrayList<>();
    List<StockCsvDTO> csvStockList = new ArrayList<>();

    try {
        String fileName = new SimpleDateFormat("yyyy_MM_dd_HHmmss").format(new Date()) + "_" + file.getOriginalFilename();
        responseMap.put("filename", fileName);

        File stockFile = new File(productsUploadFilePath + fileName);
        stockFile.getParentFile().mkdirs();
        FileOutputStream fos = new FileOutputStream(stockFile);
        fos.write(file.getBytes());
        fos.close();


        CsvTransfer csvTransfer = new CsvTransfer();


        ColumnPositionMappingStrategy ms = new ColumnPositionMappingStrategy();
        ms.setType(StockCsv.class);

        Reader reader = Files.newBufferedReader(Paths.get(productsUploadFilePath + fileName));
        CSVReader csvReader =  new CSVReader(reader);

        CsvToBean cb = new CsvToBeanBuilder(reader)
          .withType(StockCsv.class)
          .withMappingStrategy(ms)
          .withSkipLines(1)
          .build();

       csvTransfer.setCsvList(cb.parse());
       reader.close();


       csvStockList = csvTransfer.getCsvList();

    } catch (Exception e) {
        e.printStackTrace();
        responseMap.put("status", "servererror");
    }

     responseMap.put("datamap", csvStockList);

    return responseMap;
}

must you use Open Csv or any other way of validating the headers would be fine — Thomas Mwania
– Thomas Mwania, Commented May 9, 2019 at 12:05

Maksim Karatkou · Accepted Answer · 2023-07-04 09:02:40Z

1

I found the following solution:

Use @CsvBindByName with HeaderColumnNameMappingStrategy,e.g. annotate your bean properties with @CsvBindByName:

    public static class HollywoodActor {
        private int id;
        @CsvBindByName(column = "First Name")
        private String firstName;
        @CsvBindByName(column = "Last Name")
        private String lastName;
    // getter / setter
    }

Add a method like this:

    public class CsvParser {

        public <T> ParseResult<T> parseByPropertyNames(Reader csvReader, Class<T> beanClass) throws IOException {
            CSVReader reader = new CSVReaderBuilder(csvReader).withCSVParser(new 
 CSVParserBuilder().build()).build();
            CsvToBean<T> bean = new CsvToBean();
            HeaderColumnNameMappingStrategy<T> mappingStrategy = new HeaderColumnNameMappingStrategy();
            mappingStrategy.setType(beanClass);
            bean.setMappingStrategy(mappingStrategy);
            bean.setCsvReader(reader);
            List<T> beans = bean.parse();
            return new CsvParseResult<>(mappingStrategy.generateHeader(), beans);
        }

and also don't forget to add public class ParseResult

    public class ParseResult <T> {
      private final String[] headers;
      private final List<T> lines;
      // all-args constructor & getters
    }

Use them in your code:

    String csv = "Id,First Name,Last Name\n" + "1, \"Johnny\", \"Depp\"\n" + "2, \"Al\", \"Pacino\"";
    CsvParseResult<HollywoodActor> parseResult = parser
                .parseByPropertyNames(new InputStreamReader(new ByteArrayInputStream(csv.getBytes(StandardCharsets.UTF_8), HollywoodActor.class)));

From ParseResult.headers you can get actual headers from which were in your .csv file. Just compare them with what's expected.

Hope that helps!

edited Jul 4, 2023 at 9:02

answered Nov 28, 2019 at 16:08

Maksim Karatkou

113 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Abhinash Jha Over a year ago

have to pass a Type to this call: mappingStrategy.generateHeader(). Can you please explain what it should be as HeaderColumnNameMappingStrategy is initialized with type T.

Abhinash Jha Over a year ago

Also, I am not using CsvReader, but parsing an incoming csv using CsvToBean, so can i use this approach.

Zenny · Accepted Answer · 2019-05-20 06:04:25Z

Here I was comparing my csvHeader with originalHeader:

List<String> originalHeader = fileUploadUtility.getHeader(new StockCsv());

List<String> invalidHeader = csvHeader.stream().filter(o -> (originalHeader.stream().filter(f -> f.equalsIgnoreCase(o)).count()) < 1).collect(Collectors.toList());
            if(null != invalidHeader && invalidHeader.size() > 0 && invalidHeader.toString().replaceAll("\\[\\]", "").length() > 0) {
                msg = "Invalid column(s) : " + invalidHeader.toString().replace(", ]", "]") + ". Please remove invalid column(s) from file.";
                resultMap.put(1, msg);
            }


 public List<String> getHeader(T pojo) {
    // TODO Auto-generated method stub
    final CustomMappingStrategy<T> mappingStrategy = new CustomMappingStrategy<>();
    mappingStrategy.setType((Class<? extends T>) pojo.getClass());
    String header[] = mappingStrategy.generateHeader();
    List<String> strHeader = Arrays.asList(header);
    return strHeader;
  }

marc_s · Accepted Answer · 2020-08-01 13:33:12Z

Here is an alternative to your present problem.First, define what you expect your headers to look like. For example:

public static final ArrayList<String> fileFormat = new ArrayList<> (Arrays.asList("Values1", "Values2", "Values3", "Values4")); Now, write a method to return custom errors if any exist:

public String validateCsvFileDetails(MultipartFile file, Set<String> requiredHeadersArray) {
    Set<String> errors = new HashSet<>();
    try {
        InputStream stream = file.getInputStream();
        BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
        String headerLine = reader.readLine();
        if (Objects.isNull(headerLine))
            return "The file has no headers, please ensure it has the correct upload format";
        List<String> headersInFileList;
        String[] headersInFileArray;
        if (headerLine.contains(",")) {
            headersInFileArray = StringUtils.split(headerLine, ",");
            headersInFileList = Arrays.asList(headersInFileArray);
        } else//the headerline has only one headerfield
        {
            headersInFileList = Collections.singletonList(headerLine);
        }
        for (String header : requiredHeadersArray) {
            if (!headersInFileList.contains(header))
                errors.add("The file has the wrong header format, please ensure " + header + " header is present");
        }
        //if there are errors, return it
        if (!errors.isEmpty())
            return sysUtils.getStringFromSet(errors);
        //Ensure the csv file actually has values after the header, but don't read beyond the first line
        String line;
        int counter = 0;

        while ((line = reader.readLine()) != null) {
            counter++;
            if (counter > 0)
                break;
        }
        //if line is null return validation error
        if (Objects.isNull(line))
            return "Cannot upload empty file";
    } catch (Exception e) {
        logger.error(new Object() {
        }.getClass().getEnclosingMethod().getName(), e);
        return "System Error";
    }

    return null;

}

Now you can validate you file headers as follows:

  String errors = validateCsvFileDetails(file, new HashSet<>(fileFormat));
     if (errors != null)
        return error
    //proceed

Philip Chan · Accepted Answer · 2021-05-05 16:20:43Z

Give this a try using captureHeader as a pre-filter: ...

private class CustomHeaderColumnNameMappingStrategy<T> extends HeaderColumnNameMappingStrategy {
    private String[] expectedHeadersOrdered = {"Column1", "Column2", "Column3", "Column4", "Column5"};
    @Override
    public void captureHeader(CSVReader reader) throws IOException, CsvRequiredFieldEmptyException {
        String[] actualCsvHeaders = reader.peek();
        String actualHeader, expectedHeader;
        if (expectedHeadersOrdered.length > actualCsvHeaders.length) {
            throw new CsvRequiredFieldEmptyException("Missing header column.");
        } else if (expectedHeadersOrdered.length < actualCsvHeaders.length) {
            throw new IOException("Unexpected extra header column.");
        }
        // Enforce strict column ordering with index
        // TODO: you might want to employ simple hashMap, List, set, etc. as needed
        for (int i=0; i<actualCsvHeaders.length; i++) {
            actualHeader = actualCsvHeaders[i];
            expectedHeader = expectedHeadersOrdered[i];
            if ( ! expectedHeader.equals(actualHeader) ) {
                throw new IOException("Header columns mismatch in ordering.");
            }
        }

        super.captureHeader(reader); // Back to default processing if the headers include ordering are as expected
    }
}


    CustomHeaderColumnNameMappingStrategy yourMappingStrategy = new CustomHeaderColumnNameMappingStrategy<YourPOJO>();
    ourMappingStrategy.setType(YourPOJO.class);
    try {
        pojosFromCsv = new CsvToBeanBuilder<YourPOJO>(new FileReader(csvFile))
                .withType(YourPOJO.class)
                .withMappingStrategy(yourMappingStrategy)
                .build();
        pojosFromCsv.stream();

}

Inspired by Using captureHeader in OpenCSV

Cristiano Almeida · Accepted Answer · 2025-02-03 21:04:25Z

0

You can get the headers names dynamically, by getting Schema (probably a CSVSchema), like this:

// Read the file with a empty schema (not hardcoded headers names).
MappingIterator<Map<String, String>> data =
      new CsvMapper()
          .readerForMapOf(String.class)
          .with(CsvSchema.emptySchema().withHeader())
          .readValues(file);

((CsvSchema)data.getParser().getSchema()).getColumnNames();

The method .getColumnNames() returns a list of headers name.

answered Feb 3 at 21:04

Cristiano Almeida

881 silver badge8 bronze badges

Collectives™ on Stack Overflow

how to fetch and validate csv header in open csv?

5 Answers 5

2 Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

2 Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related