Executing query inside spring batch processor

Question

In my project I need to execute a query inside Spring Batch processor in order to validate some field.

How can I do that?

EDIT: Adding sources:

This is definition of step:

@Bean
public Step step1(JdbcBatchItemWriter<CaricoDTO> step1Writer) {
    return stepBuilderFactory.get("step1").<CaricoDTO, CaricoDTO>chunk(10).reader(multiResourceItemReader())
            .processor(processorStep1()).writer(step1Writer).build();

}

This is definition of multiResourceItemReader:

@Bean
public MultiResourceItemReader<CaricoDTO> multiResourceItemReader() {
    MultiResourceItemReader<CaricoDTO> resourceItemReader = new MultiResourceItemReader<CaricoDTO>();
    ArrayList<Integer> indexesToRemove = new ArrayList<Integer>();
    Resource[] inputResources = null;
    PathMatchingResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
    try {
        inputResources = patternResolver.getResources(inputPath);
    } catch (IOException e) {
        e.printStackTrace();
    }

    resourceItemReader.setResources(inputResources);
    resourceItemReader.setDelegate(step1Reader());
    resourceItemReader.setComparator(new FileComparator());
    return resourceItemReader;
}

This is step1Reader:

@Bean
public FlatFileItemReader<CaricoDTO> step1Reader() {
    FlatFileItemReader<CaricoDTO> reader = new FlatFileItemReader<CaricoDTO>();
    reader.setLinesToSkip(1);
    reader.setLineMapper(new DefaultLineMapper<CaricoDTO>() {
        {
            setLineTokenizer(new DelimitedLineTokenizer("|") {
                {
                    setNames(new String[] { ..... });
                }
            });
            setFieldSetMapper(new BeanWrapperFieldSetMapper<CaricoDTO>() {
                {
                    setTargetType(CaricoDTO.class);
                }
            });
        }
    });
    return reader;
}

This is my processor bean:

@Bean
public CaricoDTOItemProcessorStep1 processorStep1() {
    CaricoDTOItemProcessorStep1 processorStep1 = new CaricoDTOItemProcessorStep1();
    return processorStep1;
}

This is my processor definition:

public class CaricoDTOItemProcessorStep1 implements ItemProcessor<CaricoDTO, CaricoDTO> {

    private String fileName;
    private static final Logger log = LoggerFactory.getLogger(CaricoDTOItemProcessorStep1.class);
    
    @Override
    public CaricoDTO process(CaricoDTO carico) throws Exception {

        carico.setDataCaricamento(new Date(System.currentTimeMillis()));
        carico.setFileName(carico.getResource().getFilename());
        return carico;
    }

    public String getFileName() {
        return fileName;
    }

    public void setFileName(String fileName) {
        this.fileName = fileName;
    }

}

And this is my writer:

@Bean
public JdbcBatchItemWriter<CaricoDTO> step1Writer(DataSource dataSource) {
    return new JdbcBatchItemWriterBuilder<CaricoDTO>()
            .itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
            .sql(....)
            .dataSource(dataSource).build();
}

What have you done so far? How you project / components setup look like? A simple straightforward way would be to use a JdbcTemplate but there might other / better ways of doing so. — tmarwen
– tmarwen, Commented Oct 7, 2021 at 9:59
@tmarwen I have a reader reading from a csv file, a processor and a writer writing into db but i have to discard some records due to values not present in domain tables. I'm using h2 as data persistence — Rosalba Marotta
– Rosalba Marotta, Commented Oct 7, 2021 at 10:08
I still don't get the total picture. Could you add the minimal reproducible code from your code base? How are you already performing reads / writes, are using the Spring Batch ItemReader / ItemWriter? — tmarwen
– tmarwen, Commented Oct 7, 2021 at 10:16

tmarwen · Accepted Answer · 2021-10-07 10:41:27Z

3

Given little details in the main OP, I am assuming that the general solution would be accepted.

Given a classic Spring Batch project setup, you should have access to a org.springframework.jdbc.core.JdbcTemplate bean connected to the target javax.sql.DataSource that you can simply inject in one of your batch collaborator components:

org.springframework.batch.item.ItemReader
org.springframework.batch.item.ItemWriter
org.springframework.batch.item.ItemProcessor

Assuming you need to perform some validation before proceeding with an item processing (and optionally discarding it), you can inject the JdbcTemplate bean, perform you query to validate your model invariants, then proceed accordingly:

public class CaricoDTOItemProcessorStep1 implements ItemProcessor<CaricoDTO, CaricoDTO> {
    
    @Autowired
    private JdbcTemplate jdbcTemplate;

    private String fileName;
    
    private static final Logger log = LoggerFactory.getLogger(CaricoDTOItemProcessorStep1.class);
    
    @Override
    public CaricoDTO process(CaricoDTO item) throws Exception {
        boolean someValue = jdbcTemplate.queryForObject("SELECT some_field FROM some_table WHERE some_other_field = 0", (rs, rowNum) -> rs.getBoolean(0));
        if (someValue) {
            carico.setDataCaricamento(new Date(System.currentTimeMillis()));
            carico.setFileName(carico.getResource().getFilename());
            return carico;
        } else {
            return null; // causes the item to be discarded from processing
        }
    }
}

Needless to say that your query would depend on your domain use-case and needs to be updated.

answered Oct 7, 2021 at 10:41

tmarwen

16.5k5 gold badges48 silver badges65 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Rosalba Marotta Over a year ago

Thank you very much. I will test your solution.

Rosalba Marotta Over a year ago

Thank you. This solved my issue!

Collectives™ on Stack Overflow

Executing query inside spring batch processor

EDIT: Adding sources:

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

EDIT: Adding sources:

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related