JdBcPagingItemReader fails with Null NamedJdbcParameter for Step Configuration

Question

I am attempting to create a simple Step for a batch process which takes ProductDetail information and other information from the item processor to create and write the new ProductResult to a Mongodb database. I am attempting to use a JdbcPagingItemReader for reading the data from a database. However, when I start the job it fails with an error

java.lang.NullPointerException: Cannot invoke "org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.getJdbcOperations()" because "this.namedParameterJdbcTemplate" is null.

I also tested this with another reader and it works, so it is only the JdbcPagingItemReader that's the issue.

I don't know why the NamedParameterJdbcTemplate is needed since I have provided the select and from values and a configured row mapper.

Here is my code


    import com.datacom.mapper.ProductDetailRowMapper;
    import com.datacom.model.ProductDetail;
    import lombok.RequiredArgsConstructor;
    import org.springframework.batch.core.Step;
    import org.springframework.batch.core.repository.JobRepository;
    import org.springframework.batch.core.step.builder.StepBuilder;
    import org.springframework.batch.item.ItemStreamReader;
    import org.springframework.batch.item.ItemWriter;
    import org.springframework.batch.item.data.MongoItemWriter;
    import org.springframework.batch.item.data.builder.MongoItemWriterBuilder;
    import org.springframework.batch.item.database.JdbcPagingItemReader;
    import org.springframework.batch.item.database.Order;
    import org.springframework.batch.item.database.support.OraclePagingQueryProvider;
    import org.springframework.beans.factory.annotation.Qualifier;
    import org.springframework.beans.factory.annotation.Value;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.mongodb.core.MongoOperations;
    import org.springframework.transaction.PlatformTransactionManager;
    
    import javax.sql.DataSource;
    import java.util.HashMap;
    import java.util.Map;
    
    @Configuration
    @RequiredArgsConstructor
    public class ProductDetailConfigStep {
    
        @Value("${db.pageSize}")
        int pageSize;
    
        final MongoOperations mongoOperations;
    
        final @Qualifier("productsDataSource") DataSource dataSource;
    
        final PlatformTransactionManager transactionManager;
    
    
        final JobRepository jobRepository;
    
        @Value("${batch.chunkSize}") int chunkSize;
    
        final ProductResultProcessor productResultProcessor;
    
        public ItemStreamReader<ProductDetail> itemReader() {
            JdbcPagingItemReader<ProductDetail> itemReader = new JdbcPagingItemReader<>();
            itemReader.setDataSource(productsDataSource);
            itemReader.setFetchSize(chunkSize);
            OraclePagingQueryProvider queryProvider = new OraclePagingQueryProvider();
            queryProvider.setSelectClause(JDBCConstants.SELECT_QUERY);
            queryProvider.setFromClause(JDBCConstants.FROM_CLAUSE_QUERY);
            itemReader.setRowMapper(new ProductDetailRowMapper());
            final Map<String, Order> sortKeys = new HashMap<>();
            itemReader.setParameterValues(new HashMap<>());
            sortKeys.put("product_id", Order.ASCENDING);
            queryProvider.setSortKeys(sortKeys);
            itemReader.setQueryProvider(queryProvider);
            itemReader.setPageSize(pageSize);
            itemReader.setPageSize(chunkSize);
            return itemReader;
        }
    
        public ItemWriter<ProductResult> itemWriter() {
            var writer = new MongoItemWriterBuilder<ProductResult>()
                    .collection(Documents.PRODUCT_CATALOG_COLLECTION)
                    .mode(MongoItemWriter.Mode.INSERT)
                    .template(mongoOperations)
                    .build();
            writer.setTemplate(mongoOperations);
            return writer;
        }
    
        @Bean("productProcessingDetailStep")
        public Step productProcessingDetailStep() {
            return new StepBuilder("productCreationStep", jobRepository)
                    .<ProductDetail, ProductResult>chunk(chunkSize, transactionManager)
                    .reader(itemReader())
                    .processor(productResultProcessor)
                    .writer(itemWriter())
                    .build();
        }
    }

What am I missing the configuration of the JdbcPagingItemReader looks fine to me.

Spring 101: for Spring to be able to autowire classes you have to let Spring instantiate all classes it needs for autowiring. Meaning the second you use the keyword new - there will be no autowiring done in the case that you instantiated yourself. If you want Spring to autowire something into JdbcPagingItemReader then let Spring instantiate it — asgarov1
– asgarov1, Commented Dec 8, 2024 at 8:26
When using Java based configuration be as explicit about the returned type as possible. So instead of using ItemStreamReader<ProductDetail> use JdbcPagingItemReader<ProductDetail>. That way Spring is able to determine what interfaces there are on the actual class, including InitialzingBean so the setup is being done properly. The same applies to your other methods as well. — M. Deinum
– M. Deinum, Commented Dec 9, 2024 at 12:55

M. Deinum · Accepted Answer · 2024-12-09 13:05:05Z

The "problem" is your actual @Bean methods. When defining those in your configuration classes be explicit about the return type of those methods. When the methods are scanned it only has the metadata from the method. This means it knows an ItemStreamReader is being returned. It knowns nothing more.

If you take a look at the JdbcPagingItemReader you will notice that it implements more interfaces, extends a class (which could even have other callback interfaces, even for Spring Batch).

To properly detect those while scanning the @Configuration class the infrastructure will need the JdbcPagingItemReader return type and not the ItemStreamReader return type.

To properly solve this change the return type of your method(s) to be as specific as possible. And make sure that they are actual @Bean methods, as currently they aren't.


@Configuration
public class ProductDetailConfigStep {

    @Bean
    public JdbcPagingItemReader<ProductDetail> itemReader(
      @Qualifier("productsDataSource") DataSource dataSource, 
      @Value("${db.pageSize}") int pageSize, 
      @Value("${batch.chunkSize}") int chunkSize) {
        JdbcPagingItemReader<ProductDetail> itemReader = new JdbcPagingItemReader<>();
        itemReader.setDataSource(productsDataSource);
        itemReader.setFetchSize(chunkSize);
        OraclePagingQueryProvider queryProvider = new OraclePagingQueryProvider();
        queryProvider.setSelectClause(JDBCConstants.SELECT_QUERY);
        queryProvider.setFromClause(JDBCConstants.FROM_CLAUSE_QUERY);
        itemReader.setRowMapper(new ProductDetailRowMapper());
        final Map<String, Order> sortKeys = new HashMap<>();
        itemReader.setParameterValues(new HashMap<>());
        sortKeys.put("product_id", Order.ASCENDING);
        queryProvider.setSortKeys(sortKeys);
        itemReader.setQueryProvider(queryProvider);
        itemReader.setPageSize(pageSize);
        itemReader.setPageSize(chunkSize);
        return itemReader;
    }

    @Bean
    public MongoItemWriter<ProductResult> itemWriter(MongoOperations mongoOperations) {
        var writer = new MongoItemWriterBuilder<ProductResult>()
                .collection(Documents.PRODUCT_CATALOG_COLLECTION)
                .mode(MongoItemWriter.Mode.INSERT)
                .template(mongoOperations)
                .build();
        writer.setTemplate(mongoOperations);
        return writer;
    }

    @Bean("productProcessingDetailStep")
    public Step productProcessingDetailStep(
      PlatformTransactionManager transactionManager, 
      JobRepository jobRepository,
      ProductResultProcessor productResultProcessor,
      JdbcPagingItemReader itemReader,
      MongoItemWriter itemWriter) {
        return new StepBuilder("productCreationStep", jobRepository)
                .<ProductDetail, ProductResult>chunk(chunkSize, transactionManager)
                .reader(itemReader)
                .processor(productResultProcessor)
                .writer(itemWriter)
                .build();
    }
}

NOTE: I also took the liberty of rewriting the config using best practices, like injecting dependencies into the methods instead of the class. Using specific return types and added @Bean methods. Now Spring will automatically know what to wire into what and manage the instances (using the proper callbacks).

I had it as JdbcPagingItemReader before but that did not work.
Because it wasn't an @Bean method and hence not managed by Spring.

Henning · Accepted Answer · 2024-12-08 16:10:23Z

-1

The field namedParameterJdbcTemplate is null because the method afterProperties of the reader is not being called.

If you expose the reader as a bean, then Spring will call this method for you. But you can also do this yourself if there is no other need for the reader to be a bean.

It should work as expected if you change your code like this:

public ItemStreamReader<ProductDetail> itemReader() throws Exception {
    ...
    itemReader.afterPropertiesSet();
    return itemReader;
}

Please note the throws Exception in the method signature. Alternatively, you can wrap the call to afterPropertiesSet in an appropriate try-catch block.

answered Dec 8, 2024 at 16:10

Henning

4,0141 gold badge10 silver badges37 bronze badges

1 Comment

M. Deinum Dec 9, 2024 at 12:53

If you need to start calling framework supported callback methods yourself you are doing it wrong.

Collectives™ on Stack Overflow

JdBcPagingItemReader fails with Null NamedJdbcParameter for Step Configuration

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related