Java Spring Batch how to read from postgres and then write data in step

To read the table , you need to use one of Spring Batch provided readers - either use - org.springframework.batch.item.data.RepositoryItemReader or org.springframework.batch.item.database.JdbcPagingItemReader

Both readers implement pagination so your DB reading happens page by page & not whole table gets read all at once.

RepositoryItemReader has setPageSize(int pageSize) method and similar method is there in JdbcPagingItemReader too. There must be a column in your table on which ordering can be done to implement pagination.

Try to find code examples using these two readers.

These readers will read a page once , keep it in memory , and process single - single items till chunk size is reached & then commit happens. Next DB read wouldn't happen till one page is fully finished. Generally, for optimal performance , chunk size needs to be few times smaller than page size e.g. reader page size - 1000 & chunk size = 100 so 1000 items would be read once and committed in chunks of 100 - 100 items.

Next DB read happens when all of 1000 previous read have been passed to processor.

then for each row, publish that data elsewhere

To accomplish above, you will have to set chunk size to one and then in your writer , you can do whatever you wish and that way your transaction will be committed for each item.