Incremental load from multiple tables for join queries in Nifi

Question

For Incremental load we will be using QueryDatabaseTable processor which extracts data incrementally from one table. For writing sql query which extracts data from multiple tables we are using ExecuteSQL processor.

How can we extract incremental load for a join query??

mattyb · Accepted Answer · 2021-12-10 17:35:18Z

0

If I understand what you're trying to do, in NiFi that's a Lookup pattern so you'd likely use LookupRecord with a DatabaseRecordLookupService. Each one of those would "join" the incremental load table with the rows from the table specified by the DatabaseRecordLookupService. For multiple joins you'd have a LookupRecord with corresponding DatabaseRecordLookupService for each of them.

answered Dec 10, 2021 at 17:35

mattyb

12.2k20 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Anil Krishna Parvathaneni Over a year ago

Could you please provide any sample screenshot of configurations

mattyb Over a year ago

There are examples of LookupRecord at community.cloudera.com/t5/Community-Articles/… but not with DatabaseRecordLookupService

Collectives™ on Stack Overflow

Incremental load from multiple tables for join queries in Nifi

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related