Creating a multiple CSV files from a Single Table in Azure Data Factory

Question

I have a Single SQL Table named an Employee Table. In Employee Table there are the following columns 1. ID, 2. Designation, 3. Salary, 4 Joining Date .. etc.

Now let say there are 200 records in this Employee Table, with 10 different designations of Employee. Now, I want to create a CSV file of this table using Azure Data Factory, based on(filter by) designations, i.e 10 CSV files(bcoz there are 10 different designations) should generate and store this in the storage account in Azure. Kindly guide me in this.

Nandan · Accepted Answer · 2021-02-25 09:47:46Z

3

You would need to follow the below flow chart:

LookUp Activity : Query : Select distinct designations from table For each activity Input : @activity('LookUp').output.value a) Copy activity i) Source : Dynamic Query Select * from t1 where designation=@item().designation This should generate separate files for each Designation as needed

For more details: How to export from sql server table to multiple csv files in Azure Data Factory

answered Feb 25, 2021 at 9:47

Nandan

5,2252 gold badges12 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ashish Shelar Over a year ago

The second example (City) worked for me. Thanks a lot.

Nandan Over a year ago

Hey @AshishShelar, Can you please mark it as answer as it may benefit other community members as well. Thanks

Steve Johnson · Accepted Answer · 2021-02-25 11:25:10Z

2

Another way: use Data Flow, set Key partition type and choose Designation as partition column at Source. Then choose Pattern as File name option and use employee[n].csv as file name.

Details:

1.set Key partition type and choose Designation as partition column at Source.

2.choose Pattern as File name option and use employee[n].csv as file name

answered Feb 25, 2021 at 11:25

Steve Johnson

8,8081 gold badge11 silver badges21 bronze badges

Collectives™ on Stack Overflow

Creating a multiple CSV files from a Single Table in Azure Data Factory

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related