Pentaho Kettle program in java to merge multiple csv files by columns

Question

I have two csv files employee.csv and loan.csv.

In employee.csv I have four columns i.e. empid(Integer),name(String),age(Integer),education(String).

In loan.csv I have three columns i.e. loan(Double),balance(Double),empid(Integer).

Now, I want to merge these two csv files into a single csv file by empid column.So in the result.csv file the columns should be,

empid(Integer),
name(String),
age(Integer),
education(String),
loan(Double),
balance(Double).

Also I have to achieve this only by using kettle api program in Java. Can anyone please help me?

Rishu S · Accepted Answer · 2015-02-17 08:34:42Z

1

First of all, you need to create a kettle transformation as below:

Take two "CSV Input Step", one for employee.csv and another for loan.csv
Hop the input to the "Stream Lookup" step and lookup using the "emplid"
Final step : Take a Text file output to generate a csv file output.

I have placed the ktr code in here.

Secondly, if you want to execute this transformation using Java, i suggest you read this blog. I have explained how to execute a .ktr/.kjb file using Java.

Extra points:

If its required that the names of the csv files need to be passed as a parameter from the Java code, you can do that by adding the below code:

  trans.setParameterValue(parameterName, parameterValue);

where parameterName is the some variable name and parameterValue is the name of the file or the location.

I have already taken the files names as the parameter in the kettle code i have shared.

Hope it helps :)

answered Feb 17, 2015 at 8:34

Rishu S

3,9681 gold badge22 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

vishal kadu Over a year ago

Thanks Rishu for your quick reply.I appreciate your help.I am able to execute existing .ktr file with java code. But my requirement is to create .ktr file from above two .csv files using java code (not spoon). Can you please help me for this, if possible can you please share java code for the same so that i can create .ktr file using java. I want to create .ktr file using java code from two or more .csv files.

Lyd Over a year ago

@Rishu, I have looked at your code in github. My problem is that I can't find org.pentaho.di.core.KettleEnvironment class. I've downloaded all required jars but it's not there. In which jar it is supposed to be?

Rishu S Over a year ago

@Lyd KettleEnviornment class will be available in the core kettle jar. Not sure what exactly you have imported. I have used maven to import the jars...try checking link... hope it might help !!

Collectives™ on Stack Overflow

Pentaho Kettle program in java to merge multiple csv files by columns

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related