Word count program with two input files and single output file

Question

I am new to Hadoop. I have done word count program with single input file and single output file. Now I want to take 2 files as input and write that output to a single file. I tried like this:

FileInputFormat.setInputPaths(conf, new Path(args[0]), new Path(args[1]));
FileOutputFormat.setOutputPath(conf, new Path(args[2]));

This is the command in terminal:

hadoop jar test.jar Driver /user/in.txt /user/sample.txt /user/out

When I run this, its taking sample.txt as output directory and says that :

Output directory hdfs://localhost:9000/user/sample.txt already exists

Can anyone help me with this?

Just wanted to know how did you made the Jar..Is it a normal jar or runnable jar...If runnable jar then you dont have to mention the driver class name .... — Aman
– Aman, Commented May 21, 2015 at 6:25
It was a runnable jar, so I removed Driver and it worked. Thanks Aman. — MChirukuri
– MChirukuri, Commented May 21, 2015 at 11:25

salmanbw · Accepted Answer · 2015-05-20 20:02:28Z

2

May be because it is taking Driver as your first argument. why don't you try like this.

hadoop jar test.jar /user/in.txt /user/sample.txt /user/out

answered May 20, 2015 at 20:02

salmanbw

1,3112 gold badges19 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Rajesh N · Accepted Answer · 2015-05-21 05:21:18Z

1

If you have all the input files in one folder as you have mentioned (/user), the replace

hadoop jar test.jar Driver /user/in.txt /user/sample.txt /user/out

with this

hadoop jar test.jar Driver /user /user/out

This takes all the file inside /user directory as input and outputs in user/out folder in HDFS.

answered May 21, 2015 at 5:21

Rajesh N

2,5741 gold badge15 silver badges17 bronze badges

Collectives™ on Stack Overflow

Word count program with two input files and single output file

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related