Hadoop Min Max Exception in thread "main" java.lang.ClassNotFoundException:

Question

I am new in Hadoop. I try to write a program to get the min and max values per year.The data set looks like:

Product code,Station number,Year,Month,Monthly Precipitation Total (millimetres),Quality
IDCJAC0001,023093,1978,01,4.6,Y
IDCJAC0001,023093,1978,02,11.4,Y
IDCJAC0001,023093,1978,03,4.8,Y
IDCJAC0001,023093,1978,04,46.2,Y
IDCJAC0001,023093,1978,05,52.7,Y

I am using mac and conduct Hadoop on my school cluster. Here is my command line:

hadoop jar hw1-1.0-SNAPSHOT.jar /samplefolder/IDCJAC0001Data1.csv /OUTPUT

But the result shows error:Exception in thread "main" java.lang.ClassNotFoundException: /samplefolder/IDCJAC0001Data1/csv

I've searched on the website and try different commands such as hadoop jar hw1-1.0-SNAPSHOT.jar org.mycompany.hw1.SolarMinMax /samplefolder/IDCJAC0001Data1.csv /OUTPUT, but the result is same...

Please help me with this problem

Data Man · Accepted Answer · 2019-06-03 10:05:24Z

1

You need to run with

hadoop jar hw1-1.0-SNAPSHOT.jar org.mycompany.hw1.SolarMinMax /samplefolder/IDCJAC0001Data1.csv /OUTPUT

and then change your code to be

FileInputFormat.addInputPath(job, new Path(args[1]));
FileOutputFormat.setOutputPath(job, new Path(args[2]));

because args[0] is the class name.

answered Jun 3, 2019 at 10:05

Data Man

5,5596 gold badges46 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Shin Yu Wu Over a year ago

Thanks for your reply. I changed the code, and run with the command line as you said, but the result still shows: Exception in thread "main" java.lang.ClassNotFoundException: org.mycompany.hw1.SolarMinMax

Data Man Over a year ago

OK then your JAR mustn't be built correctly, or the class isn't in the package you said.

Shin Yu Wu Over a year ago

Oh, it works. But it shows another error...java.lang.NumberFormatException: For input string: "Monthly Precipitation Total (millimetres)"

Data Man Over a year ago

As for this new problem, it looks like you're reading the header row, and you're also not checking your data to make sure that each entry is actually a number. When writing MR apps you always have to assume your data won't be clean. See stackoverflow.com/questions/37541109/…

Collectives™ on Stack Overflow

Hadoop Min Max Exception in thread "main" java.lang.ClassNotFoundException:

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related