classcastException when sorting sequenceFile in hadoop?

Question

I am following Hadoop-The definitive guide 3rd edition by Tom White. I have successfully written a sequenceFile into HDFS. I followed the example the author gave in book. but when I try to run the sort (pg 138), I get the classCastException. The stacktrace is available below.

what is wrong here and what fix is needed?

hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar sort -r 1 -inFormat org.apache.hadoop.mapred.SequenceFileInputFormat -outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat -outKey org.apache.hadoop.io.IntWritable -outValue org.apache.hadoop.io.Text /output/seqfile /output/sortedfile
14/07/09 10:51:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/09 10:51:53 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/07/09 10:51:53 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
java.lang.ClassCastException: class org.apache.hadoop.mapred.SequenceFileInputFormat
    at java.lang.Class.asSubclass(Class.java:3075)
    at org.apache.hadoop.examples.Sort.run(Sort.java:104)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.examples.Sort.main(Sort.java:191)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Mike Park · Accepted Answer · 2014-07-09 18:38:30Z

1

This is possibly because you're using the old map/reduce sequence file class. Instead of using

-inFormat org.apache.hadoop.mapred.SequenceFileInputFormat
-outFormat org.apache.hadoop.mapred.SequenceFileOutputFormat

try using

-inFormat org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat;
-outFormat org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat;

answered Jul 9, 2014 at 18:38

Mike Park

11k3 gold badges38 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

brain storm Over a year ago

@cimbage: this is unrelated to the post here. But I have some issues using application manager in hadoop2. I have a SO post here. can you kindly advise me how to make it work: stackoverflow.com/questions/24518374/…

eagertoLearn Over a year ago

worked great!, Thanks!!!!. I have another problem which I posted here: stackoverflow.com/questions/24661653/…. I will accept this answer shortly

Collectives™ on Stack Overflow

classcastException when sorting sequenceFile in hadoop?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related