java.lang.NoClassDefFoundError when reading hadoop SequenceFile

Question

I am trying to read a SequenceFile with custom Writeable in it.

Here's the code:

public static void main(String[] args) throws IOException {
    //String iFile = null;
    String uri = "/tmp/part-r-00000";
    Configuration conf = new Configuration();
    FileSystem fs = FileSystem.get(URI.create(uri), conf);
    Path path = new Path(uri);
    MyClass value = new MyClass();
    SequenceFile.Reader reader = null;
    try {
        reader = new Reader(fs, path, conf);
        while(reader.next(value)){
            System.out.println(value.getUrl());
            System.out.println(value.getHeader());
            System.out.println(value.getImages().size());               
            break;
        }           

    } catch (Exception e) {// Catch exception if any
        System.err.println("Error: " + e.getMessage());
    }
    finally {
        IOUtils.closeStream(reader);
    }

}

When I run this, I get following exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37)
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34)
    at org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:196)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
    at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:409)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:395)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1418)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1319)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:109)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:210)
    at com.iathao.run.site.emr.DecryptMapReduceOutput.main(DecryptMapReduceOutput.java:32)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    ... 14 more

All libraries are packaged into the jar file and are present. What's wrong, and how do I fix this?

If you open the jar of the lib, you can find the class? org/apache/commons/configuration/Configuration How you generate your jar? — user800014
– user800014, Commented Sep 26, 2011 at 18:11
The hadoop jar is not present in the classpath. How are you running this? — Hari Menon
– Hari Menon, Commented Sep 26, 2011 at 18:25
got it. I needed following library: java2s.com/Code/Jar/ABC/Downloadcommonsconfigurationjar.htm for the program to run properly. That's strange though. Never had such a problem before — Arsen Zahray
– Arsen Zahray, Commented Sep 26, 2011 at 19:03
did so. fixed my problem. will award the correct answer to the one who'll post it here — Arsen Zahray
– Arsen Zahray, Commented Sep 27, 2011 at 1:48

Praveen Sripati · Accepted Answer · 2011-09-27 03:34:37Z

4

The hadoop-common-*.jar has to be included for the org.apache.commons.configuration.Configuration class. Put the jar as dependencies.

answered Sep 27, 2011 at 3:34

Praveen Sripati

33.7k18 gold badges85 silver badges123 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

java.lang.NoClassDefFoundError when reading hadoop SequenceFile

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related