PDF input format for Mapreduce Hadoop

Question

Hi I anm using PDFBOX external library for parsing the pdf input file in mapreduce,but i am getting the following error.

Error: java.lang.ClassNotFoundException: org.apache.pdfbox.pdmodel.PDDocument at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at com.nielsen.grfe.processor.mapreduce.Pdfparser$PdfLineRecordReader.initialize(Pdfparser.java:109) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

I am using the following dependency

<dependency>
        <groupId>org.apache.pdfbox</groupId>
        <artifactId>pdfbox</artifactId>
        <version>1.8.10</version>
    </dependency>
    <dependency>
        <groupId>org.apache.pdfbox</groupId>
        <artifactId>fontbox</artifactId>
        <version>1.8.5</version>
    </dependency>

@prashant khunt I have added Distributed cache in the code..Still i face the same error.. — Barath
– Barath, Commented Dec 9, 2015 at 13:53

prashant khunt · Accepted Answer · 2015-12-09 13:33:49Z

0

1) Place the jar file of pdfbox in hadoop lib folder too.(make library jar available to hadoop at runtime).

2) Restart hadoop cluster.

Or

1) Make sure that your pdfbox library is available to hadoop by placing it in distributed cache.

edited Dec 9, 2015 at 13:33

answered Dec 9, 2015 at 13:15

prashant khunt

1543 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Barath Over a year ago

org.apache.pdfbox.pdmodel.PDDocument is not available in the PDFBOX 1.8.10.

prashant khunt Over a year ago

As per java docs, pdfbox.apache.org/docs/1.8.10/javadocs/org/apache/pdfbox/… the class is present in PDFBOX 1.8.10. The class is also present in the jar file. can you please paste the Exception that you are getting?

Collectives™ on Stack Overflow

PDF input format for Mapreduce Hadoop

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related