error while executing select query in hive

Question

I'm using hadoop 1.1.2 , hbase 0.94.8 and hive 0.14 . I'am trying to create a table in hbase using hive and load data in it later by insert overwrite .

for the moment I was able to create the table:

CREATE TABLE hbase_table_emp(id int, name string, role string) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name,cf1:role")
TBLPROPERTIES ("hbase.table.name" = "emp");

and load data into another table that I will overwrite it into the hbase table :

hive> create table testemp(id int, name string, role string) row format delimited fields terminated by '\t';
hive> load data local inpath '/home/user/sample.txt' into table testemp;

but when I try select * from testemp; to verify that the data was loaded successfully I get this error:

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JobConf.unset(Ljava/lang/String;)V
    at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushFilters(HiveInputFormat.java:432)
    at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:76)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:443)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1129)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:622)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

could someone help me please ! thank you

Have you changed Hadoop package versions since the last time Hive was compiled? — rchang
– rchang, Commented Jan 21, 2015 at 15:06
So, nothing concrete, but despite the 0.14 release notes there seems to be at least anecdotal evidence Hive 0.14 isn't 100% compatible with Hadoop 1.1.2 just yet (see stackoverflow.com/questions/27842004/…). I have never tried to operate Hive with Hadoop 1.x myself. — rchang
– rchang, Commented Jan 21, 2015 at 15:13
I changed to hive 0.13 and nothing works I get this error even when I tried to visualise the tables (>show tables;) :( — Yosser Abdellatif Goupil
– Yosser Abdellatif Goupil, Commented Jan 21, 2015 at 15:19
Unfortunately, this may be an issue specific to Hadoop 1.x then, of which I have no firsthand experience. I'm guessing it can work with Hadoop 1.x, but perhaps just not out-of-the-box in every circumstance. — rchang
– rchang, Commented Jan 21, 2015 at 15:25

rchang · Accepted Answer · 2015-01-21 21:06:46Z

1

Unfortunately, I believe you will have to upgrade Hadoop to at least 1.2.0.

It appears that Hive is trying to access the unset method of the org.apache.hadoop.mapred.JobConf class. In looking at the API documentation for that class in Hadoop 1.1.2, you can see that method does not exist.

The first release from the 1.x series in which that method gets implemented is 1.2.0 (see the API documentation for the same class). Note that the method actually gets inherited from the org.apache.hadoop.conf.Configuration class.

answered Jan 21, 2015 at 21:06

rchang

5,2441 gold badge18 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Yosser Abdellatif Goupil Over a year ago

thank you soo much this works for me it was a version compatibility problem :)

rchang Over a year ago

@YosrAbdellatif I'm glad it worked out. It was a subtle issue to track down, to be sure, since the documentation does mention compatibility with Hadoop "1.x.y". I guess x >= 2. :)

Yosser Abdellatif Goupil Over a year ago

@ rchang now I get another error while executing insert overwrite table hbase_table_emp select * from testemp; I get this error: [[[[[ java.lang.NoClassDefFoundError: org/cliffc/high_scale_lib/Counter]]]]] problem compatibility again?

rchang Over a year ago

@YosrAbdellatif Did you build HBase locally with Maven? The missing class seems to be from com.github.stephenc.high-scale-lib, which is listed as a dependency in HBase's pom.xml file. I'd recommend checking your library directory to see if high-scale-lib-x.x.x.jar (where x.x.x is some version number) is present.

Yosser Abdellatif Goupil Over a year ago

@ rchang yes high-scale-lib-1.1.1.jar does exist in /hbase/lib and I'm just running those queries in hive shell what maven got to do with this?

|

Collectives™ on Stack Overflow

error while executing select query in hive

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related