Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

Question

I am facing problem in executing CONDITIONAL QUERIES IN HiveQL.The basic select * from tablename statement works fine. The error is :

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
java.net.ConnectException: Call to /0.0.0.0:8021 failed on connection exception: java.net.ConnectException: Connection refused
        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1134)
        at org.apache.hadoop.ipc.Client.call(Client.java:1110)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
        at org.apache.hadoop.mapred.$Proxy5.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:398)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:384)
        at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:501)
        at org.apache.hadoop.mapred.JobClient.init(JobClient.java:486)
        at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:469)
        at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:655)
        at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:516)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(
....
Job Submission failed with exception 'java.net.ConnectException(Call to /0.0.0.0:8021 failed on connection exception: java.net.ConnectException: Connection refused)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

I have tried from both Eclipse IDE and Hive shell .The output is same. Please Help!

Do you have the solution now? I too face the same issue.

Manikandan Kannan
– Manikandan Kannan

2012-12-27 06:05:56 +00:00
Commented Dec 27, 2012 at 6:05 — Manikandan Kannan
– Manikandan Kannan, Commented Dec 27, 2012 at 6:05
You might find this web app helpful. sql.treasuredata.com

Kiyoto Tamura
– Kiyoto Tamura

2014-11-06 23:17:55 +00:00
Commented Nov 6, 2014 at 23:17 — Kiyoto Tamura
– Kiyoto Tamura, Commented Nov 6, 2014 at 23:17

mrsrinivas · Accepted Answer · 2017-11-16 11:01:30Z

7

This should solve your issue as this property will set to true by default.

set hive.auto.convert.join.noconditionaltask=false

hive.auto.convert.join.noconditionaltask - Whether Hive enable the optimization about converting common join into mapjoin based on the input file size.

If this paramater is on, and the sum of size for n-1 of the tables/partitions for a n-way join is smaller than the specified size, the join is directly converted to a mapjoin (there is no conditional task).

edited Nov 16, 2017 at 11:01

mrsrinivas

35.6k13 gold badges133 silver badges132 bronze badges

answered Sep 28, 2017 at 9:01

G.v. Sridhar

1372 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Salsero06 · Accepted Answer · 2013-09-30 14:59:05Z

3

In most cases, this error is caused by a permissions issue, where the Hive user running the command does not have access rights to /user/hive

You'll want to check the configuration of the Hadoop Hive cluster

answered Sep 30, 2013 at 14:59

Salsero06

312 bronze badges

Comments

apublius · Accepted Answer · 2016-06-16 00:07:33Z

1

I have noticed that with Hive2 you need to change the way you specify the number of reducers while using HiveQL. On Hive1 I have used:

SET mapred.reduce.tasks=1

However on Hive2 I have noticed I need to use:

SET mapreduce.job.reduces=1

I had the same error message and changing this resolved the issue for me.

answered Jun 16, 2016 at 0:07

apublius

1043 bronze badges

Comments

shazzing · Accepted Answer · 2017-12-15 13:31:33Z

0

I have a cloudera quickstart image mounted using VirtualBox. I had to do the following to fix the issue

sudo vi /etc/hive/conf/hive-site.xml and set the hive.auto.convert.join property to false (you need to root to edit and hence the sudo).

I did not have the hive.auto.convert.join.noconditionaltask property in the hive-site.xml.xml.

answered Dec 15, 2017 at 13:31

shazzing

4414 silver badges4 bronze badges

Comments

swapnil shashank · Accepted Answer · 2019-11-14 06:54:55Z

0

For me, it was due to the queue not being set.

set mapred.job.queue.name=xxxx;
set mapreduce.job.reduces=1;

Setting both parameters it worked for me.

answered Nov 14, 2019 at 6:54

swapnil shashank

1,0179 silver badges11 bronze badges

Comments

Cereal Killer · Accepted Answer · 2021-10-18 12:53:35Z

0

This worked for me:

SET hive.auto.convert.join=false;

As default Hive converts simple joins to map joins which try to load input tables to memory. This option will disable loading input tables to memory.

answered Oct 18, 2021 at 12:53

Cereal Killer

213 bronze badges

Collectives™ on Stack Overflow

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related