0

I use Windows 8 with a cloudera-quickstart-vm-5.4.2-0 virtual box. I am wanting to put a "words.txt" file in the HDFS. I got a connection exception.

I checked other StackOverflow threads/solutions and tried the following:

  • I checked the status of the namenode and datanode.
    It seems that the namenode is dead while the datanode is running.

  • I restarted the namenode, but it is still not running.

Here is a screenshot of the terminal using showing all the steps: terminal.png

I have no clue what to do next in order to solve the problem I have.

Note: I am new to Hadoop, so I am not familiar with most of the technical terms associated with it. A descriptive solution is very much appreciated.

5
  • check namenode's log, any error/exception? Commented Jan 5, 2018 at 13:08
  • I am completely new to Hadoop, so I unfortunately don't know now to check the log of the namenode. Could you help me with that? I am very curious to check out the log. Commented Jan 5, 2018 at 13:42
  • Any reason you're not using Cloudera Manager to restart your NameNode? And BTW, your log file is listed in your screenshot: /var/log/hadoop-hdfs/hadoop-hdfs-namenode-quickstart.cloudera.out Commented Jan 5, 2018 at 20:49
  • Use Hue to upload files in CDH Quickstart Commented Jan 6, 2018 at 0:33
  • I fixed my problem using this link: kshitish-bigdata.blogspot.nl/2015/02/… Commented Jan 6, 2018 at 15:29

1 Answer 1

2

Fixed my own problem

Check status of all services: for service in /etc/init.d/hadoop-hdfs-*; do $service status; done; , output should be:

Hadoop datanode is running [  OK  ]  
Hadoop namenode is dead and pid file exists [FAILED]  
Hadoop secondarynamenode is running  [  OK  ]

To get the namenode running do the following:

  • stop all services: for service in /etc/init.d/hadoop-hdfs-*; do $service stop; done;
  • clear cache from cache directory: sudo rm -rf /var/lib/hadoop-hdfs/cache/*
  • reformat name node: sudo -u hdfs hdfs namenode -format
  • start all services: for service in /etc/init.d/hadoop-hdfs-*; do $service start; done;
  • check status: for service in /etc/init.d/hadoop-hdfs-*; do $service status; done;

Result should be:

Hadoop datanode is running [  OK  ]  
Hadoop namenode is running [  OK  ]  
Hadoop secondarynamenode is running [  OK  ]

As described in the following link:

http://kshitish-bigdata.blogspot.nl/2015/02/hadoop-namenode-is-dead-and-pid-file.html

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.