Passing Hadoop program commands through Java Program

Question

Can anyone please let me know if there is any way to pass hadoop commands to hdfs (eg: create for delete directory and files"hadoop fs -mkdir /test") from java program? I am trying to create and delete directory and files in hdfs through java program.

Also is there any way to check the size of the files/directory in hdfs through java program. I am trying to check the size of the directory in hdfs and if the size is 0 then I want to delete that file. I need to do all these things through java program.

Please help.

Aleksei Shestakov · Accepted Answer · 2014-07-19 12:42:46Z

Look at FileSystem, it allows you to create, delete files, etc. Simple class that creates a file and prints its size:

import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;

public class Sample extends Configured implements Tool {
    public static void main(String[] args) throws Exception {
        ToolRunner.run(new Sample(), args);
    }

    @Override
    public int run(String[] strings) throws Exception {
        FileSystem fs = FileSystem.get(getConf());
        fs.mkdirs(new Path("/sample/dir"));
        FSDataOutputStream out = fs.create(new Path("/sample/dir/file.txt"));
        out.writeBytes("hello");
        out.close();
        System.out.println(fs.getFileStatus(new Path("/sample/dir/file.txt")).getLen());
        return 0;
    }
}

Collectives™ on Stack Overflow

Passing Hadoop program commands through Java Program

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related