Java program to copy a file from local disk to Hadoop Cluster with progress bar


You need to look at the following blog posts to understand this post in a better way.

  1. Copying the File to HDFS file system
  2. A java program to read the file from HDFS
  3. A java program to read the file from HDFS – 2
  4. Java program to read a file from Hadoop Cluster 2 (with file seek)

We copied the file using CLI in example 1 given above. Lets use java to do the same.

Here comes the full java program to copy the local file to HDFS. You can get it from

import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.util.*;

public class FileCopyWithProgress{
    public static void main(String [] args) throws Exception{
        String localSrc=args[0];
        String dst=args[1];
        InputStream in=new BufferedInputStream(new FileInputStream(localSrc));
        Configuration conf=new Configuration();
        FileSystem fs = FileSystem.get(URI.create(dst), conf);
        OutputStream out=fs.create(new Path(dst),new Progressable(){

View original post 61 more words


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s