Java program to read a file from Hadoop Cluster

Hadoop

To know how to copy the file inside the hadoop cluster, read the following post.

How to copy the file to hadoop cluster

The following tutorial is the continuation of the above post. Here is a java program that will read the contents and display in on System.out.

You can download this java file from https://github.com/tomwhite/hadoop-book/blob/3e/ch03/src/main/java/URLCat.java

import java.io.*;
import java.net.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.io.*;

public class URLCat{

static{
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
}

public static void main (String [] args) throws Exception{
InputStream is = null;
try{
is=new URL(args[0]).openStream();
IOUtils.copyBytes(is,System.out,4096,false);
}finally{
IOUtils.closeStream(is);
}
}
}

Compile it.

$ export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar
$ bin/hadoop com.sun.tools.javac.Main URLCat.java

$ export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar:.
pandian@pandian-SH560:~/hadoop-2.5.0$ bin/hadoop URLCat hdfs://localhost:9000/user/pandian/anthem.hadoop.txt
Jana gaṇa mana adhināyaka jaya he
Bhārata bhāgya bidhātā
Pañjāba Sindhu Gujarāṭa Marāṭhā
Drāviḍa Utkala Baṅga
Bindhya Himācala Yamunā Gaṅgā
Ucchala jaladhi taraṅga
Taba śubha nāme jāge
Taba śubha āśiṣa māge
Gāhe taba jaya gāthā
Jana gaṇa maṅgala dāyaka jaya he

View original post 16 more words

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s