Setting up Eclipse IDE for Hadoop 2.5.0

Hadoop

You need to look at the following blog posts to understand this post in a better way.

  1. Copying the File to HDFS file system
  2. A java program to read the file from HDFS
  3. A java program to read the file from HDFS – 2
  4. Java program to read a file from Hadoop Cluster 2 (with file seek)
  5. Java program to copy a file from local disk to Hadoop Cluster with progress bar

Till now we are bugging with terminal to write the java programs. Here is how you can setup the Eclipse development environment for Hadoop.

  • This tutorial assumes you have working Hadoop 2.5.0 setup in your environment.
  • This tutorial assumes you have m2-eclipse maven plugin
  • This tutorial assumes you have installed latest version of Maven installed on your system

Set up a plain java maven project. I set it up by name my-app.

Cloud era repository is not still available…

View original post 192 more words

Maven Tutorial – 3

Lets create a simple application as we have seen in Tutorial -2.

Execute the following command. Ensure you have jdk bin and maven bin folders in path!

$ mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=org.grassfield.app -DartifactId=my-app

We are generating a simple project based on generate architype – a simple template. After issuing the above command, you will see a list of lines and characters are running on the screen. You will be asked some questions, just press enter!

maven01

You can see a new folder is created with your artifact id, which is ‘my-app’ here! If you browse through this folder, you will see the following.

$ ls
my-app
$ cd my-app/
$ ls
pom.xml src
$ cd src
$ ls
main test
$ cd main
$ ls
java
$ cd java
$ ls
org
$ cd org/grassfield/app/
$ ls
App.java

App.java – Created by Maven

maven02

Lets modify it now.

Here it is!

maven03

Lets see what’s written inside pom.xml

pandian@pandian-SH560:~/pandian/my-app$ cat pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.grassfield.app</groupId>
<artifactId>my-app</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>my-app</name>
<url>http://maven.apache.org</url&gt;

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>

POM refers to junit alone as its dependency!

Now lets compile it.

$ mvn compile

maven04

Done!

Lets test it now! The following is the maven test case.

maven05

$ mvn test

maven06

Lets package it now!

$ mvn package

maven08

Console is interesting. But you love Eclipse IDE like, right? Lets create the Eclipse project structure for this artifact.

$ mvn eclipse:eclipse

maven09

Now import the java project from the location highlighted above.

maven10 - Eclipse IDE

Lets see how to work with Eclipse using Maven in my next post.

Related posts:

Maven – 1

Maven – 2

Subversioning in windows and Eclipse integration

This is an interesting drill I have done yesterday. Microsoft VSS is mainly used version control in my organization; I am happy with SVN. So here comes the installation and creation of SVN repositories. You will find the integration of SVN with eclipse as well.

Installing the Subversion in Windows box

Prerequisites: Install Python for windows, make sure no servers are listening to port 80 (else, you can reconfigure the subversioning portal after installing)

Download the Collabnet subversioning tool from http://subversion.apache.org/packages.html (Thanks Siva for giving the link)

Install the Collabnet subversion tool downloaded. It is pretty much straight forward. At the end, You will see the local file that gives access to your SVN server. You need to wait the console status becomes ‘ready‘. Then click on the URL given with the stated credentials.

Logging in as admin/admin

Verify the server status. Make sure subversion status us UP and green. Note the repository parent path is given here. We will be using this later while integrating this with Eclipse.

Clicking on Repositories tab, we can add/modify repositories and defind the access rules.

fyi! Creating a new repository.

Users Tab, Create users here with specified user role.

Subversion is up and running now. We have created users and repositories.

Installing Eclipse plugin for SVN

Now we will be installing the subversion plugin for Eclipse IDE. Add http://download.eclipse.org/releases/helios to update sites. Select the necessary tools in ‘Collaboration’. (Don’t forget to include SVN tools).

We need to install two things.

  1. SVN plugin
  2. SVN connector

To start, lets install the plugin.

You may be asked for Eclipse restarts couple of times.

This is the time to install SVN connector. I have selected two as follows.

Thats all!

Now change the prospective to SVN.

We can add new SVN location to check out existing projects.

Else, we can share an existing project from Eclipse. Select Team>Share to push a Eclipse project into SVN.

Enter the svn location and credentials

Share the new project

Commit the code

Done! See the SVN globe symbol in the eclipse project now!

Happy stitching!