We can now build hadoop source version 2.4.0 on windows and configure it to use in eclipse. Follow the steps mentioned below to configure hadoop source on windows.

Requisites

1. Download hadoop distribution hadoop-2.4.0-src.tar.gz from here.
2. 7-ZIP (Right click on any folder to check if it is already installed in your window)
3. Download protoc-2.5.0-win32.zip from here.
4. Download Eclipse IDE for Java EE Developers
5. Download apache-maven-3.1.1-bin.zip

Installations

1. Apache Maven:
a. Unzip the distribution archive, i.e. apache-maven-3.1.1-bin.zip to the directory you wish to install Maven 3.1.1.
b. These instructions assume you chose C:. The subdirectory apache-maven-3.1.1 will be created from the archive.
c. Open up the system properties (WinKey + Pause), selecting the “Advanced” tab, and the “Environment Variables” button.
d. Add the M2_HOME environment variable: Add the M2_HOME variable in the user variables with the value C:\apache-maven-3.1.1 Be sure to omit any quotation marks around the path even if it contains spaces.
e. In the same dialog, add the M2 environment variable in the user variables with the value %M2_HOME%bin.
f. In the same dialog, update/create the Path environment variable in the user variables and prepend the value %M2% to add Maven available in the command line.
g. In the same dialog, make sure that JAVA_HOME exists in your user variables or in the system variables and it is set to the location of your JDK, e.g. C:\Program Files\Java\jdk1.7.0 and that %JAVA_HOME%\bin is in your Path environment variable.
h. Open a new command prompt (Winkey + R then type cmd) and run mvn –version to verify that it is correctly installed.

2. Protoc 2.5:
a. Unzip the protoc binary zip protoc-2.5.0-win32.zip downloaded from the link above.
b. Create a folder structure C:\protobuf-2.5\src.
c. Copy the protoc.exe binary available in unzipped protoc-2.5.0-win32 to C:\protobuf-2.5\src folder.
d. Open up the system properties (WinKey + Pause), selecting the “Advanced” tab, and the “Environment Variables” button.
e. Append the existing path variable in user and system properties with the path to protoc.exe which is C:\protobuf-2.5\src.
f. Open a new command prompt (Winkey + R then type cmd) and run protoc –version to verify that it is correctly installed.

3. Eclipse:
a. Unzip the eclipse from the folder downloaded from the link mentioned above.
b. Open eclipse and Goto Help –>Eclipse Marketplace.
c. Search and install maven plugin for your eclipse.

4. Hadoop 2.4.0:
a. Unzip the Hadoop tar file hadoop-2.4.0-src.tar.gz using windows 7-ZIP utility.
b. Open a new command prompt (Winkey + R then type cmd) and move to Hadoop source code folder.
c. To install eclipse dependencies for each module run the following commands:

cd d:hadoop-2.4.0-src
cd hadoop-hdfs-project
mvn eclipse:eclipse
cd ..
cd hadoop-common-project
mvn eclipse:eclipse
cd ..
cd hadoop-mapreduce-project
mvn eclipse:eclipse
cd ..
cd hadoop-yarn-project
mvn eclipse:eclipse

d. Make sure all the sub modules compiled successfully and have “Build Successful” Message.
e. To install and create binaries of all sub modules, execute the following commands:

cd d:hadoop-2.4.0-src
cd hadoop-hdfs-project
mvn clean install -DskipTests
cd..
cd hadoop-common-project
mvn clean install -DskipTests
cd ..
cd hadoop-mapreduce-project
mvn clean install -DskipTests
cd ..
cd hadoop-yarn-project
mvn clean install -DskipTests

f. Make sure all the sub modules compiled successfully and have “Build Successful” Message.

5. Importing Hadoop into eclipse:
a. Open eclipse
b. Goto File -> Import.
c. Type maven.
d. Select “Existing Maven Projects”
e. Browse to the Hadoop source folder and select modules to import.

Share this:

One thought on “Build Hadoop-2.4.0 Source on windows and Configure in Eclipse

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *