In the previous blog, we discussed about the HDFS High availability architecture. This blog describes the configurations for HDFS high availability in a Hadoop cluster. Pre-requisites Before configuring HDFS high availability, make sure that your Hadoop cluster has the following pre-requisites: a) You must have at-least two nodes to enable HDFS high availability. b) If you want to configure […]
Hadoop Administration
Hadoop HDFS Concepts
This presentation gives an overview of Hadoop HDFS concepts like Blocks, Rack Awareness, Safe Mode etc. Hadoop HDFS Concepts from tutorialvillage
Hadoop Cluster Setup
If you wish to deploy Hadoop Single node setup, please follow the blog here. Pre-requisites Before starting with Hadoop cluster setup, make sure that the node have the following pre-requisites: a) Any Linux Operating system b) Sun Java 1.6 or above should already be installed and the version should be same across all […]
HDFS High Availability Architecture
In the previous blog, we discussed about the need and design goals of HDFS High Availability. In this blog, we will talk about the architecture of HDFS high availability. HDFS High Availability Architecture In order to provide a HOT back-up and consistent solution for NameNode failure, a concept of using two […]
Hadoop Single Node Installation
Pre-requisites Before starting with Hadoop single node installation, make sure that the node have the following pre-requisites: a) Any Linux Operating system b) Sun Java 1.6 or above should already be installed and the version should be same across all the nodes. To install Java, you can refer to the installation steps […]
Hadoop 2 Single Node Installation with YARN
Pre-requisites Before starting with Hadoop 2 single node installation, make sure that the node have the following pre-requisites: a) Any Linux Operating system b) Sun Java 1.6 or above should already be installed and the version should be same across all the nodes. To install Java, you can refer to the installation steps […]
Prepare Node for Hadoop
Hadoop is a distributed processing framework with multiple nodes connected with each other through network. An administrator needs to prepare node for Hadoop, i.e. configure a node to be used as a part of Hadoop cluster. This blog describes a list of prerequisites and how you can configure these prerequisites before using a node as […]
Create a New VM using Oracle Virtualbox
Oracle Virtualbox is a tool to create and host Virtual machines on your system.The virtual machine is known as a Guest Operating system. This presentation covers the steps to Create a New VM using Oracle Virtualbox. To know more about Oracle Virtualbox, click here. Create New VM using Oracle Virtualbox […]
HDFS High Availability Overview
Background Single Point of Failure (SPOF) in HDFS: Each cluster had a single NameNode, and if that machine or process became unavailable, the cluster as a whole would be unavailable until the NameNode was either restarted or brought up on a separate machine. Ecosystem Dependency: The Hadoop ecosystem components like […]
Configure Static IP Address in Ubuntu
You can configure a network adapter of a machine to use a static IP address. To configure static IP address in Ubuntu, you need to edit the /etc/network/interfaces file Open the file with Sudo Option sudo nano /etc/network/interfaces Assuming that the eth1 is the adapter for which static IP address is to be […]