Increase or Decrease the Size of Static Partition in Linux

✍️ Task Description

7.1: Elasticity Task

Increase or Decrease the Size of Static Partition in Linux.

Linux partition:

Partitioning allows you to divide your hard drive into isolated sections, where each section behaves as its own hard drive. Partitioning is particularly useful if you run multiple operating systems. There are lots of powerful tools for creating, removing, and otherwise manipulating disk partitions in Linux.

Why do we need elasticity in Hadoop?

Consider a Hadoop cluster having ’n’ DataNodes contributing their storage to cluster. Consider the case when the storages of DataNodes get full, in that case, we required to attach more DataNodes leading to more RAM and CPU requirements…..Instead, we can solve issue by utilizing LVM concept to provide elasticity to DataNode storages in the Cluster so that we can increase or decrease the size of partitions as the requirement arises.

So, let’s move to the practical part..

I have a already created hdfs cluster running. We can check report of the hdfs cluster using

Step 1: Add one Hard Disk to the VM

In the first step add one hard disk to the VM, we can see the hard disk with the fdisk command.

Step 2: Create one Primary partition of 15GiB

In this step, we will create one primary partition of 1GiB with the help of fdisk command.

Command:  " fdisk  /dev/xvdg"

Step 3: Put some data in the directory

In this step, we will put some data in the directory so that we can check if the data get lost or not after resizing the partition.

Let’s restart the hdfs service in DataNodes. Now, we can see that each datanode is contributing approximately 15 GB to the cluster.

So, with the use of LVM concept we able to increase the datanode storage dynamically!!

Thanks ….



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store