Create Linux Software Array Ubuntu Server 14.04

Ubuntu Linux Specific Guides
Post Reply
User avatar
dedwards
Site Admin
Posts: 69
Joined: Wed Mar 15, 2006 8:28 pm
Contact:

Create Linux Software Array Ubuntu Server 14.04

Post by dedwards » Sat Mar 20, 2010 3:53 pm

This guide is assuming you are starting with brand new blank hard drives. If you are not and you decide to follow this guide be advised that the procedures in this guide will COMPLETELY ERASE ALL THE DATA ON YOUR HARD DRIVES!!

First things first. Take an inventory of all your hard drives in your system and identify the ones you will be mounting into your array by issuing the command below:

fdisk -l|more

You should get an output similar to below:

Code: Select all

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System


Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0e5676d3

   Device Boot      Start         End      Blocks   Id  System

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xdec498c8

   Device Boot      Start         End      Blocks   Id  System


Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x196add63

   Device Boot      Start         End      Blocks   Id  System

Disk /dev/sde: 40.0 GB, 40018599936 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00016bf2

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1        4660    37429248   83  Linux
/dev/sde2            4660        4866     1648641    5  Extended
/dev/sde5            4660        4866     1648640   82  Linux swap / Solaris


As you can see from the listing above, in my case, I have four hard drives (/dev/sda, /dev/sdb, /dev/sdc/, /dev/sdd, /dev/sde) in my system. Out of all those drives, four of them (/dev/sda, /dev/sdb, /dev/sdc, /dev/sdd) are not partitioned and one (/dev/sde) is partioned with three partitions (/dev/sde1, /dev/sde2 and /dev/sde5). Drive /dev/sde is my boot drive and the three partitions correspond to my root (/), my swap and my boot (/boot) partitions. So, knowing this, I will make sure that I do not do anything my boot drive. You system drive assignments may differ so make ABSOLUTELY sure you do not make any changes to your boot drive when you follow this guide or you system will not boot and you will lose all your data.

So, since I have identified the drives I will be using for my array, I'm going to go ahead and partition them using parted. So for EACH drive you will be using for your array, perform the following procedure:

Code: Select all

parted -a optimal /dev/sda


At the (parted) prompt, list the partitions:

Code: Select all

print


IF there are any partitions, you will get an output similar to below, notice the "Number" is the partition number:

Code: Select all

Model: DELL PERC 6/E Adapter (scsi)
Disk /dev/sda: 7195GB
Sector size (logical/physical): 512B/512B Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags
1      17.4kB  7195GB  7195GB  ext3         primary  msftdata



Remove the partition(s):

Code: Select all

rm 1


Create a partition:

Code: Select all

mklabel gpt


You will get the following prompt, enter "y" without the quotes:

Code: Select all

Warning: The existing disk label on /dev/sda will be destroyed and all data on this disk will be lost. Do you want to continue?Yes/No? yes


Verify the partition was created:

Code: Select all

print


You should get the following, make note of the size of the partition in this case 7195GB:

Code: Select all

Model: DELL PERC 6/E Adapter (scsi)
Disk /dev/sda: 7195GB
Sector size (logical/physical): 512B/512B Partition Table: gpt


Change the default unit to GB:

Code: Select all

unit GB


Create a partition equal to the maximum size as per the output earlier (7195GB):

Code: Select all

mkpart primary 0.00GB 7195GB


Set the raid flat to on:

Code: Select all

set 1 raid on


List the partitions to verify:

Code: Select all

Number  Start   End     Size    File system  Name     Flags
1      0.00GB  7195GB  7195GB               primary      raid


Save the changes and quit:

Code: Select all

quit



You use the mdadm command with the --create option to create the RAID set. In this example we use the --level option to specify RAID 5, and the --raid-devices option to define the number of partitions to use.

Code: Select all

mdadm --create --verbose /dev/md0 --level=5  --raid-devices=3 /dev/hde1 /dev/hdf2 /dev/hdg1


Code: Select all

mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: /dev/hde1 appears to contain an ext2fs file system
size=48160K  mtime=Sat Jan 27 23:11:39 2007
mdadm: /dev/hdf2 appears to contain an ext2fs file system
size=48160K  mtime=Sat Jan 27 23:11:39 2007
mdadm: /dev/hdg1 appears to contain an ext2fs file system
size=48160K  mtime=Sat Jan 27 23:11:39 2007
mdadm: size set to 48064K
Continue creating array? y

mdadm: array /dev/md0 started.


Confirm RAID Is Correctly Inititalized.
The /proc/mdstat file provides the current status of all RAID devices. Confirm that the initialization is finished by inspecting the file and making sure that there are no initialization related messages. If there are, then wait until there are none.

Code: Select all

cat /proc/mdstat


Code: Select all

Personalities : [raid5]
read_ahead 1024 sectors
md0 : active raid5 hdg1[2] hde1[1] hdf2[0]
4120448 blocks level 5, 32k chunk, algorithm 3 [3/3] [UUU]
unused devices: <none>


Notice that the new RAID device is called /dev/md0. This information will be required for the next step.

Format RAID partition
Your new RAID partition now has to be formatted. The mkfs.ext3 command is used to do this.

Code: Select all

mkfs.ext4 /dev/md0


Code: Select all

mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
36144 inodes, 144192 blocks
7209 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
18 block groups
8192 blocks per group, 8192 fragments per group
2008 inodes per group

Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables: done                           

Creating journal (4096 blocks): done

Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.


Create the mdadm.conf Configuration File

Your system doesn't automatically remember all the component partitions of your RAID set. This information has to be kept in the mdadm.conf file. The formatting can be tricky, but fortunately the output of the mdadm --detail --scan --verbose command provides you with it. Here we see the output sent to the screen.

Code: Select all

mdadm --detail --scan


Code: Select all

ARRAY /dev/md0 level=raid5 num-devices=4 

UUID=77b695c4:32e5dd46:63dd7d16:17696e09

   devices=/dev/hde1,/dev/hdf2,/dev/hdg1


Here we export the screen output to create the configuration file.

Code: Select all

mdadm --detail --scan >> /etc/mdadm/mdadm.conf


Create A Mount Point For The RAID Set

The next step is to create a mount point for /dev/md0. In this case we'll create one called /mnt/raid

Code: Select all

mkdir /mnt/raid


Next get the UUID of the array by issuing the following command:

Code: Select all

ls -l /dev/disk/by-uuid


You should get an output similar to the one below:

Code: Select all

total 0
lrwxrwxrwx 1 root root  9 Oct 14 06:00 09cf1a3f-693e-468e-9e1d-89191c19ee3a -> ../../md0
lrwxrwxrwx 1 root root 10 Oct 14 04:11 25149e7f-b472-4592-8aeb-c2c13b010ac1 -> ../../sdi1
lrwxrwxrwx 1 root root 10 Oct 14 04:11 58a1e1b5-3321-4851-9be0-b760fb68dbd7 -> ../../sdi5


Make a note of the UUID of the array, in our case the UUID is "09cf1a3f-693e-468e-9e1d-89191c19ee3a"

Edit The /etc/fstab File

The /etc/fstab file lists all the partitions that need to mount when the system boots. Add an Entry for the RAID set, the /dev/md0 device.

Code: Select all

UUID=09cf1a3f-693e-468e-9e1d-89191c19ee3a /mnt/raid               ext4    errors=remount-ro 0       1


The /dev/hde1, /dev/hdf2, and /dev/hdg1 partitions were replaced by the combined /dev/md0 partition. You therefore don't want the old partitions to be mounted again. Make sure that all references to them in this file are commented with a # at the beginning of the line or deleted entirely.

#/dev/hde1 /data1 ext3 defaults 1 2

#/dev/hdf2 /data2 ext3 defaults 1 2

#/dev/hdg1 /data3 ext3 defaults 1 2

Mount The New RAID Set

Use the mount command to mount the RAID set. You have your choice of methods:

The mount command's -a flag causes Linux to mount all the devices in the /etc/fstab file that have automounting enabled (default) and that are also not already mounted.

Code: Select all

mount -a


You can also mount the device manually.

Code: Select all

mount /dev/md0 /mnt/raid


Check The Status Of The New RAID

The /proc/mdstat file provides the current status of all the devices.

[code]raidstart /dev/md0

cat /proc/mdstat

Personalities : [raid5]
read_ahead 1024 sectors
md0 : active raid5 hdg1[2] hde1[1] hdf2[0]
4120448 blocks level 5, 32k chunk, algorithm 3 [3/3] [UUU]
unused devices: <none>
Post Reply