Summary

  Subject:

Rocky Linux LVM Concepts and Examples for a NFS Server

   Updated:

2023-01-15

   Author:

Tim Hammond ([email protected])

   Operating Systems:

Rocky 8

Rocky 9

   Background:

This guide will demendatrate how to setup and modify LVM volumes. The real purpose of this was to setup storage for a dedicated NFS server. Enjoy!

  

Table of Contents

VM Config

VM Specifications:

  • CPU: 1vCPU (4 cores)
  • RAM: 8 GB
  • HD: 100 GB (OS)
  • HD: 200 GB (File Server

Change the hostname:

sudo hostnamectl set-hostname nfs01.domain.local

Update the IP:

sudo sed -i 's/192.168.30.199/192.168.30.102/g' /etc/sysconfig/network-scripts/ifcfg-ens192 sudo cat /etc/sysconfig/network-scripts/ifcfg-ens192

LVM Concepts and Commands

Logical Volume Manager (LVM) is a way to span multiple drives or partitions together to build redudancies or performance bennefits. In the beginning they can be very confusing. I have tried to give basic descriptions/defintions below. In our example, we have a virtual machine that has two drives. One drive is for the Operating System. The other drive is for a NFS file server. We will also cover a few ways to increase the size of the NFS volume.

Physical Volume (PV)

Defintion: Think of this a physcial volume such as a disk or partition.

Command: The command to list the physcial volumes is: pvscan

Volume Group (VG))

Defintion: An agreages of of PVs put together.

Command: The command to list the volume groups is: vgscan

Logical Volume (LV)

Defintion: Think of this as a virtual partion. A VG still requires formatting. A VG can be resized.

Command: The command to list the logical volumes is: lvscan

Addtional Commands

List block devices in table format: lsblk

List all LVM block devices: lvmdiskscan

List the partiontions for sdb: fdisk -l /dev/sdb

List the sizes of partiontions: df -h

List the size of LVM: df -h /mnt/nfs

List the partition table for sdb: parted /dev/sdb print

Create a Partition

While there are many partitioning tools out there, for this, we will be using parted. Create new GPT partition and set the ID type to 8E for System Linux LVM. Create partition interactivly, type:

sudo parted

Select the disk

select /dev/sdb

Show current table

print

Make the partition GPT

mklabel gpt

Create pirmary partition that starts at 1 and used the whole disk (by using the -1).

Source: http://www.troubleshooters.com/linux/parted_cheat.htm

mkpart primary 1 -1

Make partition LVM

set 1 lvm on

Check if the first partition is correctly aligned

align-check optimal 1

To quit parted

quit

Create LVM

Create a PV for the following partiontions

sudo pvcreate /dev/sdb1

Create a volume group
Syntanx: vgcreate name device

sudo vgcreate vg-nfs /dev/sdb1

Create LV from VG
Syntanx: lvcreate -n name -l size vgNameSource
Some engineers use -l which stands for extents.

sudo lvcreate -n lv-nfs --extents 100%FREE vg-nfs

Format LVM

Format using XFS

sudo mkfs.xfs /dev/vg-nfs/lv-nfs

To display the existing label for an unmounted XFS file system and then apply a new label:

sudo xfs_admin -l /dev/vg-nfs/lv-nfs

To change the xfs label

sudo xfs_admin -L "nfs" /dev/vg-nfs/lv-nfs

Mount LVM

Create the mount points.

sudo mkdir -p /mnt/nfs

Mount the LVMs.

sudo mount /dev/vg-nfs/lv-nfs /mnt/nfs

Create the folders in the mount. Assuming they do not already exist AFTER they have been mounted.

sudo mkdir -p /mnt/nfs/isoz

Appeend the following items to the fstab config file so that the LVMs are mounted after reboots.

sudo tee -a /etc/fstab >/dev/null <<EOF /dev/vg-nfs/lv-nfs /mnt/nfs xfs defaults 0 0 EOF

Mount anything that is in the fstab that isn't mounted

sudo mount -a

LVM Expansion - Notes

When it comes to LVM, there are two options on how to increase disk space.

1. Add more disk space to the VMDK. Create a new partition on the disk. You will need to partition it then create a pv out of it. Then add the new pv to the lv. The VG will recognize the changes automagically.

The upside to this is that it is easier to expand an LVM while still online.

The difficulty with this that it adds extra layers of complexity and a drive can only handle four primary partitions. If more room is needed, a new VMDK drive will need to be created. Then you can repeat the steps above.

2. Add more disk space to the VMDK. Unmount the LVM, delete the partition and recreate it with the new size. Then update the PV and LV. The VG will recognize the changes automagically.

The upside here is that it keeps things simple. It is assumed that at this point, there is only one PV in the LVM. This process can repeated as many times as required.

The downside to this is that things need to be taken offline and it is scary as all get out deleteing a partition. Don't worry, if done correctly, there will not be any data loss ;)

Bottomline: in my humble opinion, keep things simple. Use option 2.

LVM Expansion - Pre-Requirements

Resize drive that is to be extended in the vCenter. With modern version of Linux, this can often be done on the fly. If the text box to increase the drive size is greyed out, a shutdown will be required.

When expanding the drive in VMware vCenter, do not use TB. Use GB based on the conversions below.

Example:

  • 1 TB = 1024 GB
  • 2 TB = 2048 GB
  • 4 TB = 4096 GB
  • 6 TB = 6144 GB
  • 8 TB = 8192 GB
  • 10 TB = 10240 GB
  • 12 TB = 12288 GB
  • 14 TB = 14336 GB
  • 16 TB = 16384 GB
  • 18 TB = 18432 GB
  • 20 TB = 20480 GB

For a hirearchy view of the drive and partition layout, run the following command.

sudo lsblk

Run the following command to see the size used/available for each drive/partition.

sudo df -h

Run the following command to see the physical volumes.

sudo pvdisplay

Run the following command to see the volume groups.

sudo vgdisplay

Run the following command to see the logical volumes.

Example:

  • LV Path: /dev/vg-nfs/lv-nfs
  • LV Name: lv-nfs
  • VG Name: vg-nfs
sudo lvdisplay

You can view the XFS volume details as follows.

sudo xfs_info /dev/vg-nfs/lv-nfs

If new size is not visible from the lsblik command, re-scan of iscsi bus.

sudo su - ls /sys/class/scsi_disk/ echo "1" > /sys/class/scsi_disk/0:0:0:0/device/rescan echo "1" > /sys/class/scsi_disk/1:0:0:0/device/rescan exit

Before you can umount the LVM, services that use the LVM must be stopped other wise you will get an error message: "umount: /mnt/nfs: target is busy."

sudo systemctl stop nfs-server sudo systemctl stop rpcbind

Assuming you are making changes to a non-OS LVM, unmount the LVM. This is a safe way to avoid any writes to the LVM during changes.

sudo umount /dev/vg-nfs/lv-nfs

LVM Expansion - Add new PV (Option 1)

After the the drive has has been expaned (the VMDK), we will use this space to create a new partition.

sudo fdisk /dev/sdb

Print the partition table.

print

Create a new partition.

new

Choose the partition number. Remeber you can only have a maximum of four primary partitions.

Partition number (1-128, default 1): <enter>

Choose where you would like the disk to begin. You can normally hit enter to start at the end of the last partition.

First sector (34-419430366, default 2048): <enter>

Choose where you would like the disk to end. If you want to use the new maximum size, just hit <enter>.

Last sector, +sectors or +size{K,M,G,T,P} (2048-419430366, default 419430366): <enter>

Set the partition type.

type

Select the newly created partition. I think 2 is selected as the "default" partition because that is what we worked with last.

Partition number (1,2, default 2): <enter>

If you type L during the wizard, a long list of partition types is provided. We want 31 which is the Linux LVM type.

31

Write the config and exit.

write

After you have created your new partition, you may get a message about the kernel still using the old partition table, and to reboot to use the new table. The reboot is not needed as you can also rescan for those partitions using partprobe. Run the following to scan for the newly created partition.

If that does not work, you can try to use partx command to rescan the device and add the new partitions. For example: partx -v -a /dev/sdb

sudo partprobe -s

Convert new partition to physical volume.

sudo pvcreate /dev/sdb2

Add the new PV to the VG.

sudo vgextend vg-nfs /dev/sdb2

Extend the logical volume to use all of the remaining disk space.

sudo lvextend -l +100%FREE /dev/vg-nfs/lv-nfs

Remount the LVM for the next step.

sudo mount /dev/vg-nfs/lv-nfs /mnt/nfs

For xfs, force the OS to see the extended space:

sudo xfs_growfs /dev/mapper/vg--nfs-lv--nfs

For ext3/ext4, force the OS to see the extended space.

sudo resize2fs /dev/mapper/vg--nfs-lv--nfs

It is probably wise to do a good ol' reboot.

sudo reboot

LVM Expansion - Resize PV (Option 2)

!!!IMPORTANT!!!

Delete and recreate the partition. Don't freak out, we will recreate it with new geometry. No data will be lost during this process.

The encrypted partitions signature will be detected and you will be prompted to remove the signature, DO NOT REMOVE THE SIGNATURE OR YOUR DATA WILL BE IRRECOVERABLE, enter 'n'.

If you do not get the question about the removal of the LVM signature there is something wrong and you should abort without writing by pressing 'q'.

sudo fdisk /dev/sdb

Print the partition table.

print

Delete the partition. If there is only one partition, it will be automatically selected.

delete

Create a new partition.

new

Match the same beginning sector. It is assumed it is the only partition on the disk.

Partition number (1-128, default 1): <enter>

Make BLOODY POSITIVE that the start of the disk is as shown in the printed partition table.

First sector (34-419430366, default 2048): <enter>

If this is the only partition on the disk, you can safely hit <enter>.

Last sector, +sectors or +size{K,M,G,T,P} (2048-419430366, default 419430366): <enter>

See !!!IMPORTANT!!! above.

Created a new partition 1 of type 'Linux filesystem' and of size 7 TiB. Partition #1 contains a LVM2_member signature. Do you want to remove the signature? [Y]es/[N]o: n

Set the partition type. In this case, we will use Linux LVM which is 31 if you type L in the wizard.

type 31

Write the config and exit.

write

After you have created your new partition, you may get a message about the kernel still using the old partition table, and to reboot to use the new table. The reboot is not needed as you can also rescan for those partitions using partprobe. Run the following to scan for the newly created partition.

If that does not work, you can try to use partx command to rescan the device and add the new partitions. For example: partx -v -a /dev/sdb

sudo partprobe -s

Apply the partition changes to the PV.

sudo pvresize /dev/sdb1

Extend the logical volume to use all of the remaining disk space.

sudo lvextend -l +100%FREE /dev/vg-nfs/lv-nfs

Remount the LVM for the next step.

sudo mount /dev/vg-nfs/lv-nfs /mnt/nfs

For xfs, force the OS to see the extended space:

sudo xfs_growfs /dev/mapper/vg--nfs-lv--nfs

For ext3/ext4, force the OS to see the extended space.

sudo resize2fs /dev/mapper/vg--nfs-lv--nfs

It is probably wise to do a good ol' reboot.

sudo reboot

References

Most of this material is based on previous progjects and from iterations of Linux NFS servers.