lvm – The Blog of Nitin J Mutkawoa, AKA @TheTunnelix

Tag: lvm

Recover logical volumes data from deleted LVM partition

04/18/2019 Nitin J Mutkawoa Add Comment

Have you ever deleted a logical volume by accident? Can you recover it looking into the backups? Well, the answer is YES. For those who are not familiar with Logical Volume Management (LVM) is a device mapper target that provides logical volume management for the Linux kernel.- Wikipedia. It is an abstraction layer or software that has been placed on top of your hard drive for flexible manipulation of the disk space. Some of the articles published in the past on LVM are:

All test carried out on this blog post have been tested on a CentOS machine. Please don’t make a live test on a production server.

1. So, as you can see below I have an lv called lvopt which is from a vg called centos.

2. Same is mounted on the /opt

3. There are some data in that partition as well:

4. I created a directory inside the /opt directory

5. Now, let’s pretend to remove the lvm lvopt. Or say, someone did it by accident because it was unmounted. The command lvremove will be used here to remove the lv. Note: that the lv need to be unmounted.

6. If you make an lvs, lvdisplay or vgs or even mount again the partition, you cannot do it. The data is lost. But you can still recover it. This is because the lvm contains the archive of your lv inside the folder /etc/lvm/archive. But, you cannot read the files directly.

7. But you can still, interpret part of the files. Since we deleted the volume group called “centos”, we knew that it is referenced in the file centos_… The question that arises here is which file is relevant for you. Right? So to understand which archive you want to restore, you need to use the command vgcfgrestore –list <name of volume group>. Here is an example:

8. If you observe carefully, each archive has been backup at a certain time. In my case, I deleted the LV on 18-Apr-2019 at 11:17:17 2019:

9. So, I want to restore from that last archive. You will need to copy the full patch of the vg file. In my case it is /etc/lvm/archive/centos_00004-1870674349.vg. The goal here is to restore the lv before this specific time, or simply restore back the lv before the command lvremove was fired. Here is the command:

10. If you launch the command lvs, you will notice the presence of the lv.

11. But, mounting back the lv won’t result in anything. This is because the lv is inactive. You can see it with the command lvscan. Please take note below that the lvopt is inactive.

12. To activate it you simply need to use the command lvchange.

13. Mount it back and you are done.

I believe this can be very useful especially when you have encountered a situation where someone deleted an lv. I hope you enjoy this blog post. Please share and comment below if you like it.

Linux Application, Linux System centos, linux, lvm

Configure your LVM via Ansible

03/08/2016 Nitin J Mutkawoa 4 Comments

Some days back, I gave some explanations about LVM such as creations of LVM partitions and a detailed analogy of the LVM structure as well as tips for using PVMOVE. We can also automate such task using the power of Ansible. Cool isn’t it?

So, I have my two hosts Ansible1 and Ansible2. Ansible1 is the controller and has Ansible installed and Ansible2 is the hosts that the disk will be added to the LVM.

1. Here is the status of the disk of Ansible2 where a disk /dev/sdc has been added

2. I have now added a disk of 1GB from the VirtualBox settings. You can refer to the past article on LVM how to add the disk. As we can see on the screenshot below it shows the disk sdc with the size 1GB added on the machine Ansible2 which I have formatted as LVM

4. Lets now get into the controller machine – Ansible1 and prepare our Playbook. You can view it on my Git account here. The aim is to get a 500Mb from the /dev/sdc1 to create a new VG called vgdata in the LV called lvdisk.

5. Here is the output

Articles on LVM

Articles on Ansible

Linux Application, Linux System, Scripts and codes Ansible, Automation, centos, lvm, VirtualBox

Move your /home to a new LVM partition

10/21/2015 Nitin J Mutkawoa Add Comment

To have better control and security over your Linux OS, you might want to move your say /home or/var to another LVM partition. The advantage is that you can easily increase/decrease the size at a future stage.

On this article, I will take an example of the /home directory and we will move it to a fresh disk on Virtual Box. Here is an example of a df -h on my Virtual Machine.

We will now add a new disk [sdc] by creating another VG.

Start with the following steps:

- pvcreate /dev/sdc

- vgcreate vghome /dev/sdc

- lvcreate -l 100%FREE vghome

lvrename /dev/vghome/lvol0 /dev/vghome/lvhome

Once you have successfully created the lvhome, you will need to edit your /etc/fstab before mounting the partition so that each time you reboot your machine, it will render the same configuration.

I have inserted the following parameters:

/dev/mapper/vghome-lvhome /home ext4 defaults,noatime 1 2

You will also need to format the partition before mounting the disk with this command :

mkfs.ext4 /dev/mapper/vghome-lvhome

You now need to mount the partition by using the following command mount /home

Here is now the results from a df -h

Tips:

You can also format your partition with ext3 when your /etc/fstab has been specified with the ext4 format as ext4 support ext3
- You can also specify the name of the lv directly whilst creating it by using the command lvcreate -l 100%FREE -n lvhome vghome so that you don’t need to rename anew the lv. By default on Vbox and VMware, it uses lvol0
More articles I have posted on LVM are :

Add and extend disk on Virtual Box through LVM

Managing LVM with Pvmove – Part 1

Managing LVM with Pvmove – Part 2

Linux System linux, lvm

Add and extend disk on Virtual Box through LVM

10/16/2015 Nitin J Mutkawoa Add Comment

You can easily add and extend disk on Virtual Box through some LVM manipulations. LVM (Logical Volume Partitioning) is a device mapper target that provides logical volume management for the Linux kernel. – Wikipedia. However, I have written a brief introduction about LVM on a previous post – Managing LVM with pvmove – Part 1.

Prior the extension is made you need to assure yourself there that you already know the actual state of the machine’ s hard disk.

Those commands are helpful to perform your analysis before the operation is carried out.

>> fdisk -l

> pvdisplay >> vgdisplay >> lvdisplay

>> vgs >> lvs >> vgs

>> lsblk

Here is the state of the disk before the operation is carried out.

Now, you can get into your Oracle VM VirtualBox Manager to add the new disk.

The steps are :

Click on the ‘Settings’ option on the VirtualBox Manager after having selected your virtual machine which you intend to perform a disk extension. In my case, it’s the ‘centos6’ one.

Then, on the ‘Storage’ option, next to the “Controller: SATA” there is an icon to “add new hard disk”.

Once you have click on the “add new hard disk” it will prompt you to “cancel” “choose existing disk” and “create new disk”. Choose “create new disk”. Of course, you can also choose an existing disk, but here we are adding a completely new fresh disk.

Afterward, it will prompt a “create Virtual Hard Drive” box. Choose “VDI”. Click on next, then on “dynamically allocated”. Give a new name to your hard disk. In my case, I am adding a new 2GB hard disk. Click on create and you are done.

Boot your machine if you are on VirtualBox, then fire the lsblk command to see your new hard disk. See screenshot below. You can also check with the fdisk -l command as well as the dmesg log which is really helpful.

Once the disk is detected, start by converting the disk to the PV using the command pvcreate /dev/sdb. You will notice that if you launch again a pvs the new disk is now on the PV but no part of the PV is allocated to any VG. As you can see in the picture below here is the new sdb which now forms part of the PV

Now we will extend the actual VG called vg_labo. Use the command vgextend vg_labo /dev/sdb

Once this is completed, you can now choose which LV you will extend. I am choosing the LV called lv_root. Use the command lvextend -l +100%FREE /dev/vg_labo/lv_root

The disk is now extended. You can also verify with the command df -h. You can also check out the following article on how to perform a pvmove.

Tips:

- On Virtual Box, you cannot add a new disk if your machine is running compared to VMware. To be able to solve that issue, you will need to shut down the machine to be able to add the disk.

If ever after adding a new hard disk, you noticed that the disk is not being detected just stay cool, as you might need to troubleshoot between LUNs on VCenter. Use the following command:

ls /sys/class/scsi_host/ | while read host ; do echo “- – -” > /sys/class/scsi_host/$host/scan ; done

You can also use the script rescan–scsi–bus.sh after having to install the sg3_utils package to troubleshoot for LUN detection.

Linux System lvm, VirtualBox

Managing LVM with PVMOVE – Part 2

09/30/2015 Nitin J Mutkawoa Add Comment

After a little introduction about LVM from the article Managing LVM with PVMOVE – Part 1, its time to get into the details of the pvmove command. Based on the scenario and constraints described in part 1, that I will elaborate on the pvmove operation here.

Before handling any operation, do your precheck tasks such as verification of the state of the server, URLs, services and application running, the port they are listening etc.. This is done to be able to handle complicated tasks both at the system and application level. However, in respect of the pvmove operation, I would recommend you to fire the vgs, pvs and lvs commands as well as a fdisk -l to check for the physical disk state. Do a df -h and if possible; an lsblk to list all blocks for future references. On CentOS / RedHat you need to install the package util-linux to be able to use lsblk which is very interesting.

Let us say we have a disk of 100G [lets called it sdb] and we have to replace it with another disk of 20G [sdc]. We assume that there is 10G of data being used out of the 100G hard disk which looks reasonable and possible to be handled by the 20G hard disk that we have planned to shift to. On our Sdb we have 2 LV lets call it lvsql that is being used by MySQL having the database there and lvhome handling the rest of the server applications. See illustration from the diagram on the right. Both LVs are found in the same VG called VGNAME.

So you might ask yourself how come if you perform a df -h on your machine you find that lvsql total size is 15G and that of lvhome is 80G making a total of 95G when the hard disk is a 100G [As described on the diagram above]. Is there 5G missing? The answer is no. When you fire the command pvdisplay, you will notice that the “PE Size” had consumed 5 GB. For example on this screenshot on the left, the PE Size is 4MB.

Usually, the 5Gb missing will be allocated there which means that the PE Size is used for a specific purpose for example in our situation a back up of the MySQL database or other processes. If the missing size is not found it means that it’s still in the VG and has not been allocated to the LV. It’s important to allocate some additional space there. So before start do a pvdisplay, lvdisplay, and vgdisplay as well which are important. We now have our sde hard disk as described in this picture below.

How to start? It’s up to you to decide which lv you want to allocate more space as you have control over the vg. You can also add another physical disk and extend the vgname and resize the lvsql since a database usually grows in size.

Do your pre-check tasks as described.

Stop all applications running such as MySQL and Atop in our case. You can use the lsof command to check if there are still processes and application writing on the disk.
Once all applications have been stopped you can type the mount command to check the presence of the partitions as well as you have to confirm that there is no application writing on the disk – use lsof or fuser.
Comment the two lines related to the 2 vg partitions in the /etc/fstab file. Usually it would be the lines /dev/vgname/lvsql and /dev/vgname/lvhome. You can now umount the disk.
You would notice that it would not be possible to umount the partitions if ever an application is writing on it. Another issue is that if you have ssh running on the machine, you need to stop it! Then, how do you ssh on the machine? Use the console from vSphere or if it’s a physical machine boot it with a live cd.
Next step is to do a lvreduce of the lv if possible according to the size. In our case 5GB being used out of 80. Do a lvreduce -L 7GB –resizefs /dev/vgname/lvhome. This is done because when you will move the pv it will be faster. The bigger the lv the more time it takes.
Once all lv size has been reduced to the maximum, add the disk sdc. Make sure it gets detected if you are on VMware. Use the fdisk command to check and compare from your precheck test.
Now create from the sdc a pv The command is pvcreate /dev/sdc. This means that you created a pv from the disk you have added.
After the pv has been created extend the vg called vgname from the old disk (sdb) by using the disk sdc which you just added to the same old vg called vgname. Command is vgextend vgname /dev/sdc
Now the magic starts, fire a pvmove /dev/sdb /dev/sdc – This means that you are moving the pv allocated to the vgname belonging to the PEs of hard disk sdb into sdc.
When the pvmove is completed, you just need to do a vgreduce vgname /dev/sdb. When you launch the vgreduce it will throw out the old disk out from the VG as you have reduced it completely. You can now remove the old disk.
Since you have reduced the lvhome you will notice that lvhome is at 7GB instead of 10G as we have reduced size in step 4 to accelerate the pvmove process. The step is to lvresize -l +100%FREE /dev/vgname/lvhome. You will notice that the PE Size is intact as we had not resized the lvsql.
You can now do a /sbin/resize2fs /dev/vgname/lvhome. Uncomment back the lines on fstab, mount -av and restart your applications.

Congrats you just did a pvmove. Comments below so that we can clear any ambiguities.

Linux System linux, lvm