Update: You can find a much better version of this as an article I wrote for the Odroid Magazine:https://magazine.odroid.com/article/linux-logical-volume-manager-lvm2/
I’ve been planning to upgrade my LVM2 2 disk striped setup with a new disk for a while. I am currently on the testing phase but I think I have a good way to do it without losing any data.
I plan to implement this on my Odroid XU4 based Cloudshell 2 which is running Ubuntu 16.0.4.
I used this guide as inspiration: LVM fun
I recommend reading that guide first if you are not familiar with LVM.
Phase 1: Testing with loop devices
The plan is as follows:
- Connect new spare disk via the second USB3.0 port using an external enclosure(the cloudshell only supports 2 SATA drives and both ports are occupied right now)
- Create a PV(physical volume) on the new disk
- Add new PV to existing VG(volume group)
- unmount all VG volumes and/or freeze allocation on the PV to migrate
- pvmove one of the 2 existing PVs onto this new PV
- Leave it overnight since it is going to copy sector by sector a 2 TB disk
- Reduce VG by removing old PV(the one moved at the previous step)
- Shutdown and swap out old disk with new one
- Boot and check that the LVs(logical volumes) are correctly mapped to the PVs
OK, here is how I tested.
First, create 3 loopback devices backed by 3 empty files.
1 2 3 4 5 6 7 8 9 10 |
dd if=/dev/zero of=/tmp/hdd1.img bs=1G count=1 dd if=/dev/zero of=/tmp/hdd2.img bs=1G count=1 # twice the space dd if=/dev/zero of=/tmp/hdd3.img bs=1G count=2 losetup -f losetup /dev/loop0 /tmp/hdd1.img losetup /dev/loop1 /tmp/hdd2.img losetup /dev/loop2 /tmp/hdd3.img |
Now, create PVs on all these devices
1 2 3 |
pvcreate /dev/loop0 pvcreate /dev/loop1 pvcreate /dev/loop2 |
Add the first 2 volumes to a VG:
1 |
vgcreate vgtest /dev/loop0 /dev/loop1 |
Check stuff with:
1 2 3 4 |
pvs vgs pvdisplay |
Create 2 striped volumes as such:
1 2 |
lvcreate -i2 -I4 -L1G -nlvdata1 vgtest lvcreate -i2 -I4 -l100%FREE -nlvdata2 vgtest |
The first one will have 1Gb and it will be 50/50 split across the 2 PVs.
The second one will fill up the existing space and also be split between the 2 PVs(size will be around 2.9Gb).
OK, so this is the initial setup, we have 2 disks with 2 logical volumes spread among them using striping.
Format the “disks” and write some files to them for testing( I used mkfs.ext4 on the virtual devices to create filesystems). When done, unmount them and let’s migrate the data.
Migrating the data to disk 3 from disk 2:
1 2 3 4 5 |
# Add new device to VG vgextend vgtest /dev/loop2 # Migrate data pvmove /dev/loop1 /dev/loop2 |
In the real world this is the step that takes forever. I noticed that if interrupted there is a good chance that the process can be resumed. So power outages might not result in data loss.
Resize PV to fill space on /dev/loop2(it has a size of 2GB)
1 |
pvresize /dev/loop2 |
Bonus round: create an unequal stripe by using the extra space at the end of /dev/loop2(perfomance loss, auch…)
1 |
lvextend -i1 -l +100%FREE /dev/vgtest/lvdata2 |
You will end up with this setup:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
lvdisplay -m /dev/vgtest/lvdata2 --- Logical volume --- LV Path /dev/vgtest/lvdata2 LV Name lvdata2 VG Name vgtest LV UUID vDefWQ-1ugy-1Sp5-T1JL-8RQo-BWqJ-Sldyr2 LV Write Access read/write LV Creation host, time odroid, 2018-03-06 17:43:44 +0000 LV Status available # open 0 LV Size 1.99 GiB Current LE 510 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:2 --- Segments --- Logical extents 0 to 253: Type striped Stripes 2 Stripe size 4.00 KiB Stripe 0: Physical volume /dev/loop0 Physical extents 128 to 254 Stripe 1: Physical volume /dev/loop2 Physical extents 128 to 254 Logical extents 254 to 509: Type linear Physical volume /dev/loop2 Physical extents 255 to 510 |
Now, we are going to throw away /dev/loop1.
1 2 3 |
vgreduce vgtest /dev/loop1 losetup -d /dev/loop1 |
In the real world this would be: unmap the PV and pull the physical disk, this is where you edit your fstab so the OS does not try to mount automatically(for now) and reboot.
We are now going to simulate a device name reallocation:
1 2 3 4 5 |
# Disk 3 becomes offline losetup -d /dev/loop2 # Disk comes back as different device losetup /dev/loop1 /tmp/hdd3.img |
If you run pvs, vgs and lvs your volumes should be intact!
Now mount the volumes and check that your test files are still there.
In theory, this is the process but I have yet to try it on real data.
1 2 3 4 |
# pvs PV VG Fmt Attr PSize PFree /dev/loop0 vgtest lvm2 a-- 1020.00m 0 /dev/loop1 vgtest lvm2 a-- 2.00g 0 |