Logical volume extension is used to provide additional storage and facilitate the continued operations of the virtual machine. This article will tell the process of Qcow2 type logical volume extension with thin provision allocation.
Here is the process of LV extension:
1. When the virtual disk image is stored on storage with Qcow2 format, Qemu provides a logical mapping between blocks in VM and blocks on disk.
2. If there is any VM write activity, Qemu writes access the Logical volume, and then qcow2 image grows.
3. Qemu keeps reporting the usages to vdsm. And vdsm has a default threshold( volume_utilization_percent & volume_utilization_chunk_mb) to compare the local storage allocation with actual storage usage. When free space < (100% - volume_utilization_percent%) * volume_utilization_chunk_mb, it means it has exceeded the threshold limit.
294 # Together with volume_utilization_chunk_mb, set the minimal free 295 # space before a thin provisioned block volume is extended. Use lower 296 # values to extend earlier. 297 # volume_utilization_percent = 50 298 299 # Size of extension chunk in megabytes, and together with 300 # volume_utilization_percent, set the free space limit. Use higher 301 # values to extend in bigger chunks. 302 # volume_utilization_chunk_mb = 1024
4. qemu becomes pause and vdsm issues an extend request and writes a message to SPM via storage mailbox to extend LV and keeps monitoring the SPM response in incoming mail every 2s. Related logs can be found in vdsm.log as follows:
2020-09-11 10:11:20,551-0700 INFO (periodic/187) [virt.vm] (vmId='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') Requesting extension for volume 1e4f0bee-1c3b-4f4d-9d4a-bafed6cd773f on domain a1ce94f1-8001-48f6-ade2-cc1ac11660cb (apparent: 42949672960, capacity: 2197949513728, allocated: 42637918208, physical: 42949672960 threshold_state: exceeded) (vm:1277)
2020-09-11 10:11:20,551-0700 INFO (periodic/187) [vdsm.api] START sendExtendMsg(spUUID='2dc32f8e-5b7a-4183-974f-aa96c8eb5bc2', volDict={'newSize': 44023414784, 'domainID': 'a1ce94f1-8001-48f6-ade2-cc1ac11660cb', 'name': 'xxx', 'poolID': '2dc32f8e-5b7a-4183-974f-aa96c8eb5bc2', 'clock': , 'internal': False, 'volumeID': '1e4f0bee-1c3b-4f4d-9d4a-bafed6cd773f', 'imageID': 'd1b292cb-2518-4c89-9730-6b56bae10c90'}, newSize=44023414784, callbackFunc=>) from=internal, task_id=4f42cdc3-a2a7-4b91-93d4-4eab80e57a7f (api:48)
2020-09-11 10:11:20,564-0700 INFO (mailbox-hsm) [storage.MailBox.HsmMailMonitor] HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/2dc32f8e-5b7a-4183-974f-aa96c8eb5bc2/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=4096', 'count=1', 'seek=2'] (mailbox:380)
2020-09-11 10:11:20,551-0700 INFO (periodic/187) [vdsm.api] FINISH sendExtendMsg return=None from=internal, task_id=4f42cdc3-a2a7-4b91-93d4-4eab80e57a7f (api:54)
5. If there is sufficient physical storage space available, the SPM extends the LV and reply with success message. SPM updates the metadata.
6. When the vdsm receives the successful replies to its lv extention request, it refreshes its lv map to recognize the newly allocated volume. Then vdsm will continue VM execution. Related logs can be found in vdsm.log as follows:
2020-09-11 10:11:22,615-0700 INFO (mailbox-hsm/0) [vdsm.api] START refreshVolume(sdUUID='a1ce94f1-8001-48f6-ade2-cc1ac11660cb', spUUID='2dc32f8e-5b7a-4183-974f-aa96c8eb5bc2', imgUUID='d1b292cb-2518-4c89-9730-6b56bae10c90', volUUID='1e4f0bee-1c3b-4f4d-9d4a-bafed6cd773f') from=internal, task_id=405c92c8-3b4b-4ba1-b0ba-c24b72444529 (api:48)
2020-09-11 10:11:22,780-0700 INFO (mailbox-hsm/0) [storage.LVM] Refreshing LVs (vg=a1ce94f1-8001-48f6-ade2-cc1ac11660cb, lvs=('1e4f0bee-1c3b-4f4d-9d4a-bafed6cd773f',)) (lvm:1466)
7. If there is no available physical storage space, the extention will fail and vdsm reports enospc error as follows:
2020-09-11 10:11:21,527-0700 INFO (libvirt/events) [virt.vm] (vmId='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') abnormal vm stop device ua-d1b292cb-2518-4c89-9730-6b56bae10c90 error enospc (vm:5064) 2020-09-11 10:11:21,571-0700 WARN (libvirt/events) [virt.vm] (vmId='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') device xxx out of space (vm:3986) 2020-09-11 10:12:16,882-0700 INFO (libvirt/events) [virt.vm] (vmId='0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') underlying process disconnected (vm:1066) 2020-09-11 10:12:16,882-0700 INFO (libvirt/events) [virt.vm] (vmId='0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') Release VM resources (vm:5168) 2020-09-11 10:12:16,882-0700 INFO (libvirt/events) [virt.vm] (vmId='0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx') Stopping connection (guestagent:455)
In this case, you must manually add storage capacity to expand it. When a new LUN is added to the volume group, the SPM automatically distributes the additional storage to logical volumes that need it. The automatic allocation of additional resources allows the relevant virtual machines to automatically continue operations uninterrupted or resume operations if stopped.