#10801 ensure compose VM's have discard="unmap" to pass down FITRIM to keep images sizes smaller
Closed: Fixed 2 years ago by kevin. Opened 2 years ago by chrismurphy.

  • Describe the issue

Some methods of composing images involve copying RPMs to the compose's backing image for staging. The installation then consumes those RPMs, deletes them, then issues fstrim to remove unused blocks, thus creating a sparse file. If the qemu-kvm VM doesn't pass through the FITRIM ioctl, then this garbage collection won't happen, meaning these deleted files remain in the image, must be compressed and are downloaded by everyone globally.

The XML for the VM's virtio disks should include cache="unsafe" and discard="unmap" - I'm pretty sure the former has been used for some time now. e.g.

<disk type="file" device="disk">
  <driver name="qemu" type="raw" cache="unsafe" discard="unmap"/>
  <source file="/var/lib/libvirt/images/Fedora-Workstation-Live-x86_64-36-1.5.raw"/>
  <target dev="vda" bus="virtio"/>
  <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
</disk>

Background bugs:
https://pagure.io/cloud-sig/issue/335
https://bugzilla.redhat.com/show_bug.cgi?id=1971186

  • When do you need this? (YYYY/MM/DD)

Time unspecific, this is an optimization.

  • When is this no longer needed or useful? (YYYY/MM/DD)

Time unspecific, this is an optimization.

  • If we cannot complete your request, what is the impact?
    Excessive CPU and time to compress deleted files, excessive download sizes and wasted bandwidth by everyone on both sides of the transfer.

Example

  1. Download https://kojipkgs.fedoraproject.org//packages/Fedora-Cloud-Base/Rawhide/20220518.n.0/images/Fedora-Cloud-Base-Rawhide-20220518.n.0.x86_64.raw.xz and unpack it.
  2. Attach the raw file to a virtio drive with discard="unmap" set
  3. Boot the VM by any available means, I used Fedora-Workstation-Live-x86_64-36-1.5.iso
  4. ls -ls Fedora-Cloud-Base-Rawhide-20220518.n.0.x86_64.raw on the host
  5. `mount /dev/vda5 /mnt - on the guest
  6. fstrim -v /mnt - on the guest
    6.ls -ls Fedora-Cloud-Base-Rawhide-20220518.n.0.x86_64.raw - on the host

Before fstrim

783688 -rw-r--r--. 1 root root 5368709120 May 19 11:58 Fedora-Cloud-Base-Rawhide-20220518.n.0.x86_64.raw

After fstrim

457672 -rw-r--r--. 1 qemu qemu 5368709120 May 19 12:03 Fedora-Cloud-Base-Rawhide-20220518.n.0.x86_64.raw

The image is nearly 1/2 the size, ~862M to ~447M. So it's quite a lot of waste.

Metadata Update from @phsmoura:
- Issue tagged with: low-gain, low-trouble, ops

2 years ago

We will be re-installing builders with f36 in the coming weeks. We can add this config on then...

If a test/staging/preliminary round of builders go up, let me know, I'd like to check the results and logs at that time and make sure it's working before you get too far ahead. I'm pretty sure only virtio and scsi controlled devices are supporting the unmap setting to pass through discards from guest to host.

One thing here thats odd... livemedia for example is also made on buildhw hosts (ie, bare metal fedora installs). I would think discard would be on by default there, no?

This is done. All the buildvm's were re-installed with f36 and with this set. ;)

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata
Boards 1
Ops Status: Backlog