This came up while working on https://pagure.io/appliance-tools/pull-request/15. When doing cross-arch builds with mock, it uses qemu-user-static under the hood, and that seems to create some weirdness around loop devices that confuses the btrfs tools. Here's a repro, to be run on an x86_64 fc33 or rawhide system:
qemu-user-static
$ sudo dnf install mock qemu-user-static wget $ sudo usermod -a -G mock $USER $ newgrp mock $ mock --root fedora-rawhide-armhfp --install btrfs-progs util-linux $ mock --root fedora-rawhide-armhfp --chroot 'rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && losetup /dev/loop9 foo.img && mkfs.btrfs /dev/loop9 && mkdir /foo && mount /dev/loop9 /foo && btrfs subvol create /foo/subvol && umount /foo && losetup -d /dev/loop9'
The output of the last command will be:
INFO: Unable to build arch armv7hl natively on arch x86_64. Setting forcearch to use software emulation. INFO: mock.py version 2.4 starting (python version = 3.9.0)... Start(bootstrap): init plugins INFO: selinux enabled Finish(bootstrap): init plugins Start: init plugins INFO: selinux enabled Finish: init plugins INFO: Signal handler active Start: run Start(bootstrap): chroot init INFO: calling preinit hooks INFO: enabled root cache INFO: enabled package manager cache Start(bootstrap): cleaning package manager metadata Finish(bootstrap): cleaning package manager metadata INFO: enabled HW Info plugin Mock Version: 2.4 INFO: Mock Version: 2.4 Finish(bootstrap): chroot init Start: chroot init INFO: calling preinit hooks INFO: enabled root cache INFO: enabled package manager cache Start: cleaning package manager metadata Finish: cleaning package manager metadata INFO: enabled HW Info plugin Mock Version: 2.4 INFO: Mock Version: 2.4 Finish: chroot init INFO: Running in chroot: ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && losetup /dev/loop9 foo.img && mkfs.btrfs /dev/loop9 && mkdir /foo && mount /dev/loop9 /foo && btrfs subvol create /foo/subvol && umount /foo && losetup -d /dev/loop9'] Start: chroot ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && losetup /dev/loop9 foo.img && mkfs.btrfs /dev/loop9 && mkdir /foo && mount /dev/loop9 /foo && btrfs subvol create /foo/subvol && umount /foo && losetup -d /dev/loop9'] 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.575975 s, 1.9 GB/s ERROR: device scan failed on '/dev/loop9': Function not implemented btrfs-progs v5.7 See http://btrfs.wiki.kernel.org for more information. Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication. Label: (null) UUID: 40fa6fa2-b0df-4274-847f-22c5021aa98b Node size: 16384 Sector size: 4096 Filesystem size: 1.00GiB Block group profiles: Data: single 8.00MiB Metadata: single 8.00MiB System: single 4.00MiB SSD detected: yes Incompat features: extref, skinny-metadata Runtime features: Checksum: crc32c Number of devices: 1 Devices: ID SIZE PATH 1 1.00GiB /dev/loop9 ERROR: cannot create subvolume: Function not implemented Create subvolume '/foo/subvol' Finish: chroot ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && losetup /dev/loop9 foo.img && mkfs.btrfs /dev/loop9 && mkdir -p /foo && mount /dev/loop9 /foo && btrfs subvol create /foo/subvol && umount /foo && losetup -d /dev/loop9']
Note the "ERROR: device scan failed on '/dev/loop9': Function not implemented" from the mkfs, and the "ERROR: cannot create subvolume: Function not implemented" from the subvolume creation. Note that the mkfs error repros only when using a loop device, but the subvolume error repros with an image as well (which makes sense, as I'm fairly sure under the hood it's just using a loop device anyway):
$ mock --root fedora-rawhide-armhfp --chroot 'rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && mkfs.btrfs -f foo.img && mkdir -p /foo && mount -o loop foo.img /foo && btrfs subvol create /foo/subvol && umount /foo' INFO: Unable to build arch armv7hl natively on arch x86_64. Setting forcearch to use software emulation. INFO: mock.py version 2.4 starting (python version = 3.9.0)... Start(bootstrap): init plugins INFO: selinux enabled Finish(bootstrap): init plugins Start: init plugins INFO: selinux enabled Finish: init plugins INFO: Signal handler active Start: run Start(bootstrap): chroot init INFO: calling preinit hooks INFO: enabled root cache INFO: enabled package manager cache Start(bootstrap): cleaning package manager metadata Finish(bootstrap): cleaning package manager metadata INFO: enabled HW Info plugin Mock Version: 2.4 INFO: Mock Version: 2.4 Finish(bootstrap): chroot init Start: chroot init INFO: calling preinit hooks INFO: enabled root cache INFO: enabled package manager cache Start: cleaning package manager metadata Finish: cleaning package manager metadata INFO: enabled HW Info plugin Mock Version: 2.4 INFO: Mock Version: 2.4 Finish: chroot init INFO: Running in chroot: ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && mkfs.btrfs -f foo.img && mkdir -p /foo && mount -o loop foo.img /foo && btrfs subvol create /foo/subvol && umount /foo'] Start: chroot ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && mkfs.btrfs -f foo.img && mkdir -p /foo && mount -o loop foo.img /foo && btrfs subvol create /foo/subvol && umount /foo'] 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.622159 s, 1.7 GB/s btrfs-progs v5.7 See http://btrfs.wiki.kernel.org for more information. Label: (null) UUID: ef40f9a8-0037-4179-a072-2fc77180393a Node size: 16384 Sector size: 4096 Filesystem size: 1.00GiB Block group profiles: Data: single 8.00MiB Metadata: DUP 51.19MiB System: DUP 8.00MiB SSD detected: no Incompat features: extref, skinny-metadata Runtime features: Checksum: crc32c Number of devices: 1 Devices: ID SIZE PATH 1 1.00GiB foo.img ERROR: cannot create subvolume: Function not implemented Create subvolume '/foo/subvol' Finish: chroot ['rm -f foo.img && dd if=/dev/zero of=foo.img bs=1G count=1 && mkfs.btrfs -f foo.img && mkdir -p /foo && mount -o loop foo.img /foo && btrfs subvol create /foo/subvol && umount /foo'] $
Metadata Update from @ngompa: - Issue tagged with: Dev, Kernel, Utils
@josef Could you take a look at this? It seems like it would make sense for btrfs-progs to be able to create filesystems and subvolumes even under emulation.
Metadata Update from @ngompa: - Issue assigned to josef
qemu-user requires manual work to add support for any system call that Linux exposes. While very good, it absolutely does not have 100% coverage of all Linux system calls, especially the 1000's of ioctls.
There is a patch series from a few days ago to add support for many btrfs ioctls
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg05594.html
I'd assume that without these ioctls support in qemu-user most btrfs related operations are likely to fail when run in the emulated environment.
When I run the above mock commands, it prints the missing ioctls to stderr.
We first get
Unsupported ioctl: cmd=0x5331 Unsupported ioctl: cmd=0x5331 Unsupported ioctl: cmd=0x5331 Unsupported ioctl: cmd=0x50009404 ERROR: device scan failed on '/dev/loop9': Function not implemented
0x5331 is BTRFS_IOC_GET_FSLABEL, aka FS_IOC_GETFSLABEL
0x50009404 is BTRFS_IOC_SCAN_DEV
Later we get more errors
Unsupported ioctl: cmd=0x5331 Unsupported ioctl: cmd=0x5000940e ERROR: cannot create subvolume: Function not implemented
0x5000940e is BTRFS_IOC_SUBVOL_CREATE
All three of the ioctls seen here are indeed missing. 2 of the 3 missing ioctls are implemented by the patch series I pointed to previously, but FS_IOC_GETFSLABEL is not implemented. More missing ioctls might appear once those first ones are fixed.
@berrange So we just have to wait for this to be fixed in QEMU?
Metadata Update from @ngompa: - Assignee reset
Metadata Update from @ngompa: - Issue untagged with: Kernel
Testing a scratch build, the patches on qemu-devel appear to fix the problems reported above. Once the patches are accepted by the QEMU maintainers, It'd be possible to cherry pick them into the Fedora QEMU binaries for F33/F34 at least. Please file a BZ against QEMU in Fedora if that's desired.
Bug report filed: https://bugzilla.redhat.com/show_bug.cgi?id=1872918
Looks like @berrange proposed an update to fix this: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9199d20f3f
Issue status updated to: Closed (was: Open) Issue close_status updated to: Fixed
Login to comment on this ticket.