I'm not sure what to do here. ;)

buildvm-x86-02.iad2.fedoraproject.org:

lsmod | grep fuse

fuse 208896 1

it's definitely loaded.
Perhaps we need to adjust mock somehow to be able to access/use it in mock chroots?

Ah, if this is a chroot then maybe we need to make sure this is in the /dev of the chroot.

Not sure how to do that. Any ideas?

Hum, found https://pagure.io/fedora-infrastructure/issue/9909

If I run sudo mock --old-chroot --chroot ls /dev, I don't get fuse inside the chroot.

Modifying the mock config as below makes /dev/fuse available:

$ cat /etc/mock/fedora-38-x86_64.cfg 
config_opts['releasever'] = '38'
config_opts['target_arch'] = 'x86_64'
config_opts['legal_host_arches'] = ('x86_64',)

config_opts['plugin_conf']['bind_mount_enable'] = True
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(("/dev/fuse", "/dev/fuse"))

include('templates/fedora-branched.tpl')

Would it be possible to use a different mock config just for rpm-ostree composes? I can not find that in the pungi docs or config so far.

Not without koji knowing about that as a specific type of thing. koji makes the mock config for builds.

We can override that on a site-wide config, but then it applies to every chroot. ;(

ostree composes are done in a 'runroot' job... which is basically 'make a chroot, do these commands in it' type of thing.

runroot however does take a 'mount this dir' list... I wonder if we could just add a /dev/fuse there. I think that might work... but I don't want to push new config to the builders while the mass rebuild is going... ;(

I'm looking at the Pungi/Koji code and maybe I can add the mock config addition there only for this specific case.

Getting Pungi to ask for the mount for all unified-core ostree runroot tasks would be fairly easy. I'm not sure it needs to be configurable.

diff --git a/pungi/phases/ostree.py b/pungi/phases/ostree.py
index 90578ae5..6ab0c8d0 100644
--- a/pungi/phases/ostree.py
+++ b/pungi/phases/ostree.py
@@ -173,6 +173,8 @@ class OSTreeThread(WorkerThread):
         packages = default_packages + additional_packages
         log_file = os.path.join(self.logdir, "runroot.log")
         mounts = [compose.topdir, config["ostree_repo"]]
+        if args["unified-core"]:
+            mounts += ["/dev/fuse"]
         runroot = Runroot(compose, phase="ostree")

         if compose.conf["ostree_use_koji_plugin"]:

But it would be probably good to verify it actually works. Maybe koji runroot --nowait --task-id --new-chroot --use-shell --channel-override=compose --mount=/dev/fuse f39-build x86_64 'ls-l /dev' would be a sufficient test for the mounting.

I think it's not going to work though, because runroot plugin in koji has a configuration safe_roots, which limits what can be mounted and does not include /dev.

ok, so sleeping on it I realized the runroot stuff is only affecting the compose channel and we specifically don't have that in default, so changes there would not affect the mass rebuild. So, let me try and adjust it there.

I think a pungi change would also be good for others tho...

So, I pushed this in ansible f4e8a921a6138c2c6d60ed6c51fdca865b6a8c8e but my test jobs are giving "GenericError: No such directory or mount: /dev/fuse/"

https://koji.fedoraproject.org/koji/taskinfo?taskID=103701550

I guess because it's trying to mount on to /dev/fuse/ as a directory. Do we just need to bind mount /dev/ entirely from the host?

My previous try was using an explicit bind mount, not a regular mount.

Never mind, you specified that in https://pagure.io/fedora-infra/ansible/c/f4e8a921a6138c2c6d60ed6c51fdca865b6a8c8e?branch=main.

If we can bind mount the entire /dev that's fine with me as rpm-ostree will do the installation in an isolate place anyway but that's not great from a "minimal privilege" perspective.

Note: found koji runroot in python3-koji-cli-plugins RPM

Yeah, so that didn't work out, but I think I can make it happen at the mock level after all.

Past me made the site defaults a template and we already have a section that only applies in compose channel builders.

Let me try that.

ok. I had to add:

+config_opts['nspawn_args'] += ['--bind=/dev/fuse']

also, but then I finally got something that worked:

https://koji.fedoraproject.org/koji/taskinfo?taskID=103703562

Shall we see what tomorrow's compose brings now?

Great! Thanks! Let's see if that works tomorrow.

Let's get /dev/fuse exposed by default in Mock then: https://github.com/rpm-software-management/mock/pull/1158

The builds worked! We're not failing in the installer: https://pagure.io/releng/failed-composes/issue/5201

Progress!

Hurray! Good news, I'll take it. ;)

@kevin Could you link here the commits related to the infra changes you've made for reference? We might need them "soon" in other infra.

Nevermind, I found them:

First attempt:
- https://pagure.io/fedora-infra/ansible/c/f4e8a921a6138c2c6d60ed6c51fdca865b6a8c8e?branch=main
- https://pagure.io/fedora-infra/ansible/c/f94a4d1a8fe9c137d9f8f514bfdebc1bd9871d05?branch=main
- https://pagure.io/fedora-infra/ansible/c/8735931424b009a2f8c6c7efbe9d3a63708749e8?branch=main

Working attempt:
- https://pagure.io/fedora-infra/ansible/c/7c10f88b02adbf32202dbca5152cd5bebd54f59b?branch=main
- https://pagure.io/fedora-infra/ansible/c/9e98f4c0defd2387bd7c73f899a4b4671f1bb670?branch=main

Thanks!

FTR, Mock 5.0+ does this automatically, so the manual dev/fuse mountpoint breaks newer Mock.