#2617 F35 Change: Make btrfs the default file system for Fedora Cloud
Opened 10 days ago by bcotton. Modified 2 days ago

For cloud installs of Fedora, we want to provide advanced file system features to users in a transparent fashion. Thus, we are changing the file system for the Cloud Edition to Btrfs so we can leverage its features and capabilities to improve the quality of experience for Cloud users.


Just worth mentioning, this would mean that Fedora cloud images cannot be mounted on a RHEL host. RHEL does not enable or support BTRFS at all, the module is not built, there would be no way to mount and verify a Fedora cloud image on RHEL or CentOS without actually booting it. If your image is unbootable, for whatever reason, you may have to find a different host to mount and try to fix it on.

Irrespectively of this change, and whether btrfs is the default or not in other places, not being able to access a very popular file system at all seems very user-unfriendly. Not supporting it for root file system and normal storage on RHEL systems is one thing and completely reasonable, but completely preventing users from accessing a certain file system type is quite a big step. So maybe we could convince the RHEL maintainers to issue the module in an update.

I have to agree with @zbyszek here: being completely unable to even access a btrfs filesystem is imho not a great user experience and shouldn't be a reason why the Fedora Cloud images shouldn't switch. It is rather a reason why at least read support should be added to RHEL.

RHEL makes its decisions based on a variety of factors, even if they were to change this for RHEL 8 (seems unlikely) or RHEL 9, it could take a very long time before this appears in production environments. We have absolutely no control over what they decide to build and support. For workstation and such, this doesn't matter so much. For cloud, we offer a downloadable raw image that is likely to be used on RHEL hosts as much or more than on Fedora. It is certainly worth taking into consideration.

Metadata Update from @ngompa:
- Issue tagged with: meeting

5 days ago

I don't think this is a chicken and egg problem. Fedora's mandate is to lead, not second guess itself as if it's a favor to Red Hat. Guessing is folly. Fedora needs to do what's in the best interest of the community given all available facts. And as we don't have many (or even most) of the decision making facts that go into RHEL, I choose to consider it out of scope. As in, it expressly should not be taken into consideration.

Red Hat can do anything they want. If they care about this use case they can at least provide a libguestfs appliance that supports btrfs, so that guestfish can inspect btrfs images.

So, I was pretty "meh" about this change... seemed fine, but wasn't really a big gain for cloud uses.

Thinking about it more however, would this result in our images no longer being able to be automatically resized? Or snapshotted outside the OS?

ie, https://docs.openstack.org/image-guide/openstack-images.html for openstack has no ability to auto resize btrfs (at least that it says there)

How does aws handle this?

cloud-init is capable of resizing btrfs (it works with openSUSE based images already), so I imagine AWS would use that same path anyway.

So, I was pretty "meh" about this change... seemed fine, but wasn't really a big gain for cloud uses.

Thinking about it more however, would this result in our images no longer being able to be automatically resized? Or snapshotted outside the OS?

No, none of this changes with Btrfs.

ie, https://docs.openstack.org/image-guide/openstack-images.html for openstack has no ability to auto resize btrfs (at least that it says there)

I'm pretty sure OpenStack's documentation is out of date here, since it references cloud-init stuff.

We discussed this during today's fesco meeting:
AGREED: We'll wait a week for more discussion and information about userspace access options.
ACTION: Eighth_Doctor, cmurf, dcavalca, michel will provide some more information about userspace access options

Just for more information (I was in another meeting during the meeting, so couldn't comment as much). Yes, libguestfs does include a kernel for the appliance, but it is a buildreq, it is not building its own, meaning the RHEL libguestfs still does not support btrfs. From the spec:

%if !0%{?rhel}
BuildRequires: btrfs-progs
%endif

From a kernel standpoint, I don't care too much one way or the other for most spins. Cloud images, you do have to take into consideration the host though. Even if RHEL were to turn on btrfs in RHEL 9 for some reason (I don't see any indication that this is being considered, it is still off in ARK), it would literally take years for that to roll out to the majority of users. I would guess there is considerably more RHEL 7 out there than RHEL 8 at this point. I don't know when RHEL 9 is planning to release, but It isn't in the F35 time frame for sure. This is one area that it certainly will not be Fedora leads RHEL, because there is a lot more at play there., and there are plenty of features enabled in Fedora that RHEL is happy to keep turned off for a variety of reasons. As far as I am currently aware, btrfs is one of those features. Basically the decision needs to be made assuming nothing is going to change in RHEL.

So what do we give up by having no btrfs support on the host? The images should still boot, and function as normal. You do lose the ability to debug or modify the image from a RHEL host, and basically anything with the image that does not involve booting the included kernel. No read or write access. You can't try to use the image as a container image. You limit some forms of offline backup, though some others still work. I can not comment on the Openstack compatibility, because I haven't been much of a user of that, it would need someone else to verify.

Basically the decision needs to be made assuming nothing is going to change in RHEL.

Ack.

A possible read only solution is grub2-mount: as grub can read btrfs file systems, you can use its implementation to mount a btrfs based cloud image. Unfortunately you won't be able to modify it from that.

A possible read only solution is grub2-mount: as grub can read btrfs file systems, you can use its implementation to mount a btrfs based cloud image. Unfortunately you won't be able to modify it from that.

This should even be possible from RHEL, since GRUB there has support for Btrfs.

grub2-mount is explicitly disabled in CentOS Stream 8. I've filed https://bugzilla.redhat.com/show_bug.cgi?id=1970538 to get it added and put up https://git.centos.org/rpms/grub2/pull-request/1 with the necessary changes. CentOS Stream 9 already includes it, so we should be good there.

Aslo, depending on the grub version zstd will be unsupported (see: https://btrfs.wiki.kernel.org/index.php/FAQ#Does_grub_support_Btrfs.3F)

Login to comment on this ticket.

Metadata