#1145 Silverblue/Kinoite/Sericea/Onyx: Use unified core mode
Merged 10 months ago by kevin. Opened a year ago by siosm.
siosm/pungi-fedora rpm-ostree-unified-core  into  main

file modified
@@ -752,6 +752,7 @@ 

      "^Silverblue$": {


          "force_new_commit": True,

+         "unified_core": True,

          "treefile": "fedora-silverblue.yaml",

          "config_url": "https://pagure.io/workstation-ostree-config.git",

          "config_branch": "main",
@@ -765,6 +766,7 @@ 

      "^Kinoite$": {


          "force_new_commit": True,

+         "unified_core": True,

          "treefile": "fedora-kinoite.yaml",

          "config_url": "https://pagure.io/workstation-ostree-config.git",

          "config_branch": "main",
@@ -778,6 +780,7 @@ 

      "^Sericea$": {


          "force_new_commit": True,

+         "unified_core": True,

          "treefile": "fedora-sericea.yaml",

          "config_url": "https://pagure.io/workstation-ostree-config.git",

          "config_branch": "main",
@@ -791,6 +794,7 @@ 

      "^Onyx$": {


          "force_new_commit": True,

+         "unified_core": True,

          "treefile": "fedora-onyx.yaml",

          "config_url": "https://pagure.io/workstation-ostree-config.git",

          "config_branch": "main",

rpm-ostree is moving to unified core composes and this is now working for Silverblue & Kinoite.

Requires: https://pagure.io/pungi/pull-request/1626
Requires: https://pagure.io/workstation-ostree-config/pull-request/246
See: https://github.com/coreos/rpm-ostree/issues/729
See: https://fedoraproject.org/wiki/Changes/FedoraSilverblueUnifiedCore
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2150984

Revert "Revert "Use unified core mode for Silverblue & Kinoite""

See: https://pagure.io/pungi-fedora/pull-request/1115
See: https://pagure.io/pungi-fedora/pull-request/1136

This reverts commit dd54ad9.

Signed-off-by: Timothée Ravier tim@siosm.fr

Should be good to merge to give it a try again.
CC @adamwill

Should we switch Sericea as well? AFAIU, gitlab builds were running in unfied core mode and everything has been working so far.

@alebastr Oh, yes, we should. Let me do that.

rebased onto 231a6bc7ad9b979b7fe2ec11f0ab39a1455f302d

a year ago

Can we please get a Rawhide and F38 compose done before merging this? We haven't had a compose for several days and it'd suck if this doesn't work (again) and we're stuck with old broken bits in Silverblue/Kinoite.

I can test this in openQA, but things are kind of a mess there right now (the test is failing due to the f39 toolkit container issue, there's a bunch of other failures that make it harder to see what's going on, and a billion tests have started recently because the s390x backlog cleared) so it's a bit difficult.

I fixed one of the issue with the F38 compose yesterday: https://pagure.io/fedora-infra/ansible/pull-request/1318

I'm looking at the rpm-ostree issue now.

pretty please pagure-ci rebuild

a year ago

rebased onto b59465518b07ff858492d0b8511db7b8ef016c82

a year ago

we got rawhide and branched composes through now, and I think I should be in a position to do a test run with openQA today after I clean up a bunch of other messes. I'll try and do that and report back in a bit.

Ugh. Forget the deleted comments. I just checked and the test didn't actually run with --unified-core at all due to a mistake on my end. Trying again.

F38 PR is in https://pagure.io/pungi-fedora/pull-request/1147. Both branches should be ready and working. You may merge this when you think it's ready for openQA testing.

openQA testing doesn't require it to be merged as we essentially parallel the process in openQA - I just edit the command to add the parameter. It just needed everything else to calm down so the tests weren't failing on other things! I've just edited the test on the staging instance and started an F38 test - if that one and its deps pass, we're probably good.

Unfortunately the overlay test failed. It's not failing on non-unified builds. I'll try and take a proper look into this on Tuesday, at the latest (monday is a holiday here).

OK, so I reproduced this twice, pretty sure it's accurate. When I build a Silverblue ostree+installer with --unified-core, it works fine, it can be installed fine...but when we run the overlay test on it, it fails.

The overlay test boots, switches to a VT, logs in as root, does systemctl set-default multi-user.target, does rpm-ostree install htop and reboots. At that point it fails: the system doesn't reach a tty on VT1 as it should, it just sticks at this screen:


hitting ctrl-alt-f6 does get a tty on VT6. I suspect the problem is that changing the boot target doesn't work right, rather than actually the overlay going wrong.

This only happens when building with --unified-core. Same test, on the same update, without that, passes fine.

I vaguely remember some user complaining that GDM was not being disabled. Maybe you should also explicitly disable the gdm.service unit when switching to multi-user.target?

well, we don't have to do that on conventional RPM systems, or on non unified core ostree installs, and it would rather make the 'target' mechanism pointless if you have to go around doing all the things it's supposed to do for it?

I don't know why this is happening. What I wanted to say is that this is unlikely the main use case for Silverblue (using multi-user.target) and should not have something to do with testing package layering thus the idea of working around it in the test for now.

oh, I see. well, yes, we could try that as a workaround to see if the test passes, but it's a bit worrying that unified core breaks this; if we don't know why, we don't know how wide the impact is and whether it might break something else that's more likely to be important.

OK, re-reading https://github.com/fedora-silverblue/issue-tracker/issues/401, it looks like disabling GDM would not be enough :/

This will be for F39

rebased onto 9da0c7774fde141e37e68696d8495750436a034a

a year ago

So I flipped this back on in openQA (which has its own ostree build code, so it's a handy place to test), and it looks like there's a bug ATM:


two errors about "Unexpected non-root owned path" show up there.

/usr/etc/cups/subscriptions.conf is the offending file.

I think this is https://github.com/coreos/rpm-ostree/issues/4437 - fixed in latest rpm-ostree upstream, but I don't think it is in a release yet.

https://bodhi.fedoraproject.org/updates/FEDORA-2023-7a8817d80b > The latest rpm-ostree with the fix should now be in rawhide. Can you give the openQA tests another go? Thanks

@adamwill Can you manually trigger a new openQA test for this change?

OK, I've found the issue with the tty not showing up anymore with unified core. Unified core builds do not include the following:

[silverblue@fedora deploy]$ ll /usr/etc/systemd/system/getty.target.wants/getty@tty1.service 
lrwxrwxrwx. 2 root root 38 Jun 27 08:30 /usr/etc/systemd/system/getty.target.wants/getty@tty1.service -> /usr/lib/systemd/system/getty@.service

More verbose diff (not exhaustive as files in directories as skipped):

D    /usr/etc/systemd/system/ctrl-alt-del.target
D    /usr/etc/systemd/system/dbus-org.freedesktop.oom1.service
D    /usr/etc/systemd/system/getty.target.wants
D    /usr/etc/systemd/system/multi-user.target.wants/nfs-client.target
D    /usr/etc/systemd/system/multi-user.target.wants/remote-fs.target
D    /usr/etc/systemd/system/multi-user.target.wants/systemd-oomd.service
D    /usr/etc/systemd/system/remote-fs.target.wants
D    /usr/etc/systemd/system/sockets.target.wants/systemd-journald-audit.socket
D    /usr/etc/systemd/system/sockets.target.wants/systemd-userdbd.socket
D    /usr/etc/systemd/system/sysinit.target.wants/systemd-network-generator.service
D    /usr/etc/systemd/system/systemd-journald.service.wants
D    /usr/etc/systemd/system/timers.target.wants/fstrim.timer
D    /usr/etc/systemd/user/basic.target.wants
D    /usr/etc/systemd/user/timers.target.wants

Will see if https://pagure.io/workstation-ostree-config/pull-request/246 fixes this issue.

Do you still want the openQA test run now, or should I wait for the above PR?

I'll push a fix and then you can try again. I'll ping again once I've done that. Thanks

rebased onto b73fec0db5b76f0b183502523cf7b71fca5a4724

11 months ago

I've merged https://pagure.io/workstation-ostree-config/pull-request/246 which should fix the issue surfaced in openQA when disabling GDM.

@adamwill Can you try a new openQA test with this? Thanks

Well, that one timed out, not sure if it's related to the change or just because we wound up scheduling a whole pile of jobs in stg at once for...reasons. I'm trying again.

Hmm. So on a retry the ISO build succeeded, as did the install test, and the rebase test...but the overlay test showed an error when trying to overlay postgresql and run postgresql-setup:


not totally sure if that's caused by the unified core stuff or not.

A few subsequent runs have passed, so maybe this is fine? I'll watch it for a bit longer.

Thanks for running the test!

rebased onto 7a60ec077bae1b14b3e696aaf2ed278cad9736fb

10 months ago

@adamwill Let's merge this to get more testing?

I'd rather not merge it if the postgresql test would fail on every run, but it seems like it only failed once and has succeeded on subsequent runs, so...sure, I guess.

One option is to use another package for now for testing. It's unlikely that users will overlay this specific package. If we use a "less likely to fail" package (one that does not create a user) for the test then we can move forward while we fix the underlying issue.

https://src.fedoraproject.org/rpms/postgresql/pull-request/60 has been merged.

I don't have merge powers here. Can someone who has merge this? Thanks!

rebased onto 6d82501

10 months ago

Pull-Request has been merged by kevin

10 months ago