Learn more about these different git repos.
Other Git URLs
The last successful Fedora Budgie Atomic compose is from 41.20241114.0.
When looking at the logs, they are cut in weird places: - https://kojipkgs.fedoraproject.org/compose/updates/Fedora-41-updates-20241116.0/logs/x86_64/Onyx/ostree-4/ - https://kojipkgs.fedoraproject.org/compose/updates/Fedora-41-updates-20241116.0/logs/x86_64/Onyx/ostree-4/create-ostree-repo.log - https://kojipkgs.fedoraproject.org/compose/updates/Fedora-41-updates-20241116.0/logs/x86_64/Onyx/ostree-4/runroot.log
And the failure is weird:
FAILED: BuildrootError: rm exited with status 1
For comparison, the unofficial builds have been successful so far, pointing to an issue specific to the Fedora infra: https://gitlab.com/fedora/ostree/ci-test/-/pipelines
No specific deadline. As soon as possible.
N/A
Fedora Budgie Atomic 41 users are not receiving updates.
I see:
DEBUG util.py:463: systemd-udev.post: Created symlink '/etc/systemd/system/multi-user.target.wants/systemd-homed.service' → '/usr/lib/systemd/system/systemd-homed.service'. DEBUG util.py:463: systemd-udev.post: Created symlink '/etc/systemd/system/systemd-homed.service.wants/systemd-homed-activate.service' → '/usr/lib/systemd/system/systemd-homed-activate.service'. DEBUG util.py:463: systemd-udev.post: CreateTraceback (most recent call last): DEBUG util.py:463: File "/usr/bin/pungi-make-ostree", line 33, in <module> DEBUG util.py:463: sys.exit(load_entry_point('pungi==4.7.0', 'console_scripts', 'pungi-make-ostree')()) DEBUG util.py:463: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ DEBUG util.py:463: File "/usr/lib/python3.13/site-packages/pungi/ostree/__init__.py", line 202, in main DEBUG util.py:463: func() DEBUG util.py:463: ~~~~^^ DEBUG util.py:463: File "/usr/lib/python3.13/site-packages/pungi/ostree/tree.py", line 155, in run DEBUG util.py:463: self._make_tree() DEBUG util.py:463: ~~~~~~~~~~~~~~~^^ DEBUG util.py:463: File "/usr/lib/python3.13/site-packages/pungi/ostree/tree.py", line 62, in _make_tree DEBUG util.py:463: shortcuts.run( DEBUG util.py:463: ~~~~~~~~~~~~~^ DEBUG util.py:463: cmd, DEBUG util.py:463: ^^^^ DEBUG util.py:463: ...<3 lines>... DEBUG util.py:463: universal_newlines=True, DEBUG util.py:463: ^^^^^^^^^^^^^^^^^^^^^^^^ DEBUG util.py:463: ) DEBUG util.py:463: ^ DEBUG util.py:463: File "/usr/lib/python3.13/site-packages/kobo/shortcuts.py", line 347, in run DEBUG util.py:463: lines = proc.stdout.read(buffer_size) DEBUG util.py:463: File "<frozen codecs>", line 325, in decode DEBUG util.py:463: UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 0-1: invalid continuation byte DEBUG util.py:610: Child return code was: 1
Ah, I've seen this one a few times now. I'll file a pungi issue.
Filed: https://pagure.io/pungi/issue/1804
Metadata Update from @phsmoura: - Issue tagged with: investigation, medium-gain, medium-trouble, ops
Commit 7d8f3b4b fixes this issue
Metadata Update from @siosm: - Issue status updated to: Open (was: Closed)
I don't think this is fixed. I can not access the commit ID linked above.
See: - https://pagure.io/releng/failed-composes/issue/7701 - https://kojipkgs.fedoraproject.org//work/tasks/6081/127816081/runroot.log
It's probably something in that package set emitting non-UTF8 content in its %post.
But pungi really needs to learn to deal with this by probably using utf8 replacement in its subprocess handling.
So I poked around at this and came up with https://pagure.io/pungi/pull-request/1812 . wdyt ?
Ah, for the record, I see the fix @hlin tried - https://pagure.io/pungi/c/7d8f3b4b9b2cf65967b4d3f8dd249aec2e3cbbf8?branch=master - but I can also see why it didn't work. It effectively bypasses kobo and calls subprocess.Popen directly, and handles subprocess.Popen returning a non-zero exit code, but that's just not the failure path here. The failure path is that subprocess.Popen raises a UnicodeDecodeError.
We could catch that exception and move on, but I don't like that because it means we drop rather abruptly out of the process, and we will lose a lot of output. Using subprocess' ability to replace invalid characters seems much better.
OK, I think my fix is good, Onyx didn't fail in the latest Rawhide. I'll install the same pungi build on the stable compose host.
Huh. It failed again in today's updates compose, but clearly wasn't using the new pungi. Maybe I updated the wrong compose host. I'll check tomorrow...
duh, of course, updates composes run on bodhi:
2025-01-13 00:15:52 [INFO ] Host: bodhi-backend01.iad2.fedoraproject.org 2025-01-13 00:15:52 [INFO ] Pungi version: pungi-4.7.0-8.fc41.noarch
just need to update that host...I'll do that.
d'oh, and it still doesn't work, but I see why not. we're using something from pungi really within the koji task itself, this isn't the pungi on the compose host. The 'outer' pungi is 4.8.0-3.fc41 with the patch, but the 'inner' pungi is 4.8.0-1.fc41 without it:
DEBUG package_manager.py:295: ['/usr/bin/dnf5', '--installroot', '/var/lib/mock/f41-build-56577093-6544438/root/', 'install', 'pungi', 'ostree', 'rpm-ostree', 'selinux-policy-targeted', '--setopt=deltarpm=False', '--setopt=allow_vendor_change=yes', '--allowerasing'] ... DEBUG util.py:461: Installing: DEBUG util.py:461: ostree x86_64 2025.1-1.fc41 build 621.4 KiB DEBUG util.py:461: pungi noarch 4.8.0-1.fc41 build 313.0 KiB
I don't think there's an easy way to get it to use a newer build for the 'inner' pungi besides getting the update pushed stable, so if people could karma it that'd be great - https://bodhi.fedoraproject.org/updates/FEDORA-2025-9fded5cd72
still needs one more karma or this won't be fixed for several days.
Huh. Weirdly, we got an Onyx in the most recent updates compose, but it didn't have the Pungi fix yet. Possibly someone fixed whatever package scriptlet is generating non-unicode output? But hey, it's good news.
alright. Should we close this then? Or wait a bit more?
This is resolved as far as I can see. Thanks for the fixes!
Metadata Update from @siosm: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.