I don't know how it was done before.
CC @dustymabe @walters @jlebon

I think we really need to have some SOP documentation for ostree composes in Fedora specially what steps are required at the time of branching/creating new refs.

I see there is one SOP https://docs.pagure.org/releng/sop_composing_fedora.html but it tells only about Beta and RC compose process.

I suspect the logic/code has changed here since we last branched. Probably the simplest indeed is to add a SOP to copy the rawhide commit to the branched.

Hmm, we could add an option like ostree pull --allow-not-found that would make it not an error if the target ref doesn't exist; that seems like the desired semantic here.

Although I'm a bit uncertain because it looks like the failure there is building an installer image, which shouldn't happen until the main ostree commit is built?

Let's focus on the most recent compose. The OSTree installer task (the type of the one you linked to) won't succeed if the OSTree task didn't succeed. Arguably we shouldn't run the later tasks if one they depend on doesn't succeed.

It looks like it is failing parsing the input yaml files:

?[0m?[31merror: ?[0mParsing /mnt/koji/compose/branched/Fedora-37-20220815.n.0/work/ostree-4/config_repo/fedora-common-ostree.yaml: serde-yaml: while parsing a block mapping, did not find expected key at line 122 column 1
Traceback (most recent call last):
  File "/usr/bin/pungi-make-ostree", line 33, in <module>
    sys.exit(load_entry_point('pungi==4.3.5', 'console_scripts', 'pungi-make-ostree')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pungi/ostree/__init__.py", line 159, in main
    func()
  File "/usr/lib/python3.11/site-packages/pungi/ostree/tree.py", line 151, in run
    self._make_tree()
  File "/usr/lib/python3.11/site-packages/pungi/ostree/tree.py", line 59, in _make_tree
    shortcuts.run(
  File "/usr/lib/python3.11/site-packages/kobo/shortcuts.py", line 409, in run
    raise exc
RuntimeError: ERROR running command: rpm-ostree compose tree --repo=/mnt/koji/compose/ostree/repo/ --write-commitid-to=/mnt/koji/compose/branched/Fedora-37-20220815.n.0/logs/x86_64/Silverblue/ostree-4/commitid.log --touch-if-changed=/mnt/koji/compose/branched/Fedora-37-20220815.n.0/logs/x86_64/Silverblue/ostree-4/commitid.log.stamp --add-metadata-string=version=37.20220815.n.0 --force-nocache /mnt/koji/compose/branched/Fedora-37-20220815.n.0/work/ostree-4/config_repo/fedora-silverblue.json
For more details see /mnt/koji/compose/branched/Fedora-37-20220815.n.0/logs/x86_64/Silverblue/ostree-4/create-ostree-repo.log

Somebody will need to look at the configs and make sure they don't have a syntax error.

I suspect this is fallout from https://pagure.io/workstation-ostree-config/c/b87ec3ccd0ccc59982d1f131dc8bc15c2bcfe4d8?branch=f37

Looks like it's failing on line 122 in https://kojipkgs.fedoraproject.org/compose/branched/Fedora-37-20220815.n.0/work/ostree-4/config_repo/fedora-common-ostree.yaml and that's part of commit b87ec3ccd0ccc59982d1f131dc8bc15c2bcfe4d8

@siosm Can you try to revert this commit so we can see if it fixes the compose?

The OSTree installer task (the type of the one you linked to) won't succeed if the OSTree task didn't succeed.

@dustymabe well, not really. What usually happens is we get an ostree installer image, but it deploys the last successful ostree compose (and openQA catches this in its os-release test). It's only outright failing for F37 because we've never had a successful ostree compose for F37.

On Rawhide composes, we're geting ostree installer images, but they're deploying old ostrees from before this broke.

The OSTree installer task (the type of the one you linked to) won't succeed if the OSTree task didn't succeed.

@dustymabe well, not really. What usually happens is we get an ostree installer image, but it deploys the last successful ostree compose (and openQA catches this in its os-release test). It's only outright failing for F37 because we've never had a successful ostree compose for F37.

Good to know. Even more reason to try to cut it so that the Installer task fails if the creation of the OSTree fails for that compose.

On Rawhide composes, we're geting ostree installer images, but they're deploying old ostrees from before this broke.

Yeah, not great.

I reverted the commit in both rawhide and f37 branch. There's a rawhide compose just starting now...

Thanks for the investigation and sorry for causing troubles.

Thanks all for this ticket and discussion. I can see now F37 Silverblue iso generated :)

Only ppc64le failed but its different issue that firefox build disabled on ppc64le.

Let's close this then :) Thanks, kevin!