cc: @cverna
Sometime in the last day or so, Flatpak builds in prod started failing because flatpak_base_image is not set in the reactor config map. For example:
https://koji.fedoraproject.org/koji/taskinfo?taskID=33159676
This has been needed since:
https://github.com/projectatomic/osbs-client/pull/808
So I'm guessing that osbs-client got upgraded. The reactor configs for prod and staging need:
flatpak: base_image: <appropriate value>
where the appropriate values are those used for flatpak_base_image in playbooks/groups/buildvm.yml. The references to flatpak_base_image in buildvm.yml and roles/osbs-client/templates/osbs.conf.j2 can then be removed.
playbooks/groups/buildvm.yml
buildvm.yml
roles/osbs-client/templates/osbs.conf.j2
Metadata Update from @cverna: - Issue assigned to cverna
Metadata Update from @cverna: - Issue priority set to: None (was: Needs Review)
I have pushed https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=43bbc530070b7ec7ebfac386779e208fbf0dc8b9
and run the following playbooks :
sudo rbac-playbook groups/buildvm.yml -l buildvm-stg -t osbs sudo rbac-playbook groups/buildvm.yml -l buildvm -t osbs sudo rbac-playbook groups/osbs-cluster.yml -l osbs-masters-stg -t osbs-orchestrator-namespace sudo rbac-playbook groups/osbs-cluster.yml -l osbs-masters -t osbs-orchestrator-namespace
@otaylor you should be able to give it another try.
Thanks
Now it fails with "Image build failed. Error in plugin bump_release: AttributeError('Dockerfile has not yet been generated'). OSBS build id: gnome-sound-recorder-master-a8e9b-2 Output orchestrator.log (tail) osbs-client.log (tail)"
See https://koji.fedoraproject.org/koji/taskinfo?taskID=33178989
The problem is in the build.json that was logged - I have no idea right now how osbs-client could create a build.json with resolve_module_compose but without flatpak_create_dockerfile. I'll look more a bit later. @cverna: What is the exact version of osbs-client we have deployed on the builders now?
{ "source": { "provider": "git", "uri": "https://src.fedoraproject.org/modules/gnome-sound-recorder.git", "provider_params": { "git_commit": "ded784a3483d6c3fbf3e8fc4b5e7f6ef2f9d9864" } }, "image": "feborges/gnome-sound-recorder:f29-flatpak-candidate-23928-20190305084151", "openshift_build_selflink": "/apis/build.openshift.io/v1/namespaces/osbs-fedora/builds/gnome-sound-recorder-master-a8e9b-2", "prebuild_plugins": [ { "name": "reactor_config" }, { "name": "check_and_set_rebuild", "args": { "label_key": "is_autorebuild", "label_value": "true" } }, { "name": "check_and_set_platforms", "required": false, "args": { "koji_target": "f29-flatpak-candidate" } }, { "name": "resolve_module_compose" }, { "name": "pull_base_image", "args": { "check_platforms": true } }, { "name": "bump_release", "args": { "append": true } }, { "name": "add_labels_in_dockerfile" }, { "name": "koji_parent" } ], "buildstep_plugins": [ { "name": "orchestrate_build", "args": { "platforms": [ "x86_64", "ppc64le", "aarch64", "i686", "s390x", "armv7hl" ], "build_kwargs": { "component": "gnome-sound-recorder", "git_branch": "master", "git_ref": "ded784a3483d6c3fbf3e8fc4b5e7f6ef2f9d9864", "git_uri": "https://src.fedoraproject.org/modules/gnome-sound-recorder.git", "koji_task_id": 33178989, "user": "feborges", "arrangement_version": 6, "reactor_config_map": "reactor-config-map", "target": "f29-flatpak-candidate", "flatpak": true }, "config_kwargs": { "build_image": "buildroot:latest" } } } ], "postbuild_plugins": [ { "name": "fetch_worker_metadata" }, { "name": "compare_components" }, { "name": "tag_from_config", "args": { "tag_suffixes": { "unique": [ "f29-flatpak-candidate-23928-20190305084151" ], "primary": [ "latest", "{version}", "{version}-{release}" ] } } }, { "name": "group_manifests" }, { "name": "pulp_tag" }, { "name": "pulp_sync", "args": { "publish": false } } ], vpxc "prepublish_plugins": [], "exit_plugins": [ { "name": "pulp_publish" }, { "name": "pulp_pull", "args": { "insecure": true } }, { "name": "delete_from_registry" }, { "name": "koji_import" }, { "name": "koji_tag_build", "args": { "target": "f29-flatpak-candidate" } }, { "name": "store_metadata_in_osv3" }, { "name": "sendmail" }, { "name": "remove_built_image" }, { "name": "remove_worker_metadata" } ] }
The version is osbs-client-0.52-1.fc29.noarch we have osbs-client 0.49 in the buildroot so I ll update the buildroot so we have the same version and see if that helps
@otaylor we now osbs-client osbs-client-0.52-1.fc29.noarch in the buildroot if you want to give it another try.
This turned out to be the problem - after an hour of debugging, I remembered that the build.json now gets regenerated when the build starts, so the osbs-client version in the buildroot was what was causing the problem. (The old version still expected flatpak_base_image in osbs.conf, and removed the flatpak_create_dockerfile plugin when it wasn't found.)
https://koji.fedoraproject.org/koji/taskinfo?taskID=33189799 is now running and has gotten past this problem. Hopefully it won't hit any more!
x86_64 succeeded, this can be closed as fixed, aarch64 is not working again.
Metadata Update from @otaylor: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.