Learn more about these different git repos.
Other Git URLs
Describe the issue Now that modules can be built for the el8 platform, they need to be included in the bodhi release process.
el8
When do you need this? (YYYY/MM/DD) ASAP
When is this no longer needed or useful? (YYYY/MM/DD) A long time from now.
If we cannot complete your request, what is the impact? EPEL8 module builds are not usable.
@merlinm @kevin
Shell script to add tags (removed the perms on the tag since its only staging and autosign doesn't work properly)
release=epel8 modular_release=${release}-modular stg-koji add-tag --parent "${release}" "${release}-modularity" stg-koji add-tag "${modular_release}" stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}" stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-candidate" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-testing" stg-koji add-tag --parent "${modular_release}-updates-testing" "${modular_release}-updates-testing-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-override" stg-koji add-tag --parent "${modular_release}-updates-testing-pending" "${modular_release}-signing-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-pending" #stg-koji edit-tag --perm=fedora-override "${modular_release}-override" #stg-koji edit-tag --perm=admin "${modular_release}-updates" #stg-koji edit-tag --perm=admin "${modular_release}-updates-testing" #stg-koji edit-tag --perm=autosign "${modular_release}-updates-testing-pending" #stg-koji edit-tag --perm=admin "${modular_release}-updates-pending" #stg-koji edit-tag --perm=autosign "${modular_release}-signing-pending" #stg-koji edit-tag --perm=autosign "${modular_release}" release=epel8-playground modular_release=${release}-modular stg-koji add-tag --parent "${release}" "${release}-modularity" stg-koji add-tag "${modular_release}" stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}" stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates-candidate"
Infra ansible diff for bodhi epel8 composes:
diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2 index d6571fb..1150e73 100644 --- a/roles/bodhi2/backend/templates/pungi.module.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2 @@ -20,14 +20,22 @@ sigkeys = [ '3c3359c4', [% endif %] {% if env == "staging" %} - None + [% elif release.version_int == 8 %] + '2f86d6a1', + [% endif %] {% endif %} ] module_defaults_dir = { 'scm': 'git', 'repo': 'https://pagure.io/releng/fedora-module-defaults.git', - 'branch': 'f[[ release.version_int ]]', + {% if env == "staging" %} + [% elif release.version_int == 8 %] + 'branch': 'epel[[ release.version_int ]]', + [% endif %] + {% else %} + 'branch': 'f[[ release.version_int ]]', + {% endif %} 'dir': '.' }
epel8-playground module composes can be enabled by using https://pagure.io/fork/mohanboddu/pungi-fedora/diff/master..epel8-playground-modules-stg For stg, we can use the one in my fork directly
Updated diff for bodhi:
diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2 index d6571fb..54e4543 100644 --- a/roles/bodhi2/backend/templates/pungi.module.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2 @@ -20,14 +20,22 @@ sigkeys = [ '3c3359c4', [% endif %] {% if env == "staging" %} - None + [% if release.version_int == 8 %] + '2f86d6a1', + [% endif %] {% endif %} ] module_defaults_dir = { 'scm': 'git', 'repo': 'https://pagure.io/releng/fedora-module-defaults.git', - 'branch': 'f[[ release.version_int ]]', + {% if env == "staging" %} + [% if release.version_int == 8 %] + 'branch': 'el[[ release.version_int ]]', + [% endif %] + {% else %} + 'branch': 'f[[ release.version_int ]]', + {% endif %} 'dir': '.' }
Just for reference, https://pagure.io/fork/mohanboddu/pungi-fedora/diff/epel8-playground..epel8-playground-modules-stg is a more appropriate diff to examine.
Just need to figure out how https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n177 and https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n178 works with epel8 tags.
Final script for tags:
release=epel8 modular_release=${release}-modular echo $release echo $modular_release stg-koji add-tag --parent "${release}" "${release}-modularity" stg-koji add-tag "${modular_release}" stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}" stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-candidate" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-testing" stg-koji add-tag --parent "${modular_release}-updates-testing" "${modular_release}-updates-testing-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-override" stg-koji add-tag --parent "${modular_release}-updates-testing-pending" "${modular_release}-signing-pending" stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-pending" stg-koji edit-tag --perm=fedora-override "${modular_release}-override" stg-koji edit-tag --perm=admin "${modular_release}-updates" stg-koji edit-tag --perm=admin "${modular_release}-updates-testing" stg-koji edit-tag --perm=autosign "${modular_release}-updates-testing-pending" stg-koji edit-tag --perm=admin "${modular_release}-updates-pending" stg-koji edit-tag --perm=autosign "${modular_release}-signing-pending" stg-koji edit-tag --perm=autosign "${modular_release}" release=epel8-playground modular_release=${release}-modular echo $release echo $modular_release stg-koji add-tag --parent "${release}" "${release}-modularity" stg-koji add-tag "${modular_release}" stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}" stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates-candidate" stg-koji edit-tag --perm=autosign "${modular_release}"
Diff for infra ansible:
I am not sure what are the keys in staging right now, I can check once my sigul starts working.
diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2 index d6571fb..8307069 100644 --- a/roles/bodhi2/backend/templates/pungi.module.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2 @@ -20,14 +20,24 @@ sigkeys = [ '3c3359c4', [% endif %] {% if env == "staging" %} - None + [% if release.version_int == 8 %] + '2f86d6a1', + [% endif %] {% endif %} ] module_defaults_dir = { 'scm': 'git', 'repo': 'https://pagure.io/releng/fedora-module-defaults.git', - 'branch': 'f[[ release.version_int ]]', + {% if env == "staging" %} + [% if release.version_int == 8 %] + 'branch': 'el[[ release.version_int ]]', + [% else %] + 'branch': 'f[[ release.version_int ]]', + [% endif %] + {% else %} + 'branch': 'f[[ release.version_int ]]', + {% endif %} 'dir': '.' } diff --git a/roles/robosignatory/templates/robosignatory.toml.j2 b/roles/robosignatory/templates/robosignatory.toml.j2 index a06043d..518cd2f 100644 --- a/roles/robosignatory/templates/robosignatory.toml.j2 +++ b/roles/robosignatory/templates/robosignatory.toml.j2 @@ -257,6 +257,21 @@ handlers = ["console"] keyid = "{{ (env == 'production')|ternary('3c3359c4', 'd300e724') }}" type = "modular" + #epel8 modular tags + [[consumer_config.koji_instances.primary.tags]] + from = "epel8-modular-signing-pending" + to = "epel8-modular-updates-testing-pending" + key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}" + keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}" + type = "modular" + + [[consumer_config.koji_instances.primary.tags]] + from = "epel8-playground-modular-updates-candidate" + to = "epel8-playground-modular" + key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}" + keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}" + type = "modular" + # Gated bodhi updates [[consumer_config.koji_instances.primary.tags]]
@mohanboddu,
Based on my interpretation of the logic in get_content_generator_build_koji_tag() at https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/scheduler/handlers/modules.py#_268, the MBS will use KOJI_CG_BUILD_TAG_TEMPLATE (config.py#n177) as the template for the Koji tag--filling in the stream of the base module. Thus, for an el8 base module, it will use tag el8-modular-updates-candidate. If it can't figure out a base module--which AFAIK should never happen, it will fall back to KOJI_CG_DEFAULT_BUILD_TAG (config.py#n178) and use tag modular-updates-candidate.
get_content_generator_build_koji_tag()
KOJI_CG_BUILD_TAG_TEMPLATE
el8-modular-updates-candidate
KOJI_CG_DEFAULT_BUILD_TAG
modular-updates-candidate
I don't think there's currently a way to get an el8 base module to map to tag epel8-modular-updates-candidate while keeping Fedora modules mapping to f##-modular-updates-candidate.
epel8-modular-updates-candidate
f##-modular-updates-candidate
Perhaps @mprahl or @jkaluza can confirm or deny my analysis.
@merlinm Oh, shoot, I looked at it and changed the signing tags for playground to use -updates-candidate instead of -signing-pending, but I didn't realize the epel8 vs el8 problem :sob:
@mohanboddu What is the path forward with this? How are you expecting MBS to tag modules built for el8 and el8_playground? Is there a problem with MBS tagging module builds as el8-modular-updates-candidate rather than epel8-modular-updates-candidate? (Likewise for el8_playground-modular-updates-candidate vs. epel8-playground-modular-updates-candidate) Can't that just be adjusted in roles/robosignatory/templates/robosignatory.toml.j2?
el8_playground
el8_playground-modular-updates-candidate
epel8-playground-modular-updates-candidate
roles/robosignatory/templates/robosignatory.toml.j2
So, I changed the candidate tags to el8-{playground-}updates-candidate.
$ stg-koji edit-tag epel8-playground-modular-updates-candidate --rename el8_playground-modular-updates-candidate $ stg-koji edit-tag epel8-modular-updates-candidate --rename el8-modular-updates-candidate
And to create the bodhi epel 8 modular releases in stage:
bodhi releases create --name EPEL-8M --long-name "Epel 8 Modular" --id-prefix FEDORA-EPEL-MODULAR --version 8 --branch epel8m --dist-tag epel8-modular --stable-tag epel8-modular-updates --testing-tag epel8-modular-updates-testing --candidate-tag el8-modular-updates-candidate --pending-stable-tag epel8-modular-updates-pending --pending-testing-tag epel8-modular-updates-testing-pending --pending-signing-tag epel8-modular-signing-pending --override-tag epel8-modular-override --state current --user mohanboddu --staging
And again, the diff from infra ansible:
diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2 index d6571fb..feeb320 100644 --- a/roles/bodhi2/backend/templates/pungi.module.conf.j2 +++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2 @@ -20,14 +20,25 @@ sigkeys = [ '3c3359c4', [% endif %] {% if env == "staging" %} - None + [% if release.version_int == 8 %] + 'd300e724', + [% endif %] {% endif %} ] module_defaults_dir = { 'scm': 'git', - 'repo': 'https://pagure.io/releng/fedora-module-defaults.git', - 'branch': 'f[[ release.version_int ]]', + {% if env == "staging" %} + 'repo': 'https://pagure.io/modularity/fedora-stg-module-defaults.git', + [% if release.version_int == 8 %] + 'branch': 'el[[ release.version_int ]]', + [% else %] + 'branch': 'f[[ release.version_int ]]', + [% endif %] + {% else %} + 'repo': 'https://pagure.io/releng/fedora-module-defaults.git', + 'branch': 'f[[ release.version_int ]]', + {% endif %} 'dir': '.' } diff --git a/roles/robosignatory/templates/robosignatory.toml.j2 b/roles/robosignatory/templates/robosignatory.toml.j2 index 2df04d3..8126648 100644 --- a/roles/robosignatory/templates/robosignatory.toml.j2 +++ b/roles/robosignatory/templates/robosignatory.toml.j2 @@ -257,6 +257,21 @@ handlers = ["console"] keyid = "{{ (env == 'production')|ternary('3c3359c4', 'd300e724') }}" type = "modular" + #epel8 modular tags + [[consumer_config.koji_instances.primary.tags]] + from = "epel8-modular-signing-pending" + to = "epel8-modular-updates-testing-pending" + key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}" + keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}" + type = "modular" + + [[consumer_config.koji_instances.primary.tags]] + from = "el8_playground-modular-updates-candidate" + to = "epel8-playground-modular" + key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}" + keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}" + type = "modular" + # Gated bodhi updates [[consumer_config.koji_instances.primary.tags]]
@merlinm @kevin One last review and I ready to push the changes.
@mohanboddu The koji tag will need to be el8_playground-modular-updates-candidate (rather than el8-playground-modular-updates-candidate) to exactly match platform:el8_playground. Likewise in roles/robosignatory/templates/robosignatory.toml.j2.
el8-playground-modular-updates-candidate
platform:el8_playground
There's also a separate module defaults repo for staging. module_defaults_dir['repo'] (referenced in roles/bodhi2/backend/templates/pungi.module.conf.j2) should be set to https://pagure.io/modularity/fedora-stg-module-defaults.git in staging.
module_defaults_dir['repo']
roles/bodhi2/backend/templates/pungi.module.conf.j2
https://pagure.io/modularity/fedora-stg-module-defaults.git
As far as I know, the rest looks good. Thanks!
Just updated the comment so that I will everything in one place.
The parts I kinda understand LGTM!
@mohanboddu I just noticed that https://pagure.io/fork/mohanboddu/pungi-fedora/blob/epel8-playground-modules-stg/f/epel8-playground.conf#_17 refers to the production modular defaults repo. Unless there's a straight forward way to create separate vs. production contents in epel8-playground.conf, we should be OK leaving it pointing at the production modular defaults. If it turns out to be a problem, we can create a custom staging configuration later.
epel8-playground.conf
:information_source: The above changes were deployed to staging on 2019-11-15. A modular el8 bodhi update attempt failed upon submit with the error described by https://github.com/fedora-infra/bodhi/issues/3779. Fix https://github.com/fedora-infra/bodhi/pull/3782 was applied and the bodhi update was submitted successfully.
Here is a patch to add the production EPEL 8 virtual platform module definitions to the Infra ansible repo.
Attached <img alt="0001-Adding-production-epel8-platform-module-definitions.patch" src="/releng/issue/raw/files/f89acad74154eb83bbbf1a63eba129932393a1fd5e7fa0d9e3dd4d41ed9038e5-0001-Adding-production-epel8-platform-module-definitions.patch" />
To move this production, in addition to re-implementing everything above, the following is also needed...
@merlinm Is this now complete?
Yes!
And the production release is tracked in https://pagure.io/releng/issue/9091
Metadata Update from @merlinm: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.