#7870 RFR: Deploying coreos-koji-tagger in Fedora Openshift
Closed: Fixed a month ago by dustymabe. Opened 3 months ago by dustymabe.

  • Software: coreos-koji-tagger
  • Advantage for Fedora: Aid in development of Fedora CoreOS
  • Sponsor: needed

I would like to request to deploy coreos-koji-tagger in Fedora Openshift.

The point of this service is to monitor input files in git repositories and tag packages into the coreos-pool tag when necessary. Currently the source code lives in https://github.com/coreos/fedora-coreos-koji-tagger. It will be moved to a more permanent location before deploying to production openshift.

More Info:
- https://github.com/coreos/fedora-coreos-tracker/issues/179
- https://github.com/coreos/fedora-coreos-tracker/issues/188

for future reference for me: https://fedora-infra-docs.readthedocs.io/en/latest/sysadmin-guide/sops/requestforresources.html#ticket-comment-template


@mizdebsk - are you willing to be a sponsor for me for this initiative ?

I'll sponsor this RFR. We can move to planning phase.
@dustymabe Please start discussion on infra list, if not already done.

Metadata Update from @mizdebsk:
- Issue assigned to mizdebsk
- Issue priority set to: Waiting on Reporter (was: Needs Review)

3 months ago

I'll sponsor this RFR. We can move to planning phase.

Thanks!

@dustymabe Please start discussion on infra list, if not already done.

done: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/MO2RF632PEXHLWCVQFYZGKFRKBVS7Q7B/

@mizdebsk - looks like there was no negative discussion on mailing list. What would you like me to do next?

RFR SOP has been updated in the meantime and it looks like there are extra steps required before staging instance can be created. I will familiarize myself with updated SOP and let you know.

@dustymabe The only remaining step before creating staging instance is determining who is going to be involved in maintenance of the app.

Metadata Update from @mizdebsk:
- Issue tagged with: request-for-resources

3 months ago

@dustymabe The only remaining step before creating staging instance is determining who is going to be involved in maintenance of the app.

@mizdebsk - can we name me as well as anyone in the sysadmin-coreos group.

ACK. I hereby confirm that the resource is ready to move to the next phase - creating staging instance. @dustymabe Please commit Ansible roles and playbooks to ansible.git and let me know once done. I will grant you permission to run the playbook.

Thanks @mizdebsk. Sorry it took me a few days to get these set up. The files I'm proposing for ansible are in https://github.com/dustymabe/fedora-infra-ansible/pull/1. I have a few questions/comments:

  • Previously @kevin created a keytab for us that is associated with the coreos-bot FAS user for manipulating these tags using bots.
  • From other apps (like the bodhi one) I see that those actually call out to the openshift-keytab which calls out to the keytab/service role.
  • Which way should I go? Should we embed our keytab for coreos-bot into the ansible private vars and populate it that way or should we call openshift-keytab?
  • If we should call openshift-keytab can you help me set that up?

I was talking with patrick in IRC yesterday about compose-tracker and security audit and coreos-koji-tagger came up as well. Here are a few generic questions he asked and my answers:

@puiterwijk
1. what way is it deployed (i.e. does it just "pip install" on all its deps, or does it install deps from Fedora and then just install the code itself from git)
2. what access does it have
3. what is the worst that could happen, etc

My answers:

  1. all deps are from fedora + github source code for python script
  2. access to koji coreos-pool tag via keytab
  3. if someone got the keytab they could much with our coreos-pool tag - not ideal, but can be recovered from easily

@mizdebsk - can we make some progress on this this week?

We should use a different keytab, created by one of existing Ansible roles. The keytab can be linked to the existing coreos-bot user if desired.

ok - the code in that PR was pushed to ansible repo. Can we add permissions to run them and set it up in staging?

Permission granted for sysadmin-coreos members to run the playbook.

hey @mizdebsk - i'm seeing an issue using the keytab you provided:

2019-06-27 20:29:44,364 ERROR coreos_koji_tagger - COMMAND: ['/usr/bin/kinit', '-k', '-t', '/etc/coreos-koji-tagger-keytab/koji-keytab', 'coreos-koji-tagger/coreos-koji-tagger.stg.phx2.fedoraproject.org@STG.FEDORAPROJECT.ORG']
2019-06-27 20:29:44,364 ERROR coreos_koji_tagger -  STDOUT: None
2019-06-27 20:29:44,364 ERROR coreos_koji_tagger -  STDERR: None
kinit: Keytab contains no suitable keys for coreos-koji-tagger/coreos-koji-tagger.stg.phx2.fedoraproject.org@STG.FEDORAPROJECT.ORG while getting initial credentials

looks like it should have been: coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG

ok - now the username is right but I don't can't talk to koji:

2019-06-27 21:36:16,343 INFO coreos_koji_tagger - Authenticating with keytab: /etc/coreos-koji-tagger-keytab/koji-keytab
2019-06-27 21:36:16,852 ERROR coreos_koji_tagger - Running command returned bad exitcode
2019-06-27 21:36:16,853 ERROR coreos_koji_tagger - COMMAND: ['/usr/bin/koji', 'moshimoshi']
2019-06-27 21:36:16,853 ERROR coreos_koji_tagger -  STDOUT: b''
2019-06-27 21:36:16,853 ERROR coreos_koji_tagger - STDERR: b'2019-06-27 21:36:16,824 [ERROR] koji: AuthError: unable to obtain a session\n'

I know @mizdebsk mentioned that the user would need to be granted permissions. Is that the problem here? Can you add the same permissions to the user as the coreos-bot user has?

@mizdebsk can you create the coreos_koji_tagger_webhook_secret ansible var too and pass that to me in a private message?

he created the coreos_koji_tagger_webhook_secret_prod and coreos_koji_tagger_webhook_secret_stg variables for me today and I updated the roles to use them.

current status: still having this issue: https://pagure.io/fedora-infrastructure/issue/7870#comment-579894

Are you sure you are connecting to staging koji with tthe stg keytab? ie, stg-koji or koji -p staging ?

there was a bug where I wasn't using stg-koji.. but i just updated it and added debug output. Looks like I still have the same problem:

2019-06-28 22:15:54,714 INFO coreos_koji_tagger - Authenticating with keytab: /etc/coreos-koji-tagger-keytab/koji-keytab
2019-06-28 22:15:54,714 DEBUG coreos_koji_tagger - Running command: ['/usr/bin/klist', '-k', '/etc/coreos-koji-tagger-keytab/koji-keytab']
2019-06-28 22:15:54,720 DEBUG coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG in keytab
2019-06-28 22:15:54,720 INFO coreos_koji_tagger - Using principal coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG
2019-06-28 22:15:54,720 DEBUG coreos_koji_tagger - Running command: ['/usr/bin/kinit', '-k', '-t', '/etc/coreos-koji-tagger-keytab/koji-keytab', 'coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG']
2019-06-28 22:15:54,860 DEBUG coreos_koji_tagger - Running command: ['/usr/bin/stg-koji', 'moshimoshi']
2019-06-28 22:15:55,235 ERROR coreos_koji_tagger - Running command returned bad exitcode
2019-06-28 22:15:55,236 ERROR coreos_koji_tagger - COMMAND: ['/usr/bin/stg-koji', 'moshimoshi']
2019-06-28 22:15:55,236 ERROR coreos_koji_tagger -  STDOUT: b''
2019-06-28 22:15:55,236 ERROR coreos_koji_tagger -  STDERR: b'2019-06-28 22:15:55,208 [ERROR] koji: AuthError: unable to obtain a session\n'
Traceback (most recent call last):
  File "/usr/bin/fedora-messaging", line 11, in <module>
    load_entry_point('fedora-messaging==1.7.0', 'console_scripts', 'fedora-messaging')()
  File "/usr/lib/python3.7/site-packages/click/core.py", line 763, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 716, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1136, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/click/core.py", line 955, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 554, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/fedora_messaging/cli.py", line 144, in consume
    callback, bindings=bindings, queues=queues
  File "/usr/lib/python3.7/site-packages/fedora_messaging/api.py", line 108, in twisted_consume
    callback = _check_callback(callback)
  File "/usr/lib/python3.7/site-packages/fedora_messaging/api.py", line 51, in _check_callback
    callback_object = callback()
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 117, in __init__
    self.kinit()
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 274, in kinit
    check_koji_connection(check=True) # Make sure it works
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 463, in check_koji_connection
    cp = runcmd(cmd, check=check, capture_output=True)
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 279, in runcmd
    cp = subprocess.run(cmd, **kwargs)
  File "/usr/lib64/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/stg-koji', 'moshimoshi']' returned non-zero exit status 1.

thought as as sanity check can someone confirm that stg-koji moshimoshi works? I'm using moshimoshi as a check that we can talk to the remote koji instance but it looks like that is failing for stage, where it works for prod.

though as as sanity check can someone confirm that stg-koji moshimoshi works?

just checked, it does work for my user (dustymabe) if I kinit to stg first. From the debug output above you can see that I kinit first and then run moshimoshi. This should work.

According to kevin it turns out that user doesn't exist in stage koji.

Any update here? If I understand correctly the code fails to authenticate to staging Koji. Please let me know if you need any assistance with debugging the issue.

@mizdebsk yes please.. the keytab that you provided in stage doesn't seem to be able to connect with stg koji. Here are some logs I obtained from the last crashed pod:

2019-07-26 23:04:31,091 DEBUG coreos_koji_tagger - Found principal coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG in keytab
2019-07-26 23:04:31,092 INFO coreos_koji_tagger - Using principal coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG
2019-07-26 23:04:31,092 DEBUG coreos_koji_tagger - Running command: ['/usr/bin/kinit', '-k', '-t', '/etc/coreos-koji-tagger-keytab/koji-keytab', 'coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG']
2019-07-26 23:04:31,227 DEBUG coreos_koji_tagger - Running command: ['/usr/bin/stg-koji', 'moshimoshi']
2019-07-26 23:04:31,617 ERROR coreos_koji_tagger - Running command returned bad exitcode
2019-07-26 23:04:31,617 ERROR coreos_koji_tagger - COMMAND: ['/usr/bin/stg-koji', 'moshimoshi']
2019-07-26 23:04:31,617 ERROR coreos_koji_tagger -  STDOUT: b''
2019-07-26 23:04:31,617 ERROR coreos_koji_tagger -  STDERR: b'2019-07-26 23:04:31,587 [ERROR] koji: AuthError: unable to obtain a session\n'
Traceback (most recent call last):
  File "/usr/bin/fedora-messaging", line 11, in <module>
    load_entry_point('fedora-messaging==1.7.1', 'console_scripts', 'fedora-messaging')()
  File "/usr/lib/python3.7/site-packages/click/core.py", line 763, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 716, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 1136, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/click/core.py", line 955, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/click/core.py", line 554, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/fedora_messaging/cli.py", line 144, in consume
    callback, bindings=bindings, queues=queues
  File "/usr/lib/python3.7/site-packages/fedora_messaging/api.py", line 108, in twisted_consume
    callback = _check_callback(callback)
  File "/usr/lib/python3.7/site-packages/fedora_messaging/api.py", line 51, in _check_callback
    callback_object = callback()
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 117, in __init__
    self.kinit()
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 274, in kinit
    check_koji_connection(check=True) # Make sure it works
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 471, in check_koji_connection
    cp = runcmd(cmd, check=check, capture_output=True)
  File "/usr/lib/python3.7/site-packages/coreos_koji_tagger.py", line 279, in runcmd
    cp = subprocess.run(cmd, **kwargs)
  File "/usr/lib64/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/stg-koji', 'moshimoshi']' returned non-zero exit status 1.

I've just tested the keytab locally on my laptop and works fine for me.

The code I used:

$ ipython3 
Python 3.7.3 (default, May 11 2019, 00:38:04) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import koji                                                                                                                                            

In [2]: ks = koji.ClientSession('https://koji.stg.fedoraproject.org/kojihub')                                                                                  

In [3]: ks.gssapi_login('coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG', '/home/kojan/tmp/kt/kt')                          
Out[3]: True

Running that code resulted in creation of Koji user: https://koji.stg.fedoraproject.org/koji/userinfo?userID=coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG

My conclusion is that configuration of Koji CLI (/etc/koji.conf or /etc/koji.conf.d/) is not correct. Like I said before, my recommendation is to use Koji Python API instead of Koji CLI.

I've just tested the keytab locally on my laptop and works fine for me.

can you try using the commands from the log output and see if it works for you?

/usr/bin/kinit -k -t /etc/coreos-koji-tagger-keytab/koji-keytab coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG
/usr/bin/stg-koji moshimoshi

The code I used:
$ ipython3
Python 3.7.3 (default, May 11 2019, 00:38:04)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import koji

In [2]: ks = koji.ClientSession('https://koji.stg.fedoraproject.org/kojihub')

In [3]: ks.gssapi_login('coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG', '/home/kojan/tmp/kt/kt')
Out[3]: True

Running that code resulted in creation of Koji user: https://koji.stg.fedoraproject.org/koji/userinfo?userID=coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG
My conclusion is that configuration of Koji CLI (/etc/koji.conf or /etc/koji.conf.d/) is not correct. Like I said before, my recommendation is to use Koji Python API instead of Koji CLI.

I don't think "rewrite your code" is the right response here. It would be different if the code never worked but it has literally been working for months with the keytab that @kevin provided to us a while back. While I agree that using the python API would be better in the future (maybe we'll get some time freed up for a rewrite), spending time on that right now is not something I want to do.

Can you help my find the real problem? We can even grab a terminal on one of the pods in stg and debug from there if you think that would be useful.

ok i think this is starting to look more like an environment issue. Using your same commands from above I get an error when trying this in the stage openshift environment:

sh-5.0$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37)
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import koji
>>> ks = koji.ClientSession('https://koji.stg.fedoraproject.org/kojihub')
>>> ks.gssapi_login('coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG', '/etc/coreos-koji-tagger-keytab/koji-keytab')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/site-packages/koji/__init__.py", line 2315, in gssapi_login
    raise AuthError('unable to obtain a session')
koji.AuthError: unable to obtain a session

Can you try this from the running pod in stage (use the terminal from the web interface)? The pod is sleeping forever.

ok this was all due to the infra kerberos config needing to be different than when you are outside of infra. I added a configmap that will add in the infra kerberos config to the pod and now we are getting farther.

I need someone to set up some tags in stg-koji just like is done in prod.

$ koji list-tags | grep coreos | grep -v module
coreos-pool
coreos-release
f29-coreos-continuous
f29-coreos-signing-pending
f30-coreos-continuous
f30-coreos-signing-pending
f31-coreos-signing-pending

I've configured staging Koji tags and targets as requested.
I used the following Ansible playbook with koji-ansible:

- name: Configure Koji for CoreOS
  hosts: localhost
  gather_facts: False
  tasks:
    - name: Configure Koji tags
      local_action:
        module: koji_tag
        name: "{{ item.name }}"
        extra: "{{ item.get('extra', dict()) }}"
        inheritance: "{{ item.get('inheritance') }}"
        external_repos: "{{ item.get('external_repos') }}"
        packages: "{{ item.get('packages') }}"
        groups: "{{ item.get('groups') }}"
        arches: "{{ item.get('arches') }}"
      with_items:
        - name: coreos-pool
          arches: "aarch64 ppc64le s390x x86_64"
          extra:
            tag2distrepo.enabled: true
            tag2distrepo.keys: "429476b4 cfc659b9 3c3359c4"
        - name: coreos-release
        - name: f29-coreos-continuous
          arches: "aarch64 ppc64le s390x x86_64"
          extra:
            tag2distrepo.enabled: true
        - name: f29-coreos-signing-pending
          inheritance:
            - parent: coreos-pool
              priority: 0
        - name: f30-coreos-continuous
          arches: "aarch64 ppc64le s390x x86_64"
          extra:
            tag2distrepo.enabled: true
        - name: f30-coreos-signing-pending
          inheritance:
            - parent: coreos-pool
              priority: 0
        - name: f31-coreos-signing-pending
          inheritance:
            - parent: coreos-pool
              priority: 0
      loop_control:
        label: "{{ item.name }}"

    - name: Configure Koji build targets
      local_action:
        module: koji_target
        name: "{{ item.name }}"
        build_tag: "{{ item.build_tag }}"
        dest_tag: "{{ item.dest_tag }}"
      with_items:
        - name: f29-coreos-continuous
          build_tag: f29-build
          dest_tag: f29-coreos-continuous
        - name: f30-coreos-continuous
          build_tag: f30-build
          dest_tag: f30-coreos-continuous
      loop_control:
        label: "{{ item.name }}"

Thanks @mizdebsk - can you create the coreosbot user in stage koji?

When we add packages to a tag we define the owner of that package as coreosbot.

sh-5.0$ stg-koji add-pkg coreos-pool ostree --owner coreosbot
User coreosbot does not exist

Also it looks like maybe the user associated with the keytab you created doesn't have appropriate permissions either. If I workaround the coreosbot issue by using my own username I get a permission error:

sh-5.0$ stg-koji add-pkg coreos-pool ostree --owner dustymabe
Adding 1 packages to tag coreos-pool
2019-08-08 13:08:06,818 [ERROR] koji: ActionNotAllowed: policy violation (package_list)

Thanks @mizdebsk - can you create the coreosbot user in stage koji?

Koji users are created upon login to Koji. You need to create the user in staging FAS, create keytab for the user and then log into staging Koji with the keytab.

Also it looks like maybe the user associated with the keytab you created doesn't have appropriate permissions either.

The coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG user doesn't have any permissions. Which permissions do you need granted? coreos-continuous?

Thanks @mizdebsk - can you create the coreosbot user in stage koji?

Koji users are created upon login to Koji. You need to create the user in staging FAS, create keytab for the user and then log into staging Koji with the keytab.

For some reason I thought this would have been synced from prod already.. I'll create that user in stage FAS.

Also it looks like maybe the user associated with the keytab you created doesn't have appropriate permissions either.

The coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG user doesn't have any permissions. Which permissions do you need granted? coreos-continuous?

Yes I think that's the only set of permissions for coreos users we have right now. I think this is what we need to be set up in stage: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/koji_hub/templates/hub.conf.j2#n94

Koji users are created upon login to Koji. You need to create the user in staging FAS, create keytab for the user and then log into staging Koji with the keytab.

For some reason I thought this would have been synced from prod already.. I'll create that user in stage FAS.

That didn't work. I went to the sign up page and entered the email and I never got an email to confirm so I could continue the account sign up process.

ok @smooge got the coreosbot account set up in stage and I can talk to stg koji:

$ stg-koji moshimoshi
안녕하세요, coreosbot!

You are using the hub at https://koji.stg.fedoraproject.org/kojihub
Authenticated via GSSAPI

So the account in koji is set up.

The only part that is left is:

The coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG user doesn't have any permissions. Which permissions do you need granted? coreos-continuous?

Yes I think that's the only set of permissions for coreos users we have right now. I think this is what we need to be set up in stage: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/koji_hub/templates/hub.conf.j2#n94

I've granted coreos-continuous permission to appropriate users (staging only):

$ koji --noauth -p stg list-history --permission coreos-continuous --editor mizdebsk --active
Mon Aug 12 10:42:33 2019 permission coreos-continuous granted to coreosbot by mizdebsk [still active]
Mon Aug 12 10:42:34 2019 permission coreos-continuous granted to coreos-koji-tagger/coreos-koji-tagger.stg.fedoraproject.org@STG.FEDORAPROJECT.ORG by mizdebsk [still active]

Thanks @mizdebsk. Looks like it's working for now. Is that configuration set up so that when stage get's rebuilt it will get re-instated?

Since the permissions were already granted in production Koji, they should be preserved during staging Koji sync.

ok new blocker. I need it set up such that tagging things into the signing tags will get signed and then moved to the coreos-pool tag like is done in prod:

$ git diff
diff --git a/roles/robosignatory/files/robosignatory.staging.py b/roles/robosignatory/files/robosignatory.staging.py
index 81f622f23..692daec2c 100644
--- a/roles/robosignatory/files/robosignatory.staging.py
+++ b/roles/robosignatory/files/robosignatory.staging.py
@@ -39,6 +39,24 @@ config = {
                 # Temporary tags
                 # Infra tags
                 # Gated coreos-pool tag
+                {
+                    "from": "f29-coreos-signing-pending",
+                    "to": "coreos-pool",
+                    "key": "fedora-29",
+                    "keyid": "429476b4"
+                },
+                {
+                    "from": "f30-coreos-signing-pending",
+                    "to": "coreos-pool",
+                    "key": "fedora-30",
+                    "keyid": "cfc659b9"
+                },
+                {
+                    "from": "f31-coreos-signing-pending",
+                    "to": "coreos-pool",
+                    "key": "fedora-31",
+                    "keyid": "3c3359c4"
+                },
                 # Gated rawhide and branched
                 {
                     "from": "epel8-signing-pending",

FYI: we moved the source for this project into a subdir of a common coreos releng repo. It is now located here: https://github.com/coreos/fedora-coreos-releng-automation/tree/master/coreos-koji-tagger

Metadata Update from @dustymabe:
- Issue priority set to: Waiting on Assignee (was: Waiting on Reporter)

a month ago

can I please get someone to help me with this today?

ok @kevin helped me get the robosig setup applied. Unfortunately it still doesn't seem to be working:

sh-5.0$ stg-koji untag-build f30-coreos-signing-pending systemd-241-2.gita09c170.fc30
sh-5.0$ echo $?
0
sh-5.0$ stg-koji tag-build f30-coreos-signing-pending systemd-241-2.gita09c170.fc30
Created task 90009571
Watching tasks (this may be safely interrupted)...
90009571 tagBuild (noarch): free
90009571 tagBuild (noarch): free -> closed
  0 free  0 open  1 done  0 failed

90009571 tagBuild (noarch) completed successfully
sh-5.0$ stg-koji list-tagged f30-coreos-signing-pending
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------
systemd-241-2.gita09c170.fc30             f30-coreos-signing-pending  zbyszek
sh-5.0$ stg-koji list-tagged coreos-pool
Build                                     Tag                   Built by
----------------------------------------  --------------------  ----------------

ok @kevin helped me get this working in stg. thanks @kevin

Metadata Update from @dustymabe:
- Issue priority set to: Waiting on Reporter (was: Waiting on Assignee)

a month ago

ok I've addressed all issues from @puiterwijk in the security audit and also the code is now rewritten to use the python koji library as suggested by @mizdebsk and @puiterwijk. I think we can now move this to prod.

any opposed ?

+1 for prod, I don't see any further blockers off hand...

Thanks @kevin. I'll attempt to move it to prod later today.

Metadata Update from @dustymabe:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

a month ago

Login to comment on this ticket.

Metadata