#8641 fedora-messaging cert (stg|prod) for coreos-ostree-importer
Closed: Fixed 3 months ago by dustymabe. Opened 3 months ago by dustymabe.

Describe what you would like us to do:


As part of the coreos ostree importing we would like to:

  • consume org.fedoraproject.prod.coreos.build.request.ostree-import messages (sent by the fedora coreos pipeline running in CentOS CI)
  • run an ostree import
  • publish a org.fedoraproject.prod.coreos.build.request.ostree-import.finished message

The coreos-ostree-importer is an openshift project so storing them in as a variable private ansible repo should suffice.

Can we have a username and queue name of coreos-ostree-importer?

When do you need this to be done by? (YYYY/MM/DD)


Sometime this week would be nice


Metadata Update from @smooge:
- Issue assigned to smooge

3 months ago

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

3 months ago

coreos-ostree-importer and coreos-ostree-importer.stg have been created in the trees.

Metadata Update from @smooge:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 months ago

Thanks @smooge - I finally got through my issues in Fedora infra openshift and have the pods up and running I'm seeing this issue when staring up a fedora messaging consumer:

2020-02-17 20:55:16,278 WARNING pika.channel - Received remote Channel.Close (403): "ACCESS_REFUSED - access to queue 'coreos-ostree-importer.stg' in vhost '/pubsub' refused for user 'coreos-ostree-importer.stg'" on <Channel number=2 OPEN conn=<pika.adapters.twisted_connection._TwistedConnectionAdapter object at 0x7fe6511f7610>>
2020-02-17 20:55:16,279 ERROR fedora_messaging.cli - Unable to declare the queue object on the AMQP broker. The broker responded with (403, "ACCESS_REFUSED - access to queue 'coreos-ostree-importer.stg' in vhost '/pubsub' refused for user 'coreos-ostree-importer.stg'"). Check permissions for your user.

So it looks like the coreos-ostree-importer.stg user doesn't have access to the coreos-ostree-importer.stg queue.

I'll re-open this for now. Will re-close it if I'm doing something dumb or have some sort of misconfiguration.

Metadata Update from @dustymabe:
- Issue status updated to: Open (was: Closed)

3 months ago

You need to have somewhere in ansible the queue/user being created.

You could add it to roles/rabbitmq_cluster/tasks/apps.yml or since your app is already in ansible, add it to that...

See for example the bodhi playbook:

  • role: rabbit/user
    username: "bodhi{{ env_suffix }}"
  • role: rabbit/queue
    username: "bodhi{{ env_suffix }}"
    queue_name: "{{ bodhi_message_queue_name }}"
    routing_keys: "{{ bodhi_message_routing_keys }}"
    thresholds:
    warning: 10
    critical: 100

I believe I already do have this: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/playbooks/openshift-apps/coreos-ostree-importer.yml#n29

I pinged you last friday when this part of my playbook failed. I saw some updates in the thread on the mailing list so I thought everything in staging was good now so I proceeded.

@abompard perhaps you have some idea here?

here is a snippet from the last time I ran the playbook:

TASK [rabbit/user : Validate parameters] *************************************************************************************************************
Monday 17 February 2020  20:39:07 +0000 (0:00:00.065)       0:00:07.728 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org] => {                         
    "changed": false,                                                     
    "msg": "All assertions passed"                                        
}                                                                         

TASK [rabbit/user : Create the user in RabbitMQ] *****************************************************************************************************
Monday 17 February 2020  20:39:07 +0000 (0:00:00.099)       0:00:07.828 *******                                                                     
changed: [os-master01.stg.phx2.fedoraproject.org -> rabbitmq01.stg.phx2.fedoraproject.org]                                                          

TASK [rabbit/queue : Validate parameters] ************************************************************************************************************
Monday 17 February 2020  20:39:13 +0000 (0:00:05.837)       0:00:13.666 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org] => {                         
    "changed": false,                                                     
    "msg": "All assertions passed"                                        
}                                                                         

TASK [rabbit/queue : Validate the user parameter] ****************************************************************************************************
Monday 17 February 2020  20:39:13 +0000 (0:00:00.080)       0:00:13.747 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org] => {                         
    "changed": false,                                                     
    "msg": "All assertions passed"                                        
}                                                                         

TASK [rabbit/queue : Create the coreos-ostree-importer.stg user in RabbitMQ] *************************************************************************
Monday 17 February 2020  20:39:13 +0000 (0:00:00.108)       0:00:13.855 *******                                                                     
changed: [os-master01.stg.phx2.fedoraproject.org -> rabbitmq01.stg.phx2.fedoraproject.org]                                                          

TASK [rabbit/queue : Create the coreos-ostree-importer.stg queue in RabbitMQ] ************************************************************************
Monday 17 February 2020  20:39:18 +0000 (0:00:04.533)       0:00:18.389 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org -> rabbitmq01.stg.phx2.fedoraproject.org]                                                               

TASK [rabbit/queue : Bind the coreos-ostree-importer.stg queue to the amq.topic exchange] ************************************************************
Monday 17 February 2020  20:39:19 +0000 (0:00:00.802)       0:00:19.191 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org -> rabbitmq01.stg.phx2.fedoraproject.org] => (item=org.fedoraproject.*.coreos.build.request.ostree-import)

TASK [rabbit/queue : Bind the coreos-ostree-importer.stg queue to the zmq.topic exchange] ************************************************************
Monday 17 February 2020  20:39:20 +0000 (0:00:00.785)       0:00:19.977 *******                                                                     
ok: [os-master01.stg.phx2.fedoraproject.org -> rabbitmq01.stg.phx2.fedoraproject.org] => (item=org.fedoraproject.*.coreos.build.request.ostree-import)

and I just ran the playbook again and it seems that it always shows changed for the following two plays:

  • TASK [rabbit/user : Create the user in RabbitMQ]
  • TASK [rabbit/queue : Create the coreos-ostree-importer.stg user in RabbitMQ]

I just dug into the rabbit/user/tasks/main.yml file and I do notice it says: Ensure a user exists in RabbitMQ with permissions to only publish.

In this case I'm trying to read from the queue (as mentioned in the description of this issue) as well as publish. Is that not allowed?

ok I just used a small script to confirm I don't get an error when publishing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#!/usr/bin/python3
from fedora_messaging import api, message
topic = 'org.fedoraproject.stg.coreos.build.request.ostree-import'
body = {
    "build_id": "31.20200210.2.0",
    "stream": "testing",
    "basearch": "x86_64",
    "commit_url": "https://builds.coreos.fedoraproject.org/prod/streams/testing/builds/31.20200210.2.0/x86_64/fedora-coreos-31.20200210.2.0-ostree.x86_64.tar",
    "checksum": "sha256:99255f7bdd27011eb58355c82cffcbd6b1dfed7fdfe337d949696acd5554c844",
    "ostree_ref": "fedora/x86_64/coreos/testing",
    "ostree_checksum": "3e4a6a48ed8d6817081c902bc2aa2bbe9df3302e659d4f42d933f9abb22914e8",
    "target_repo": "compose"
}
api.publish(message.Message(topic=topic, body=body))

but I don't see the message show up in datagrepper. I'm clearly doing something wrong.

ahh, nevermind. I found it. the topic had org.fedoraproject.stg in it twice and that's why I couldn't find it.

So the real question is: can we really not consume/publish using the same config?

You can consume and publish at the same time, just use the rabbit/queue role and not the rabbit/user role. The queue role will create the user. The user role is only for apps that need to publish and not consume.

Metadata Update from @dustymabe:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Metadata