#9385 Please turn on: server-named queues for fedora-messaging
Closed: Fixed 2 years ago by asaleh. Opened 3 years ago by astepano.

Hello.

Please help me understand, why fedora-messaging doesn't support Server-named Queues?
Is there any decisions that this feature is turned off?

https://www.rabbitmq.com/queues.html#server-named-queues

In AMQP 0-9-1, the broker can generate a unique queue name on behalf of an app. To use this feature, pass an empty string as the queue name argument... 

Moreover, in tutorial it is suggested to use empty name for queue:

https://www.rabbitmq.com/tutorials/tutorial-three-python.html

Firstly, whenever we connect to Rabbit we need a fresh, empty queue. To do it we could create a queue with a random name, or, even better - let the server choose a random queue name for us. We can do this by supplying empty queue parameter to queue_declare:
result = channel.queue_declare(queue='')

According to https://fedora-messaging.readthedocs.io/en/latest/quick-start.html#fedora-s-public-broker queue name is required.

From this decision errors like this emerge:

https://github.com/jenkinsci/jms-messaging-plugin/issues/200#event-3741621507

Please turn this feature ON. Because it is part of standard. Otherwise, please help me understand why it turned OFF, and maybe update document: https://fedora-messaging.readthedocs.io/en/latest/quick-start.html#fedora-s-public-broker with explanation.

Thank you!


Metadata Update from @pingou:
- Issue tagged with: rabbitmq

3 years ago

Metadata Update from @mohanboddu:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

3 years ago

I think it was considered by @jcline when designing the public broker. I'm not sure why he decided on client-specific, uuid-based queue names. Perhaps for permissions or garbage-collecting reasons. Jeremy, could you chime in?

Metadata Update from @abompard:
- Issue untagged with: medium-gain, medium-trouble, ops
- Issue priority set to: Needs Review (was: Waiting on Assignee)

3 years ago

I'm hazy on the choices I made, and of course there may not have been a strong reason at all.

If I remember correctly it was mostly because I assumed folks would overwhelmingly want their queues to survive longer than an AMQP channel and forcing them to name the queue would avoid the "I don't understand why all my messages disappeared when my channel closed" type complaints?

As far as a know the broker does support server-generated names, but obviously the permissions restrictions might clash with whatever names it generates. I probably reasoned it was just as easy to generate a name client-side, but maybe that wasn't such a good idea.

If it's something folks want it's probably as easy as figuring out the RabbitMQ name generation and adding the appropriate access controls/garbage collection for such names.

@jcline thank you for the input.

I need to add: server-named queues for fedora-messaging + in scope of access to fedora-s-public-broker. I am totally support to disable server generated queues for private virtual host.

However, this feature It would be useful to access publicly accessible virtual host.

Such queues already have garbage collector + limit in size + inactivity cleanup.

For public virtual hosts, I think it is OK for consumers to generate each time new UUID + set exclusive = true. This is useful feature.

If I specify queue name as empty string '' broker creates a queue!
But, fedora user cannot access it:

Error: Channel closed by server: 403 (ACCESS-REFUSED) with message "ACCESS_REFUSED - access to queue 'amq.gen-fQdrfUzszithmUDuGB1Cgw' in vhost '/public_pubsub' refused for user 'fedora'"

This is because fedora user has access to queues that have distinct UUID format.
But, queue was created: amq.gen-fQdrfUzszithmUDuGB1Cgw.

https://fedora-messaging.readthedocs.io/en/latest/quick-start.html#fedora-s-public-broker

Queue names *must* be in the normal UUID format

Some kind of inconsistency, I think.

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

3 years ago

@astepano would you be willing to write a PR to move this forward? or figure out what format the named queues are so we can adjust cluster policy for them?

@abompard @jcline So, if I understand this right we need to adjust our permissions so server named queues work ?
Or did I misread what the actual action here is?

Hi, this is minor issue.

When connections is setup with public credentials [1].
Right now : public credentials MUST always specify queue name according to required pattern.
However, rabbitmq-standard allows that broker-server generates queue name.
Actually broker generates queue name, but public-credentials cannot access queue with generated name.

[1] https://raw.githubusercontent.com/fedora-infra/fedora-messaging/master/configs/fedora-cert.pem

@abompard @jcline So, if I understand this right we need to adjust our permissions so server named queues work ?
Or did I misread what the actual action here is?

Correct. I think just this block needs to have its regex changed: https://pagure.io/fedora-infra/ansible/blob/main/f/roles/rabbitmq_cluster/tasks/main.yml#_371

And the block above it controls cleaning up unused queues, it looks like it matches everything and should be fine.

@abompard can you do this when you add the acls? or just anytime before/after?

It's unrelated to the topic-authorization ticket, so yes I could do this anytime. But we should also probably update the documentation in fedora-messaging to explain that it's not necessary anymore to generate a queue name with uuid, and update the provided example configuration file too.

I could change the ACLs first as a form of "beta sneak peek" though, and update the docs later. (However "I'll update the docs later" really sounds like a receipe for oversight)

OK, tested on staging and deployed to prod. Server-named queues should now be available.

The changes needed to add support for that in the fedora-messaging library are backwards-incompatible, so it'll take a bit of time before users of the library can use server-named queues.
But if I understand correctly, it's not the case of the requester, so you should be all set.

As there are no new comments for past two weeks, I assume abompards changes are sufficient and am closing this issue. If this is not the case, please do reopen :-)

Metadata Update from @asaleh:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Hey! Just a heads-up: fedora-messaging 3.0 has landed in Fedora & EPEL 8, with support for anonymous (server-named) queues.

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Done