Hello.
RabbitMQ broker sets messageId in message properties with different format.
messageId
properties
Message from RabbitMQ has properties. Possible properties are:
export interface MessageProperties { contentType: any | undefined; contentEncoding: any | undefined; headers: MessagePropertyHeaders; deliveryMode: any | undefined; priority: any | undefined; correlationId: any | undefined; replyTo: any | undefined; expiration: any | undefined; messageId: any | undefined; <------------- this is inconsistent for different route-keys (topics). timestamp: any | undefined; type: any | undefined; userId: any | undefined; appId: any | undefined; clusterId: any | undefined; }
Issue: messageId is inconsistent in properties.
Messages org.fedoraproject.prod.buildsys.tag route-key (topic) have messageId in format:
org.fedoraproject.prod.buildsys.tag
842579b6-700b-4050-a092-72d4fbb50ebc
Messages org.centos.prod.ci.koji-build.test.complete route-key have messageId in format: 2021-90574e88-59c8-4164-872e-d6d180149c38
org.centos.prod.ci.koji-build.test.complete
2021-90574e88-59c8-4164-872e-d6d180149c38
msg_id
Datagrepper always shows prefix for msg_id: started with 2021-: https://apps.fedoraproject.org/datagrepper/id?id=2021-842579b6-700b-4050-a092-72d4fbb50ebc&is_raw=true&size=extra-large
2021-
I am sure for 100% that RabbitMQ-broker delivers org.fedoraproject.prod.buildsys.tag messages to queues messageId property without 2021- prefix.
I am sure for 100% that RabbitMQ-broker delivers org.centos.prod.ci.koji-build.test.complete messages to queues messageId property with 2021- prefix.
Useful links:
https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.buildsys.tag&delta=127800
https://apps.fedoraproject.org/datagrepper/raw?topic=org.centos.prod.ci.koji-build.test.complete&delta=127800
https://pagure.io/fedora-infrastructure/issue/9666
@abompard @jcline
The message ID is set by the client (if using fedora-messaging, see https://github.com/fedora-infra/fedora-messaging/blob/a383960e7927ff2371b53c66a10bd0a6c0f70187/fedora_messaging/message.py#L329).
fedora-messaging
Whatever client publishes org.centos.prod.ci.koji-build.test.complete must be setting it with the prefix if the message is delivered with that message id to all subscribers (which isn't following the rules at https://fedora-messaging.readthedocs.io/en/stable/wire-format.html#message-id). If you're only seeing this prefix in datagrepper, though, it could be going through https://github.com/fedora-infra/datanommer/blob/3822d849028162ce6469125ecffa2c19eea2c531/datanommer.models/datanommer/models/__init__.py#L131 and thus the client isn't setting a message ID at all.
@jcline thank you for the info.
Let's try find our where this prefix comes from.
Source of messages that have prefix in messageId is: https://github.com/jenkinsci/jms-messaging-plugin
This has code:
this.msgId = Calendar.getInstance().get(1) + "-" + UUID.randomUUID().toString();
https://github.com/jenkinsci/jms-messaging-plugin/blob/master/plugin/src/main/java/com/redhat/jenkins/plugins/ci/messaging/data/RabbitMQMessage.java#L75
daee7469 (Oliver Gondža 2020-09-01 20:52:34 +0200 75) this.msgId = Calendar.getInstance().get(1) + "-" + UUID.randomUUID().toString();
@jcline could be this code be reason for this? Yeah, this is somewhat...
Sender add its own prefix, and datagrepper also (sometimes) add another prefix :-)
How could we find consistency? Is this broker to fix or sender? Thank you.
How could we find consistency? Is this broker to fix or sender?
This is not something the broker should fix - nor could it for datagrepper. Publishers (for better or worse) are supposed to set the message ID. Datagrepper's choice to generate a message ID is an interesting one; I lean towards ignoring malformed messages, but it's datagrepper's choice..
As for the jms-messaging-plugin, it should be adjusted.
@jcline thank for the info. I think this ticket can be closed.
I opened ticket for jms-messaging-plugin:
https://github.com/jenkinsci/jms-messaging-plugin/issues/218
Metadata Update from @kevin: - Issue close_status updated to: Will Not/Can Not fix - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.