#7983 Provide documentation on how ci pipelines can be hooked into Fedora Gating
Opened a month ago by bookwar. Modified 20 days ago

Afaik the messaging flow looks as follows:
Gating_Messages.png

But we need exact data: messaging endpoint, topics, schema for each type of message and so on.

There are some rumors and guesses flying around, but we need an authoritative source of this information.


I can help you with some pieces of this document but I think a lot of this is going to be coming from you :)

Sorry, but no :)

We can talk about future plans and adjustments, but the actual information should come directly from resultsdb-updater and other services as it is implemented now in Fedora Infra.

We need a starting point to talk about what we can and should change.

As an added point of discussion, this spawned from a ticket I filed in ci/general about what topic to use for the system that is going to be replacing taskotron eventually.

Are message topics supposed to represent the nature of the messages, the intent of the messages, the machines that currently emit the messages or some combination of the above?

What topics is the gating system expecting? Is there some general guidance on designing message topics that I've not found?

I don't know of any documentation on this. In fedmsg we had a fedmsg-infra-meta to note all the topics, but with fedora-messaging I think it's up to the sending app to define things.

Personally, I would say the message topics:
should represent a status at a point in time.
should have the application/emitter clearly noted so you could tell what thing emitted it.
Be machine/instance agnostic. It should reflect the application/user not the specific instance, thats a detail.
Should note prod or staging
Shoud note owning 'org'
Should start broadly and then get more specific if desired.

Perhaps @jcline could chime in here with any more pointers.

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

a month ago

As an added point of discussion, this spawned from a ticket I filed in ci/general about what topic to use for the system that is going to be replacing taskotron eventually.
Are message topics supposed to represent the nature of the messages, the intent of the messages, the machines that currently emit the messages or some combination of the above?

Message topics are for clients to filter messages on, so they should be about important parts of the message. For example, a message from koji about a package build should probably start with koji.build. Clients are probably also interested in filtering messages by build status, so it could include a segment with queued, running, completed, failed, canceled, or similar. Finally, clients are probably also interested in filtering messages by package (and possibly by distro or something) so an example topic would be koji.build.running.f30.kernel. Users who care about kernel builds in any state for f30 could get those messages by subscribing to koji.build.*.f30.kernel. Consumers that care about all koji builds can do koji.build.# or if they want only f30 builds koji.build.*.f30.*.

It's best to start more general and get more specific for each section of the topic. Message topics don't currently look like this, but if you're making a new message you should make a topic that allows for easy filtering (since that's its only purpose). Topics should map to a message schema, so if you decide to change the message schema, make a new topic. For details on AMQP topic rules, see https://www.rabbitmq.com/amqp-0-9-1-reference.html#queue.bind.routing-key.

What topics is the gating system expecting? Is there some general guidance on designing message topics that I've not found?

https://fedora-messaging.readthedocs.io/en/stable/publishing.html#overview is a place to start, and there's more stuff in https://fedora-messaging.readthedocs.io/en/stable/messages.html. There's not a huge number of restrictions, but my recommendation is to keep it simple, include the smallest amount of data you need and nothing else, and write a detailed schema for it with documentation.

I don't know of any documentation on this. In fedmsg we had a fedmsg-infra-meta to note all the topics, but with fedora-messaging I think it's up to the sending app to define things.
Personally, I would say the message topics:
should represent a status at a point in time.
should have the application/emitter clearly noted so you could tell what thing emitted it.
Be machine/instance agnostic. It should reflect the application/user not the specific instance, thats a detail.
Should note prod or staging

I don't think this is useful in the topic. The URL namespaces the staging broker from the production broker, there's no value in namespacing beyond that.

Shoud note owning 'org'

If this is recommending that the topics start with org.fedoraproject I don't think that's useful, either. The broker namespaces the messages, and topic lengths are limited to 255 characters. It's long, and offers no filtering value.

Should start broadly and then get more specific if desired.
Perhaps @jcline could chime in here with any more pointers.

So for package ci, would a reasonable topic for results from a check not specific to the package (ie not stored in dist-git) against the ever popular foo-1.2-3.fc30 be ci.fedora.prod.package.generic.testname.foo.fc30.1.2-3 or is that too much specific information in the topic?

So for package ci, would a reasonable topic for results from a check not specific to the package (ie not stored in dist-git) against the ever popular foo-1.2-3.fc30 be ci.fedora.prod.package.generic.testname.foo.fc30.1.2-3 or is that too much specific information in the topic?

I think that may be too much, what are the chances people want to subscribe to a test for a particular version?

I think keeping it to org.centos.{prod|stg}.ci.fedora.prod.package.generic.testname.foo.fc30 would be enough and provide sufficient information for anyone to filter based on topics :)

Interesting discussion. We wanted to standardize the topic and message content here:

https://pagure.io/fedora-ci/messages

This is also what we use internally for CI. Is there some problem with this standard?

Unfortunately we did not get to the point where we migrate the current Fedora CI pipeline and the Fedora resultsdb updater to this message format :(

So looking at:
org.fedoraproject.prod.ci.<artifact>.<event>.{queued,running,complete,error}

if you define artifact as : foo and event as testname you're close to Tim's proposal. The only part that is missing is fc30.

I'd stick with org.centos.{prod|stg} since this is what emits those messages though.

Metadata Update from @pingou:
- Issue assigned to pingou

a month ago

Interesting discussion. We wanted to standardize the topic and message content here:
https://pagure.io/fedora-ci/messages

This is also what we use internally for CI. Is there some problem with this standard?
Unfortunately we did not get to the point where we migrate the current Fedora CI pipeline and the Fedora resultsdb updater to this message format :(

I managed to miss the fact that the topic was part of the standard.

Other than that, I have no issues with the standard, just trying to figure out what's currently being used. I've been having trouble getting consistent answers on what's expected and what currently exists

I'd stick with org.centos.{prod|stg} since this is what emits those messages though.

Doesn't that conflict with some of what @jcline wrote above? It makes sense to me that the messages should reflect their usage rather than where they were created.

That being said, if that's what resultsdb-updater is looking for, that's what going to be used, I guess.

Tbh, while I think it's good to standardize the content of the message, I'm not sure we should lock ourselves in with the topics.
Jeremy has given some very good input as to what a good topic is and it would be a pity not to try to take advantage of it.

So maybe we can improve on the standard, or rather makes suggestions in it and just "require" that the most essential steps (running/complete) be there?

Food for thought, I won't fight this battle :)

Doesn't that conflict with some of what @jcline wrote above? It makes sense to me that the messages should reflect their usage rather than where they were created.

I don't think one goes against the other :)

That being said, if that's what resultsdb-updater is looking for, that's what going to be used, I guess.

resultsdb-listener is looking for whatever we tell it to look for, so that's not really an issue :)

Doesn't that conflict with some of what @jcline wrote above? It makes sense to me that the messages should reflect their usage rather than where they were created.

I don't think one goes against the other :)

I don't understand how "message topics should represent the content of the messages so that they can be more easily filtered" and "messages should reflect where they're emitted from" don't conflict some. "org.centos" seems superfluous from a content perspective unless there's something I'm not understanding.

I also think that using "org.centos" for Fedora CI is short-sighted but I can't argue that it's the convention that we're already using. It's also out of scope for the discussion here.

resultsdb-listener is looking for whatever we tell it to look for, so that's not really an issue :)

That's great but I still don't understand whether resultsdb-updater needs to be changed to look for new message topics. If it does require changes, how does it need to be changed? Do I need to be making these changes? Can I just file a ticket? Who do I pester once the changes are in place?

"org.centos" seems superfluous from a content perspective unless there's something I'm not understanding.

It kinda is superfluous, but it also does no harm and gives the people working on the infrastructure some clues as to where the message comes from.
Technically, it could also be used as a mechanism to monitor if messages sent by org.centos are reaching the bus fine as they should be.

That's great but I still don't understand whether resultsdb-updater needs to be changed to look for new message topics.

Yes it does, it has a defined list of topics it is look at, so if you want it to listen to more, that list needs to be adjusted.

If it does require changes, how does it need to be changed? Do I need to be making these changes? Can I just file a ticket?

You can just file a ticket, but like everything else in FOSS, it's normally quicker if you send a pull-request.

Who do I pester once the changes are in place?

You don't pester anyone, you kindly ask for a status update (either in the PR or in a ticket if you think it is appropriate) :)

Login to comment on this ticket.

Metadata
Attachments 1
Attached a month ago View Comment