#90 ci-resultsdb-listener and openqa results
Opened a year ago by lrossett. Modified 4 months ago

Hello,

Opening this issue based on what was discussed in IRC (my username is "odra").

We are in the process of migrating/deploying ResultsDB to kubernetes/openshift which includes "ci-resultsdb-listener" as well.

I was wondering if "openqa results" and "ci-resultsdb-listener" are basically doing the same thing and if it would be possible to merge both into "ci-resultsdb-listener" by making it more configurable - this change should also include using basic auth requests in "ci-resultsdb-listener".

There were some concerns such as log format in "openqa results" being different than the format used in "ci-resultsdb-listener" which could be discussed in here instead of IRC.


They don't do the same thing in technical terms; the openQA result publishing doesn't use the CI messages at all (primarily because they didn't exist when we wrote the result publishing).

But they do the "same thing" in terms of what is ultimately achieved: publish a result to resultsdb for completed tests.

It's been long enough since we wrote this stuff and the things that use it that I don't remember all the ins and outs off the top of my head, and I'm not familiar with exactly how ci-resultsdb-listener works or what format it publishes results in. We'll have to look into the details. The most important thing that consumes the results is Greenwave; there are also fedora_nightlies and check-compose, off the top of my head.

The existing reporting code is here. It uses resultsdb_conventions, which was my attempt to write something to make it convenient to report results in consistent format from multiple test systems, but no other systems really adopted it.

The current CI message publishing code is here. You can obviously see how typical results and CI messages look in the resultsdb web UI and datagrepper.

I was actually wondering if the logs/message format could be configured from an environment variable or even just loading a python file from a config file to deal with it, this way we could have one flexible app - by one app I mean one code base.

honestly I'd kinda rather keep things as consistent as possible :D bodhi/greenwave already have to do some dumb stuff to account for different resultsdb formats. I think basing the resultsdb message on the - fairly well standardized and documented/schema'd - CI format would be great overall. just need to find time to dig into the details and make adjustments to things that consume the results where appropriate. right now there's all this fire all over the place...

sorry I haven't got to this yet :( it's constantly the next thing on my list at the end of the day, then something sets on fire the next morning...

So, now F35 is done, I have some time for this. One thing that immediately comes up is: what's the relationship between ci-resultsdb-listener and resultsdb-updater? They appear to do much the same thing. @mvadkert @lholecek

For using ci-resultsdb-listener, I see several issues right away. It basically assumes all messages will be for Koji builds, it makes several assumptions of that. In the building of the data dict:

    data = {
        "build_url": message.body["run"]["url"],
        "repo": message.body["artifact"]["component"],
        "nvr": message.body["artifact"]["nvr"],
        "item": message.body["artifact"]["nvr"],
        "scratch": message.body["artifact"]["scratch"],
        "username": message.body["artifact"]["issuer"],
        "koji_task_id": message.body["artifact"]["id"],
        "namespace": message.body["test"]["namespace"],
        "type": message.body["artifact"]["type"].replace("-", "_"),
        "test_type": message.body["test"]["type"],
        "category": message.body["test"]["category"],
    }

lots of those are unsafe assumptions, particularly the things pulled from the "artifact" dict. If you look at the message schemas, in the koji-build schemas - e.g. koji-build.test.complete - the artifact is defined as an instance of the rpm-build schema, and that has all the properties ci-resultsdb-listener expects. But that's not the only thing an 'artifact' can be! If you look at the fedora-update schemas - e.g. fedora-update.test.complete - the artifact in those messages is defined as an instance of the fedora-update schema. And that doesn't have the same properties as an rpm-build at all, and this code would choke on it all over the place. Ditto productmd-compose messages; for these the artifact is a productmd-compose, which again doesn't have the same properties at all.

Just from this quick look, the approach of resultsdb-updater seems superior, as it doesn't keep this kind of mapping logic 'in-house' but uses mappings that are stored in the message schema repository; so I could theoretically just write mappings for the topics openQA publishes on, and the tool would then be able to report the results.

@kevin just tagging you for info - this is one way to fix the "openQA can't report to resultsdb if auth is enabled" problem. The other way would just be to add auth support to fedora-openqa's own resultsdb reporting code, but I do like the idea of using one of these systems if we can make it work.

@adamwill

One thing that immediately comes up is: what's the relationship between ci-resultsdb-listener and resultsdb-updater

I guess no relationship, this piece (resultsdb updater from amqp messages) was never unified.

resultsdb-updater is being deprecated on downstream in favor of Hydra.

@lholecek @draval @timitra is it unreal to expect Hydra could be sometime deployed in Fedora for fedora-mesagging? So we finally get rid of this unneeded duplicity?

@mvadkert @lholecek @draval @timitra ping? Where does this hydra thing live, anyway? Is it inside the firewall?

AFAIK, it's an internal redhat project that lives inside the firewall, yes

On Mon, Jan 24, 2022 at 2:06 PM Adam Williamson pagure@pagure.io wrote:

adamwill added a new comment to an issue you are following:
@mvadkert @lholecek @draval @timitra ping? Where does this hydra thing live, anyway? Is it inside the firewall?

To reply, visit the link below or just reply to this email
https://pagure.io/fedora-qa/fedora_openqa/issue/90

Yeah, I'm following up on that internally and have found it now. It's currently RH internal-only and proprietary. I'm trying to see if this can be changed. If it stays that way, we can't really use it in Fedora, and would I guess have to keep shadowing it with ci-resultsdb-listener (and contribute improvements there) or adopt the 'obsolete' resultsdb-updater.

I believe we are moving in a direction we will submit things directly to ResultsDB. Right @msrb ?

oh, well that seems sort of a shame? I liked the concept of CI forwarding, because we have schemas for CI messages, so if we forward CI results to resultsdb we get a lot of consistency 'for free'. Whereas we don't have a successful schema-type thing for resultsdb (since nobody else adopted my resultsdb_conventions project or proposed an alternative), so there's nothing imposing any kind of consistency on resultsdb data. There is a lot of awkward inconsistency between existing resultsdb results which causes some awkwardness in greenwave and bodhi, and if more things start reporting direct to resultsdb with no kind of framework for consistency this might just get worse...

Login to comment on this ticket.

Metadata