#298 Greenwave queries Bodhi when Bodhi queries Greenwave
Closed: Fixed 5 years ago Opened 5 years ago by bowlofeggs.

It seems that Greenwave queries Bodhi when Bodhi queries Greenwave. This causes significant extra load on Bodhi (and last night caused it to exhaust its database connection pool and begin to refuse other requests) by a factor of about 2x for Bodhi web requests (since every Bodhi web request becomes effectively two).

If Greenwave needs more info about an update, why not have Bodhi pass that extra info along with the initial request so we don't need the extra round trip? It should perform better, and will save 50% of Bodhi's db connections in its pool.


If decision query contains subject with type bodhi_update, Greenwave will also get all NVRs in the update (asking Bodhi) and base the decision on these (types koji_build).

But it looks like Bodhi already passes all builds in decision query.

So if this is undesired behavior, we can remove it. We would also need to omit publishing decision change messages with for decision type bodhi_update when a test result for a build from given Bodhi update changes (since the data for simple decision with type bodhi_update wouldn't change when an underlying build changes).

If Greenwave needs more info about an update, why not have Bodhi pass that extra info along with the initial request so we don't need the extra round trip?

Ah, the reason we introduced this in the first place is because greenwave itself needs to know the list of builds that bodhi would pass it, so that it can ask itself the same question when new results arrive on the message bus. (i.e., a new result arrives about a koji build: it can announce decision changes about that koji build in isolation, but what we really want is for it to announce decision changes about real questions expected of it: questions about bodhi updates). This was what the "api v2" stuff was all about a few months ago that @dcallagh was driving.

I see the problem, though. Every request to bodhi will turn into two requests to bodhi.

WDYT, @lholecek - do you have a preferred way out of this logjam? We will have a similar problem with eventual Errata Tool integration.

Can we keep message publication on the level of bodhi_update while still making the API optionally accept a full list of builds along the the update id, reducing the total request time when bodhi queries for decisions about itself?

I would rather remove the dependency on Bodhi from Greenwave completely. My proposal is to handle bodhi_update subject types only by checking any test results for the update itself but not the builds it contains - user will have to ask explicitly for the builds.

It would simplify a lot of things for GW but it would also mean that when a decision changes for some koji_build, GW won't publish decision change message for any Bodhi updates - Bodhi itself will have to check which updates contain the build.

Is that feasible?

@lholecek I think that's better. It's strange to have an API such that the caller has the information the callee needs, but doesn't pass it to it, having the callee call back the caller to get it.

Also, as I understand it, Bodhi is already passing the full list of NVRs to Greenwave anyway:

https://github.com/fedora-infra/bodhi/blob/3.10.0/bodhi/server/models.py#L1961-L1976

Does that look right to you?

Yes! Let me remove all the code. :)

Metadata Update from @lholecek:
- Issue assigned to lholecek

5 years ago

Metadata Update from @lholecek:
- Issue tagged with: enhancement

5 years ago

Is there any reason to ask for a decision for subject type bodhi_update? (E.g. {'item': 'FEDORA-...', 'type': 'bodhi_update', ...}) I don't see any policy with this subject type in configuration [1].

The reason I'm asking is because after removing some code related to Bodhi, I get HTTP 404 when asking just for the bodhi_update (because there is no applicable policy).

[1] https://greenwave.fedoraproject.org/api/v1.0/policies

We don't currently have any results coming in in terms of a bodhi_update, but I think we were planning for the future with that.. expecting that some changes would have tests that only make sense in terms of the update as a whole (repo closure-type things).

Hey, if we're removing this ability to have a request for a decision about a bodhi_update not automatically expand into a full decision about all the builds on the update.. can we make sure we retain the functionality on the backend where a resultsdb result about a single build can still trigger a message about a decision update for the bodhi update as a whole? Make it modal, so that questions posed directly to the frontend by bodhi don't get automatically expanded into questions about all the nvrs on the update (so as not to double-back and put 2X load on bodhi), but so that the backend can still request an aggregate decision for messaging purposes.

Today, bodhi doesn't use those messages so we might consider throwing them away... however, I think our friends in the Errata Tool are expecting to someday use messages like that for their purposes (except, for a new subject_type like advisory which we haven't implemented yet).

can we make sure we retain the functionality on the backend where a resultsdb result about a single build can still trigger a message about a decision update for the bodhi update as a whole?

I would rather keep the decision change consistent with the decision itself. I'm not even sure how it would work with the decision_update decisions not being affected by test results for its builds anymore.

I.e. if a build triggers decision change (for the koji_build) and it is found in an update, greenwave also publishes decision change message about the update. The problem will be that decision for the bodhi_update will not change because will no longer depend on any of its builds.

It's probably easier for Bodhi and Errata Tool to check if an build is in an update/advisory than leave the job to Greewave.

That's fair. @bowlofeggs, @sbaird - are you guys okay with taking that on?

This would mean (if you ever want to do message-driven interaction with greenwave) that you have to listen to result messages from resultsdb directly and waiver messages from waiverdb directly -- and on each one, identify if it is for a build that is on an update or advisory in your respective databases, and if it is then you need to POST a query to greenwave asking if this made any decisions change for that update or advisory, and if it did... take some action (automatically promote the advisory or update).

We were trying to to handle this responsibility in greenwave's messaging backend. We don't have any message-driven interaction between bodhi and greenwave or between errata-tool and greenwave today, so it's okay on face value to drop this today. Any major concerns over not having this responsibility provided by greenwave in the future?

Login to comment on this ticket.

Metadata
Related Pull Requests
  • #319 Merged 5 years ago