Greetings!
Bodhi 4 was deployed to production today. It is unfortunately not backwards compatible in a way that breaks that Bodhi 3 client, which is the current stable version in EPEL 7, Fedora 29, and Fedora 30.
So far, the breakage I am aware of is that it cannot render updates. It is able to create them, it seems, but it crashes after creating them while trying to print them to the screen. Thus, it's not highly severe, but it is confusing for packagers.
I have so far made Bodhi 4 available to Rawhide and I have also put it into Copr at bowlofeggs/bodhi.
While I could patch the Bodhi 3 client in the stable releases to be compatible with the Bodhi 4 server, that would then mean that the client in those releases would not be compatible with the server that is also shipped in those releases.
There are some packages that depend on Bodhi's client in Fedora. Some of them use it as a subprocess and some of them import it and use it as a Python library. There were some changes to both usages so we'd have to inspect each one case-by-case to see what they are doing to find out if it is safe or not.
Another complicating factor is that there are at least two other Bodhi deployments in the world, aside from Fedora's, and I cannot easily know if they are relying on Fedora's client or not. The Bodhi 4 client may just as well be incompatible with the Bodhi 3 server, so upgrading could break those deployments as well.
I don't have a proposal at this time. It would be good to gather more information so I can identify which dependencies might be affected if I were to update Bodhi to version 4.
Any suggestions or recommendations from FESCo here?
Would it be possible to patch the bodhi 3 client to be compatible with both versions? Maybe it doesn't have to provide the full informative output, just not crash.
$ dnf repoquery --whatrequires python3-bodhi-client bodhi-client-0:3.14.0-1.fc30.noarch bodhi-server-0:3.14.0-1.fc30.noarch fedora-easy-karma-0:0-0.40.20181206gitb6d97e0.fc30.noarch fedpkg-0:1.37-1.fc30.noarch packit-0:0.4.1-1.fc30.noarch $ dnf repoquery --whatrequires bodhi-client cmake-fedora-0:2.9.3-4.fc30.noarch fedora-packager-0:0.6.0.2-5.fc30.noarch
I'm with @zbyszek - we should make the client compatible with both, even if it has a large if.
For what it's worth, I'd much rather see the Bodhi 4 client capable of talking to both, that way we can just upgrade everyone everywhere cleanly.
@bowlofeggs This would be a tremendous hack, but could you just do auto-detection of which server you're talking to and then just keep both codebases in the library? Just drop the entirety of the v3 codebase there and fall back to it if you're talking to v3?
AFAIU this broke fedora-easy-karma: https://bugzilla.redhat.com/show_bug.cgi?id=1714925 Not sure if it is still the major tool to submit karma but if it is, there might be big loss of testing feedback now. This could make a lot of people unhappy so a quick solution would be great.
Here's another bug about one of the crashes: https://bugzilla.redhat.com/show_bug.cgi?id=1714950
I considered making the Bodhi 3 client be able to talk to the Bodhi 4 server. It could be done but will take some effort. Making Bodhi 4 talk to Bodhi 3 doesn't seem as good to me because then I have to maintain that code in the long term, and also it would mean upgrading the server to Bodhi 4 in stable Fedoras as well (which is very majorly backwards incompatible).
Another problem is that EPEL 7 actually still has Bodhi 2, and that code is even older. So a patch that works on Bodhi 3 would likely not apply on Bodhi 2…
Users who are hitting this can work around it for now by using my Copr, if they wish: https://copr.fedorainfracloud.org/coprs/bowlofeggs/bodhi
I had a chat with @churchyard in #fedora-admin today, and I think we came up with a reasonable proposal together (most of it his ideas ☺):
Note that this would mean we are not maintaining a Bodhi 3 server anymore for either release, just a client. To be transparent that is mostly just to save effort on our part.
If we can get this approved and implemented, we will make a similar proposal to the EPEL SC for EPEL 7.
we came up with a reasonable proposal together
What happens with packages that depend on python3-bodhi-client and bodhi-client? Will they be updated to the new API in F29/30?
They need to be, yes. It doesn't really serve any purpose if they use bodhi3-client and cannot talk to Fedora's Bodhi server.
On Wed, 2019-05-29 at 16:27 +0000, Zbigniew J=C4=99drzejewski-Szmek wrote:
We would need to check each one and see if it should work with bodhi 4 or not. If it doesn't, we can switch it to use the bodhi3 package.
They need to be, yes. It doesn't really serve any purpose if they use bodhi3-client and cannot talk to Fedora's Bodhi server. We would need to check each one and see if it should work with bodhi 4 or not. If it doesn't, we can switch it to use the bodhi3 package.
Those answers are contradictory ;) I think @churchyard is right here — it would be pointless if they can't talk to our Bodhi server. Until we know if they actually work with the new client, we can't judge the impact.
I was directed here from a thread about an error when running the hourly yum cronjob. The error received is:
Updateinfo file is not valid XML: <open file="" '="" var="" cache="" yum="" x86_64="" 7="" epel="" 92f2e15cad66d79ea1ad327e2af7af89d98e4d153d7a3e27ff41946f476af5b4-updateinfo.xml.zck',="" mode="" 'rt'="" at="" 0x7f29f8719660="">
The system does not have anything related to bodhi installed, so I cannot install a new version of that to fix it.
I have tried 'yum clean all' and have even deleted everything under /var/cache/yum ... but on running the cronjob or "yum check-update" the error still happens. I have seen messages indicating the repository has been fixed ... but that does not seem to be the case for me. Maybe some mirrors aren't yet, and the fastest mirror plugin is choosing one of those?
On Wed, 2019-05-29 at 20:41 +0000, Zbigniew J=C4=99drzejewski-Szmek wrote:
Those answers are contradictory ;) I think @churchyard is right here =E2=80=94 it would be pointless if they can't talk to our Bodhi server. U= ntil we know if they actually work with the new client, we can't judge the impact.
Yeah, I also agree with Miro's assessment.
The interfaces to the client haven't changed with 4, but the data that is returned from the server, and thus serialized by and expected by the client, is what changed. Since the interfaces haven't changed, I would expect the dependent packages to continue working unless they are using the data fields that were altered on the server end.
On Wed, 2019-05-29 at 21:13 +0000, Shawn Heisey wrote:
Updateinfo file is not valid XML: <open file="" '="" var="" cache="" yum="" x86_64="" 7="" epel="" 92f2e15cad66d79ea1ad327e2af7af89d98e4d1="" 53d7a3e27ff41946f476af5b4-updateinfo.xml.zck',="" mode="" 'rt'="" at="" 0x7f29f8719660="">
This is a separate issue from what we are discussing here, but the good news is that the code that caused it is now fixed and the next EPEL compose (which I think starts in about 2.5 hours) should resolve it (so, expect the issue to go away over the next day or three, depending on how long it takes to propagate to the mirror you use).
That issue is documented here: https://pagure.io/releng/issue/8392
Until we know if they actually work with the new client, we can't judge the impact.
Note that the problem already happened. Updating bodhi in stable Fedoras won't make it any worse, it can only help mitigate it.
The impact here would only be on 3rd party Bodhi servers. And while we should not break that assumption of stability, being able to talk to our own Bodhi server beats that IMHO.
On Wed, 2019-05-29 at 21:13 +0000, Shawn Heisey wrote: That issue is documented here: https://pagure.io/releng/issue/8392
I thought I was on that issue, didn't even realize I had switched to another one! Sorry for the mixup.
OK. So +1 to the plan in https://pagure.io/fesco/issue/2137#comment-573003.
I'm +1 as well, for the record.
OK. So +1 to the plan in https://pagure.io/fesco/issue/2137#comment-573003. +1 from me as well
+1 here also
Tagged with meeting to get fast resolution.
I won't be able to attend the meeting tomorrow, but you know my vote.
Metadata Update from @churchyard: - Issue tagged with: meeting
+1 to @bowlofeggs 's plan
+1 from me too
This was discussed during the FESCo meeting day: The proposal in https://pagure.io/fesco/issue/2137#comment-573003 is APPROVED (+5, 0, 0).
Metadata Update from @zbyszek: - Issue close_status updated to: Accepted - Issue status updated to: Closed (was: Open)
I have submitted a package review for the bodhi3 package: https://bugzilla.redhat.com/show_bug.cgi?id=1715936
Once we have that in updates-testing, I can add Bodhi 4 to Fedora 29/30.
Here is an update for Fedora 30 that updates to Bodhi 4.0.2 and introduces a bodhi3 compat client package: https://bodhi.fedoraproject.org/updates/FEDORA-2019-352078fd2b
Metadata Update from @zbyszek: - Issue untagged with: meeting
Here is an update for Fedora 29 that updates to Bodhi 4.0.2 and introduces a bodhi3 compat client package: https://bodhi.fedoraproject.org/updates/FEDORA-2019-5ce9adadc1
Login to comment on this ticket.