So, I think this is actually a real bug somewhere, but I can't point a finger anywhere convincing yet, so filing here first for us to track.

The cockpit test that checks automatic updates work is failing about half the time on F30 and F31 updates. Normally that means the retry mechanism hides the problem from us, but sometimes we get unlucky and both attempts for an update fail.

The failure case looks like this: the test clicks on the 'Automatic Updates' slider, the thing where Cockpit confirms installation of dnf-automatic then does it happens, and then...the slider stays off. I can't tell if enabling automatic updates actually happened and the slider just wasn't updated, or it really failed, but the slider stays in off state. And so the test fails, as it expects it to go to the 'on' state.

This seems to be happening on F30 and F31 update tests. It does not seem to be happening on F32 update tests. It seems to have started happening about 22 days ago on F31, and 14 days ago on F30. AFAICS that time frame doesn't line up very well with a Cockpit update, or a Firefox update, or a DNF update, or a PackageKit update. F30's Cockpit hasn't been updated since 2020-02-27, its DNF not since January, and its PackageKit not since October last year. There were Firefox updates for F30 and F31 around the 'right' time frame, so that's the best suspect if anything, but...the earliest case of the issue on F30 was actually just before that update went stable, and I confirmed in the logs of that test that it used the older firefox-73.0.1-1.fc30. So even that doesn't seem to add up.

For now just filing this for tracking, if I find time I'll try and maybe reproduce this manually or get openQA to enable the browser console so we can get an idea what's going on in the javascript at least...

@lruzicka @martinpitt for info.