#10938 Please send openh264-2.3.0-1.fc36 and openh264-2.3.0-1.fc37 to Cisco
Closed: Fixed 3 months ago by humaton. Opened 4 months ago by kalev.

Please sign etc and send the following two builds to Cisco for hosting:


This is a new openh264 2.3.0 upstream release together with gstreamer1-plugin-openh264 update from 1.20.0 to 1.20.3.


Metadata Update from @phsmoura:
- Issue tagged with: medium-gain, medium-trouble, ops

4 months ago

Update: Now that branching is done, I've also done openh264-2.3.0-1.fc38 build. Please send this to Cisco as well.

Sorry, the builds were sent to cisco last week, expecting an update from them soon.

The packages are now on ciscos CDN. composes have been pushed to sundries and mm.

Metadata Update from @humaton:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

4 months ago

Thanks, humaton! Something seems to be wrong though and I'm getting a checksum error:

# rm -rf /var/cache/dnf/fedora-cisco-openh264*
# dnf update
Fedora 37 openh264 (From Cisco) - x86_64                                                                                                                      1.6 kB/s | 2.3 kB     00:01    
Errors during downloading metadata for repository 'fedora-cisco-openh264':
  - Downloading successful, but checksum doesn't match. Calculated: 97344260b9b0de31b1cec26a6427705c5de0c9c938edfceb605530495e1a4a8976fa437c6aa01a710fb6f1cd9171f5edfe4249d7368535fada388389a7f64f42(sha512)  Expected: eddfca52fc408fc79745e751c4c759dcead8b8366b1a8d5da0b4981a6b09edbb6349dc906a77b39d9fc17ba9ac4c69b174ead81e773a8e34d0a2a1d884e223f3(sha512) 
Error: Failed to download metadata for repo 'fedora-cisco-openh264': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Metadata Update from @kalev:
- Issue status updated to: Open (was: Closed)

4 months ago

Yes. Our Fedora CoreOS tests are failing on multiple streams (f37/f36) with this problem.

I don't understand where is the expected checksum coming from. @adrian can you help with this?

Looking into it. The database has the correct values. Not sure where the wrong ones are coming from.

It should be fixed in about 40 minutes. It is not clear, however, why it was not autodetected.

The necessary scripts runs once every day but only running it manually fixed it. Unfortunately I ran the script before looking closer at the database values and now I do not know the state of the DB before running the script.

We probibly have a database backup from last night if that would help?

The repo checksum issue seems fixed now (thanks, adrian!) but now package GPG check is failing with "Package is not signed" error:

# dnf update
Fedora 37 openh264 (From Cisco) - x86_64                                                                                                                      3.3 kB/s | 2.5 kB     00:00    
Dependencies resolved.
 Package                                                Architecture                       Version                                    Repository                                         Size
 gstreamer1-plugin-openh264                             x86_64                             1.20.3-1.fc37                              fedora-cisco-openh264                              25 k
 mozilla-openh264                                       x86_64                             2.3.0-1.fc37                               fedora-cisco-openh264                             428 k
 openh264                                               x86_64                             2.3.0-1.fc37                               fedora-cisco-openh264                             423 k

Transaction Summary
Upgrade  3 Packages

Total download size: 876 k
Is this ok [y/N]: y
Downloading Packages:
(1/3): gstreamer1-plugin-openh264-1.20.3-1.fc37.x86_64.rpm                                                                                                     32 kB/s |  25 kB     00:00    
(2/3): openh264-2.3.0-1.fc37.x86_64.rpm                                                                                                                       479 kB/s | 423 kB     00:00    
(3/3): mozilla-openh264-2.3.0-1.fc37.x86_64.rpm                                                                                                               479 kB/s | 428 kB     00:00    
Total                                                                                                                                                         844 kB/s | 876 kB     00:01     
Package gstreamer1-plugin-openh264-1.20.3-1.fc37.x86_64.rpm is not signed
Package mozilla-openh264-2.3.0-1.fc37.x86_64.rpm is not signed
Package openh264-2.3.0-1.fc37.x86_64.rpm is not signed
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

Did the packages that were sent to Cisco actually have signatures attached?

Unfortunately this is an emergency. Attempts to update Fedora will fail until the Cisco repo is fixed. Users have noticed on reddit:


Please ask Cisco to remove these packages ASAP.

So, I made new repodata for f36/f37 that just points to the previous signed packages (ignoring the new unsigned ones).

Rawhide I just made new empty repos/metadata.

I think that should finish rolling out soon.

We can then regroup, send cisco new packages and update again.

Everything is back to working from my viewpoint. (Of course the new versions are now not available).

So, we should resign them and get them to cisco to update.

I put in a PR to update our SOP to add information about signing and some more info about updating things.


Keeping this ticket open to track the update...

Thanks for dealing with this mess, Kevin!

I have tagged back the builds generated new composes and send new signed rpms to cisco.

Well bad news, we found an ABI break when trying to update GNOME to OpenH264 2.3.0, see here for details. Suffice to say the update is broken and should not be used. Cisco will need to release a new version with a soname bump.

Drat! Please ask Cisco not to ship it. It seems the earlier trouble with the RPM signing and now their slowness to update their repo has saved us here.

No, please don't ask Cisco to not ship it, we can make things work even though there was an ABI break. gstreamer1-plugin-openh264 and mozilla-openh264 already got automatically rebuilt (because they come from the same SRPM as openh264) and ffmpeg that's in Fedora repos is already getting fixed. See https://bugzilla.redhat.com/show_bug.cgi?id=2124073

It was unexpected to me that ffmpeg had started dlopening openh264 so I didn't know to look out for a potential ABI break here. Sorry! I'll know to look out for ffmpeg next time.

In any case, it's much easier and quicker to fix ffmpeg that we have in Fedora proper than to ask a 3rd party to pull packages.

OK, well GNOME and freedesktop-sdk both decided to pull the broken update and wait for Cisco to release a corrected update, so hopefully there will be a 2.3.1 soon regardless. Neither GNOME nor freedesktop-sdk accepts surprise ABI breaks without a soname bump. I've already communicated this to Cisco, and it would be ideal to maintain a consistent message.

Since 2.3.0 is out I am closing this ticket

Metadata Update from @humaton:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Boards 1
Ops Status: Backlog