#10992 tracking down what's going wrong with RPM IMA signatures
Closed: Fixed with Explanation a year ago by kevin. Opened a year ago by fche.

This issue is to track findings in the ongoing investigation into what's going or not going on with RPM IMA signatures, which are to have been activated as of f37 mass-rebuilds months ago. https://pagure.io/fesco/issue/2887


  • downloading rpms from koji via the webapp is not dispositive, because it doesn't return the post-robosign rpms but the raw built ones - https://koji.fedoraproject.org/koji/buildinfo?buildID=2074988
  • koji build logs also don't contain references to signatures, which occur later
  • on log01, keyctl log records indicate apparently successful signing efforts for f37, f38 rpms, e.g.:
Nov 16 04:25:29 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory.tagconsumer INFO] Build libreoffice-7.4.2.3-1.fc38 (2074988) tagged into f38-signing-pending on primary
Nov 16 04:25:29 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory.tagconsumer INFO] Going to sign libreoffice-7.4.2.3-1.fc38 with fedora-38 (eb10b464) and file signing key fedora-38-ima and move to f38-updates-testing-pending
Nov 16 04:25:29 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory.tagconsumer INFO] Signing and writing build libreoffice-7.4.2.3-1.fc38
Nov 16 04:27:44 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory.tagconsumer INFO] RPMs to sign and move: ['libreoffice-7.4.2.3-1.fc38.src (32479856, signed: False)', 'libreoffice-calc-debuginfo-7.4.2.3-1.fc38.i686 (32479857, signed: False)', 'libreoffice-core-debuginfo-7.4.2.3-1.fc38.i686 (32479858, signed: False)', 'libreoffice-core-7.4.2.3-
1.fc38.i686 (32479859, signed: False)', 'libreoffice-debugsource-7.4.2.3-1.fc38.i686 (32479860, signed: False)', 'libreoffice-sdk-doc-7.4.2.3-1.fc38.i686 (32479861, signed: False)', 'libreoffice-writer-debuginfo-7.4.2.3-1.fc38.i686 (32479862, signed: False)', 'libreoffice-debuginfo-7.4.2.3-1.fc38.i686 (32479863, signed: False)', 'libreoffice-help-dz-7.4.2.3-1.
fc38.i686 (32479864, signed: False)', 'libreoffice-help-el-7.4.2.3-1.fc38.i686 (32479865, signed: False)', 'libreoffice-ure-debuginfo-7.4.2.3-1.fc38.i686 (32479866, signed: False)', 'libreoffice-help-bg-7.4.2.3-1.fc38.i686 (32479867, signed: False)', 'libreoffice-help-ja-7.4.2.3-1.fc38.i686 (32479868, signed: False)', 'libreoffice-help-bn-7.4.2.3-1.fc38.i686 (
32479869, signed: False)', 'libreoffice-help-ru-7.4.2.3-1.fc38.i686 (3247987 [...]
[...]
Nov 16 04:45:02 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory INFO] Received message from fedora-messaging with topic: org.fedoraproject.prod.buildsys.tag
Nov 16 04:45:02 autosign01.iad2.fedoraproject.org keyctl[1749602]: [robosignatory.tagconsumer INFO] Build libreoffice-7.4.2.3-1.fc38 (2074988) tagged into f38-updates-testing-pending on primary
  • taking libreoffice-7.4.2.3-1.fc38.x86_64.rpm as an example, /mnt/fedora_koji_prod/koji/packages/libreoffice/7.4.2.3/1.fc38/x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm is the unsigned version that is served from the koji webapp
  • and /mnt/fedora_koji_prod/koji/packages/libreoffice/7.4.2.3/1.fc38/data/signed/eb10b464//x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm is the signed version that I believe goes into bodhi/compose outputs
  • from a fedora infra machine that has access to both files raw:
$ rpm -qivp x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm | grep Sign
Signature   : (none)
$ rpm -qivp data/signed/eb10b464/x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm | grep Sign
warning: data/signed/eb10b464/x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID eb10b464: NOKEY
Signature   : RSA/SHA256, Wed 16 Nov 2022 04:32:57 AM UTC, Key ID 809a8d7ceb10b464
$ rpm -q --qf='%{FILESIGNATURES}\n' -p data/signed/eb10b464/x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm
warning: data/signed/eb10b464/x86_64/libreoffice-7.4.2.3-1.fc38.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID eb10b464: NOKEY
(none)

FYI, I talked to @pbrobinson about this, and he indicated he talked to @puiterwijk (who set things up) and he said he would look into it, so hopefully he will provide some insight here soon. :)

Metadata Update from @phsmoura:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: dev

a year ago

Not sure we need another ticket for this.

ok. We think this is fixed.

It was a issue with rpm on the sign-vault servers. It wasn't entirely doing the right thing with the ima signatures. ;(

Patrick patched it and I helped debug and we got something that seems working well now.

The fixed build is:

https://koji.fedoraproject.org/koji/buildinfo?buildID=2112313

uboot-tools-2023.01-1.fc38 was the first package signed correctly. Everything else that has ima signatures set in config after that should also be working.

ie:

~ koji download-build --key eb10b464 uboot-tools-2023.01-1.fc38 --arch x86_64
~ sudo dnf install rpm-plugin-ima
~ sudo dnf install uboot-tools-2023.01-1.fc38.x86_64.rpm
~ getfattr --absolute-names -d -m - /usr/bin/bmp_logo                        
# file: /usr/bin/bmp_logo
security.ima=0sAwIE57DIWQBGMEQCIGhC3fNy/zNADAp3fbH0eg5w7358JSKzVMR28I67Tg3jAiBu+TTbjmGsm/JyPG71hczCYdJ5e9N5sHIOEb4UBr9GUw==
security.selinux="system_u:object_r:bin_t:s0"

So, hopefully we are back on track here.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

a year ago

(also please let's get a process to publish the signing keys somewhere like https://getfedora.org/security/ or inside an rpm)

Yep. Agreed. Patrick is going to send me a script to get this and we can then put it on security or the like. Perhaps it should also be in the fedora-repos package with the gpg keys.

Yep. Agreed. Patrick is going to send me a script to get this and we can then put it on security or the like. Perhaps it should also be in the fedora-repos package with the gpg keys.

Yes, that was the plan to have it on disk somewhere for local tools and remotely for independent validation.

Login to comment on this ticket.

Metadata
Boards 1
dev Status: Backlog