#7377 The Rawhide container has GPG issues
Closed: Fixed 2 years ago Opened 2 years ago by bowlofeggs.

Greetings!

I've been having a few of my Bodhi CI tests fail recently due to the Rawhide container not being able to use the new F29 key:

$ sudo docker run --rm -it registry.fedoraproject.org/fedora:rawhide dnf install -y yubibomb
Fedora - Rawhide - Developmental packages for the next Fedora release                                                                                                              2.8 MB/s |  60 MB     00:21    
Last metadata expiration check: 0:00:27 ago on Tue Mar  6 18:08:46 2018.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                           Arch                                            Version                                                  Repository                                        Size
===================================================================================================================================================================================================================
Installing:
 yubibomb                                          x86_64                                          0.2.0-1.fc28                                             rawhide                                          105 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 105 k
Installed size: 247 k
Downloading Packages:
yubibomb-0.2.0-1.fc28.x86_64.rpm                                                                                                                                                   170 kB/s | 105 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                               73 kB/s | 105 kB     00:01     
warning: /var/cache/dnf/rawhide-2d95c80a1fa0a67d/packages/yubibomb-0.2.0-1.fc28.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 429476b4: NOKEY
Importing GPG key 0x9DB62FB1:
 Userid     : "Fedora 28 (28) <fedora-28@fedoraproject.org>"
 Fingerprint: 128C F232 A937 1991 C8A6 5695 E08E 7E62 9DB6 2FB1
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64
Key imported successfully
Import of key(s) didn't help, wrong key(s)?
Public key for yubibomb-0.2.0-1.fc28.x86_64.rpm is not installed. Failing package is: yubibomb-0.2.0-1.fc28.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

I've talked to Patrick about this, and it sounds like we just need to rebuild the Rawhide container and publish it.


Container is rebuilt everynight. we likely just need to push anew one for rawhide @mohanboddu do you know how to push base images

@ausil I just did sync the rawhide container base images.

@bowlofeggs could you please give it a try and let us know.

Hmm, the issue does not seem to be resolved, though I did see a new container image:

[rbarlow@ohm ~]$ sudo docker pull registry.fedoraproject.org/fedora:rawhide
Trying to pull repository registry.fedoraproject.org/fedora ... 
sha256:ac9a4842b4a6ff1d668ab07086f9ba175f06f538710e94d0d4be100977ae7add: Pulling from registry.fedoraproject.org/fedora
47354f1458ef: Pull complete 
Digest: sha256:ac9a4842b4a6ff1d668ab07086f9ba175f06f538710e94d0d4be100977ae7add
Status: Downloaded newer image for registry.fedoraproject.org/fedora:rawhide
[rbarlow@ohm ~]$ sudo docker run --rm -it registry.fedoraproject.org/fedora:rawhide dnf install -y yubibomb
Fedora - Rawhide - Developmental packages for the next Fedora release                                                                                                              3.1 MB/s |  60 MB     00:19    
Last metadata expiration check: 0:00:26 ago on Tue Mar  6 22:45:19 2018.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                           Arch                                            Version                                                  Repository                                        Size
===================================================================================================================================================================================================================
Installing:
 yubibomb                                          x86_64                                          0.2.0-1.fc28                                             rawhide                                          105 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 105 k
Installed size: 247 k
Downloading Packages:
[MIRROR] yubibomb-0.2.0-1.fc28.x86_64.rpm: Downloading successful, but checksum doesn't match. Calculated: 1ce728b091d10aa9bf0eca69397702c4ac42b99d97e140c76d3dbbf227fac9d9(sha256)  Expected: 4fa89799fa40188137752cda52ffc44a9dc176e586ba57f90d1c1af42149c409(sha256) 
yubibomb-0.2.0-1.fc28.x86_64.rpm                                                                                                                                                    91 kB/s | 105 kB     00:01    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                               79 kB/s | 105 kB     00:01     
warning: /var/cache/dnf/rawhide-2d95c80a1fa0a67d/packages/yubibomb-0.2.0-1.fc28.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 429476b4: NOKEY
Importing GPG key 0x9DB62FB1:
 Userid     : "Fedora 28 (28) <fedora-28@fedoraproject.org>"
 Fingerprint: 128C F232 A937 1991 C8A6 5695 E08E 7E62 9DB6 2FB1
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64
Key imported successfully
Import of key(s) didn't help, wrong key(s)?
Public key for yubibomb-0.2.0-1.fc28.x86_64.rpm is not installed. Failing package is: yubibomb-0.2.0-1.fc28.x86_64
 GPG Keys are configured as: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-x86_64
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

It looks like the current rawhide image was created on January 5:

$ http https://registry.fedoraproject.org/v2/fedora/manifests/rawhide
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 0
AppServer: proxy06.fedoraproject.org
AppTime: D=119920
Connection: Keep-Alive
Content-Length: 2164
Content-Type: application/vnd.docker.distribution.manifest.v1+prettyjws
Date: Tue, 06 Mar 2018 22:49:17 GMT
Docker-Content-Digest: sha256:8e0e0099f0cb8069f70bc01e58986d888c2025dd8079c8d5ca14317062826298
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:8e0e0099f0cb8069f70bc01e58986d888c2025dd8079c8d5ca14317062826298"
Keep-Alive: timeout=15, max=500
Referrer-Policy: same-origin
Server: Apache/2.4.29 (Fedora)
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept
Via: 1.1 varnish (Varnish/5.1)
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Varnish: 6961862
X-Xss-Protection: 1; mode=block

{
   "schemaVersion": 1,
   "name": "fedora",
   "tag": "rawhide",
   "architecture": "amd64",
   "fsLayers": [
      {
         "blobSum": "sha256:47354f1458ef73173d9f0b0fc9d8f321ee8b96132ad36ab29595896623a536ac"
      }
   ],
   "history": [
      {
         "v1Compatibility": "{\"architecture\":\"amd64\",\"comment\":\"Created by Image Factory\",\"config\":{\"Systemd\":false,\"Hostname\":\"\",\"Entrypoint\":null,\"Env\":[\"DISTTAG=f27container\",\"FGC=f27\"],\"OnBuild\":null,\"OpenStdin\":false,\"MacAddress\":\"\",\"User\":\"\",\"VolumeDriver\":\"\",\"AttachStderr\":false,\"AttachStdout\":false,\"NetworkDisabled\":false,\"StdinOnce\":false,\"Cmd\":[\"/bin/bash\"],\"WorkingDir\":\"\",\"AttachStdin\":false,\"Volumes\":null,\"Tty\":false,\"Domainname\":\"\",\"Image\":\"\",\"Labels\":{\"version\":\"27\",\"vendor\":\"Fedora Project\",\"name\":\"fedora\",\"license\":\"MIT\"},\"ExposedPorts\":null},\"container_config\":{\"Systemd\":false,\"Hostname\":\"\",\"Entrypoint\":null,\"Env\":null,\"OnBuild\":null,\"OpenStdin\":false,\"MacAddress\":\"\",\"User\":\"\",\"VolumeDriver\":\"\",\"AttachStderr\":false,\"AttachStdout\":false,\"NetworkDisabled\":false,\"StdinOnce\":false,\"Cmd\":null,\"WorkingDir\":\"\",\"AttachStdin\":false,\"Volumes\":null,\"Tty\":false,\"Domainname\":\"\",\"Image\":\"\",\"Labels\":null,\"ExposedPorts\":null},\"created\":\"2018-01-05T13:30:55Z\",\"docker_version\":\"1.10.1\",\"id\":\"5c4435a47c5d28dab697fdabfb5b7ec8229f1a3454440feeffcdd579033c8373\",\"os\":\"linux\"}"
      }
   ],
   "signatures": [
      {
         "header": {
            "jwk": {
               "crv": "P-256",
               "kid": "QTLU:LB3L:NV57:RZ66:M77U:TE6R:V5LK:IMO6:V6RC:XYVA:Y7SN:3SAV",
               "kty": "EC",
               "x": "gvrnFunR2CfLJv1SguzV1UHx0Idrl0LIgcErGTTwgGQ",
               "y": "pDKBdCT7_lso71xZOoAzl42-sivMkos48SooSRIe268"
            },
            "alg": "ES256"
         },
         "signature": "sIpjgf9ofKgb86136I1mdnpcFmHhgS4RP0wiX4duutrHLco5NuyhwM5FFAB_zOhZnmLeKI4SWrq1LOCY__k5Ew",
         "protected": "eyJmb3JtYXRMZW5ndGgiOjE1MTcsImZvcm1hdFRhaWwiOiJDbjAiLCJ0aW1lIjoiMjAxOC0wMy0wNlQyMjo0OToxN1oifQ"
      }
   ]
}

(The timestamp is 2018-01-05T13:30:55Z and is buried in the "history" section of the JSON document.)

this also works:

$ skopeo inspect docker://registry.fedoraproject.org/fedora:rawhide | grep Created
    "Created": "2018-01-05T13:30:55Z",

so some bits of info:

The last FINISHED_INCOMPLETE rawhide we had was Fedora-Rawhide-20180303.n.0. The container base task 25482545 from that one did work (for primary architectures anyway). but I don't see the artifacts anywhere in the output directory: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20180303.n.0/compose/

Questions:
- Did the rename from Docker to Container mess up the pushing of containers to the registry?
- Did the rename from Docker to Container mess up the directory structure of the output pungi artifacts?

AFAIK pushing the container base images up to the registry is entirely manual and needs someone to run the process @maxamillion do we have that all documented somewhere?

AFAIK pushing the container base images up to the registry is entirely manual and needs someone to run the process @maxamillion do we have that all documented somewhere?

Mohan knows how to do that. I think the problem is more related to the two questions I asked above:

Questions:
- Did the rename from Docker to Container mess up the pushing of containers to the registry?
- Did the rename from Docker to Container mess up the directory structure of the output pungi artifacts?

It is kinda documented, but let me take a look.

@dustymabe @ausil @bowlofeggs

skopeo inspect docker://registry.fedoraproject.org/fedora:rawhide | grep Created
"Created": "2018-03-04T21:06:35Z",

Dusty questions:

Did the rename from Docker to Container mess up the pushing of containers to the registry?

I fixed it, and able to push new content as mentioned above

Did the rename from Docker to Container mess up the directory structure of the output pungi artifacts?

Still not sure what happened here, need little more digging.

I fixed it, and able to push new content as mentioned above

Thanks. @bowlofeggs can you test?

Still not sure what happened here, need little more digging.

@mohanboddu can we open a new ticket for this?

@dustymabe I will open a ticket if I dont find anything after some digging. I am guessing it might be related to pungi or pungi config.

This issue does seem to be resolved now:

$ sudo docker run --rm -it registry.fedoraproject.org/fedora:rawhide dnf install -y yubibomb                                                                                                    Fedora - Rawhide - Developmental packages for the next Fedora release                                                                                                              2.4 MB/s |  60 MB     00:24    
Last metadata expiration check: 0:00:27 ago on Wed Mar  7 17:09:56 2018.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                           Arch                                            Version                                                  Repository                                        Size
===================================================================================================================================================================================================================
Installing:
 yubibomb                                          x86_64                                          0.2.0-1.fc28                                             rawhide                                          105 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 105 k
Installed size: 247 k
Downloading Packages:
yubibomb-0.2.0-1.fc28.x86_64.rpm                                                                                                                                                   175 kB/s | 105 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                               80 kB/s | 105 kB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                           1/1 
  Installing       : yubibomb-0.2.0-1.fc28.x86_64                                                                                                                                                              1/1 
  Running scriptlet: yubibomb-0.2.0-1.fc28.x86_64                                                                                                                                                              1/1 
  Verifying        : yubibomb-0.2.0-1.fc28.x86_64                                                                                                                                                              1/1 

Installed:
  yubibomb.x86_64 0.2.0-1.fc28                                                                                                                                                                                     

Complete!

Metadata Update from @bowlofeggs:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata