fedora-infrastructure

#7206 I think the staging OpenShift registry might be broken
Closed: Fixed Opened by bowlofeggs.

Closed: Fixed
Reopen Issue
  • Describe what you need us to do:
    It seems that the staging OpenShift registry might be broken. I first noticed that staging Bodhi doesn't work. When I looked into it, I saw an error like this:
$ oc describe pod bodhi-web-21-q4cct
<snip>
Events:
  Type     Reason            Age                From                                           Message
  ----     ------            ----               ----                                           -------
  Normal   Scheduled         14m                default-scheduler                              Successfully assigned bodhi-web-21-q4cct to os-node02.stg.phx2.fedoraproject.org
  Warning  Failed            14m (x2 over 14m)  kubelet, os-node02.stg.phx2.fedoraproject.org  Error: ImagePullBackOff
  Normal   Pulling           14m (x2 over 14m)  kubelet, os-node02.stg.phx2.fedoraproject.org  pulling image "docker-registry.default.svc:5000/bodhi/bodhi-web@sha256:f35d85f9145646974a18a2a5338bb9ed77e0071d702544515d13bd9232077a87"
  Warning  Failed            14m (x2 over 14m)  kubelet, os-node02.stg.phx2.fedoraproject.org  Failed to pull image "docker-registry.default.svc:5000/bodhi/bodhi-web@sha256:f35d85f9145646974a18a2a5338bb9ed77e0071d702544515d13bd9232077a87": rpc error: code = Unknown desc = pinging docker registry returned: Get https://docker-registry.default.svc:5000/v2/: dial tcp 172.30.19.56:5000: getsockopt: connection refused

I thought that was weird, but maybe temporary, and I wanted to deploy a new Bodhi anyway, but then I found that my Bodhi build failed in a similar way when it tried to push the new image:

Pushing image docker-registry.default.svc:5000/bodhi/bodhi-web:latest ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Warning: Push failed, retrying in 5s ...
Registry server Address: 
Registry server User Name: serviceaccount
Registry server Email: serviceaccount@example.org
Registry server Password: <<non-empty>>
error: build error: Failed to push image: After retrying 6 times, Push image still failed due to error: Get https://docker-registry.default.svc:5000/v1/_ping: dial tcp 172.30.19.56:5000: getsockopt: connection refused

  • When do you need this? (YYYY/MM/DD)
    As soon as reasonable for staging.

  • When is this no longer needed or useful? (YYYY/MM/DD)
    If it starts working again, or if we stop using OpenShift because some new hipster thing replaces it.

  • If we cannot complete your request, what is the impact?
    I am unable to test the Bodhi 3.10.0 beta in staging and will be forced to deploy it to production on a Friday before going on PTO for 3 weeks.

(just in case anybody freaks out, the thing about deploying on a Friday was a joke. I'd never do that. I'd deploy it on a Friday before going on PTO for 3 weeks and wouldn't even check to see if the deployment worked at all before leaving!)

This should be all fixed now hopefully...

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
Metadata
None

OpenShift staging

None
None
Normal
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.