#8173 Update/fix varnish/apache setup for registry.fp.o
Closed: Fixed a month ago by kevin. Opened a month ago by jcajka.

It seems that the BZ#1737471 is not really a Go stdlib bug, but it is triggered by bug in tls stack of the proxy/cdn/cache(possibly openssl) of registry.fp.o as pointed out in the upstream issue.

Opening this issue to track it so it gets fixed when fix lands in Fedora.


ok. Currently our proxies are Fedora 29 instances, running the current updated openssl in Fedora 29.

After Fedora 31 is released we plan to move them up to that.

Is it worth keeping this open here? Or can you just open/reopen it when there's a fix thats landed in Fedora?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

a month ago

There are a few bodhi updates that attempt to fix this problem, but they are against f30 and f31.

Can we use those by any chance?

No, but we could use the f29 one. ;) https://bodhi.fedoraproject.org/updates/FEDORA-2019-f5810ab475

Will need a freeze break.

Can we duplicate this problem in staging? can we do so and then apply the update to confirm it fixes it?

Can we duplicate this problem in staging? can we do so and then apply the update to confirm it fixes it?

Yes we should be able to use staging to test that update.

now that the beta is a GO we are going to get heat from this if people can't use the registry. Yes they can use other registries OR set an env var to workaround, but would be nice if the user didn't hit a bug as soon as they started using f31 beta.

Hum. So, I checked this issue against our staging server(s):

podman pull registry.stg.fedoraproject.org/fedora:30
Trying to pull registry.stg.fedoraproject.org/fedora:30...
Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
Error: error pulling image "registry.stg.fedoraproject.org/fedora:30": unable to pull registry.stg.fedoraproject.org/fedora:30: unable to pull image: Error initializing source docker://registry.stg.fedoraproject.org/fedora:30: pinging docker registry returned: Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message

then I upgraded openssl on the proxy server:
[root@proxy01 ~][STG]# rpm -q openssl
openssl-1.1.1c-6.fc29.x86_64

Rebooted the server.

But the problem persists:
% podman pull registry.stg.fedoraproject.org/fedora:30
Trying to pull registry.stg.fedoraproject.org/fedora:30...
Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
Error: error pulling image "registry.stg.fedoraproject.org/fedora:30": unable to pull registry.stg.fedoraproject.org/fedora:30: unable to pull image: Error initializing source docker://registry.stg.fedoraproject.org/fedora:30: pinging docker registry returned: Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message

What am I missing here? httpd doesn't need to be rebuilt does it?

Hum. So, I checked this issue against our staging server(s):
podman pull registry.stg.fedoraproject.org/fedora:30
Trying to pull registry.stg.fedoraproject.org/fedora:30...
Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
Error: error pulling image "registry.stg.fedoraproject.org/fedora:30": unable to pull registry.stg.fedoraproject.org/fedora:30: unable to pull image: Error initializing source docker://registry.stg.fedoraproject.org/fedora:30: pinging docker registry returned: Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
then I upgraded openssl on the proxy server:
[root@proxy01 ~][STG]# rpm -q openssl
openssl-1.1.1c-6.fc29.x86_64
Rebooted the server.
But the problem persists:
% podman pull registry.stg.fedoraproject.org/fedora:30
Trying to pull registry.stg.fedoraproject.org/fedora:30...
Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
Error: error pulling image "registry.stg.fedoraproject.org/fedora:30": unable to pull registry.stg.fedoraproject.org/fedora:30: unable to pull image: Error initializing source docker://registry.stg.fedoraproject.org/fedora:30: pinging docker registry returned: Get https://registry.stg.fedoraproject.org/v2/: local error: tls: unexpected message
What am I missing here? httpd doesn't need to be rebuilt does it?

Yes, might need the mod_ssl package to be rebuilt using the latest version on openssl

Eagerly waiting for all kinds of updates needed on my production Fedora 31 Silverblue system to get toolbox working. Without that I can't do my development work.

tested the corrected update and it does indeed fix things in stg. Sent a FBR in.

Production now fixed.

➜ ~ podman pull registry.fedoraproject.org/fedora:30 
WARN[0000] The configuration is using `runtime_path`, which is deprecated and will be removed in future.  Please use `runtimes` and `runtime` 
WARN[0000] If you are using both `runtime_path` and `runtime`, the configuration from `runtime_path` is used 
Trying to pull registry.fedoraproject.org/fedora:30...
Getting image source signatures
Copying blob ed60cb1abc2e done
Copying config 02781e9f50 done
Writing manifest to image destination
Storing signatures
02781e9f507f260dded796ec8784c36f9067a3d3739eb9e237afcd2735345b81

:panda_face:

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

a month ago

I confirm I can pull container images without any issues. Thank you Kevin.
:thumbsup:

Login to comment on this ticket.

Metadata