Hi!
To speed up some of the testing we do on bare-metal machines provisioned through Duffy, I would like to pull pre-build images from the OpenShift registry. The images are built through a BuildConfig and placed in an ImageStream.
Now, it seems that the Duffy provisioned bare-metal systems can not pull from the internal OpenShift registry:
[root at n46 ~]# podman pull image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/ceph-csi/ceph-csi:test Trying to pull image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/ceph-csi/ceph-csi:test... Get https://image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/v2/: dial tcp 172.19.0.254:5000: connect: no route to host Error: error pulling image "image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/ceph-csi/ceph-csi:test": unable to pull image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/ceph-csi/ceph-csi:test: unable to pull image: Error initializing source docker://image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/ceph-csi/ceph-csi:test: error pinging docker registry image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000: Get https://image-registry.openshift-image-registry.svc.apps.ocp.ci.centos.org:5000/v2/: dial tcp 172.19.0.254:5000: connect: no route to host
I wonder if this is intentional, or if this access can be allowed.
Thanks!
Metadata Update from @arrfab: - Issue tagged with: centos-ci-infra, feature-request
Hi @devos
This usecase is one we have not considered supporting, we will have to do some research as it could be a little tricky. Hmm I'm not overly keen on exposing the internal registry via a route which would probably be the easiest solution.
As a workaround, might I suggest that you instead make use of one of the public image registries instead [1][2]? Could make use of the following to push your images to an external registry once built see: [3] as an example.
Hi David,
Yes, pushing the image to a public repository would work too. However the images are only expected to be used during tests. Normally these images get built on-demand, which is basically for each job. To save time and network bandwidth, it would be useful to have the images locally in the CI environment.
There is no need to expose the registry to the world, only the machines provisioned by Duffy would need access. Not sure how to configure that though, https://docs.openshift.com/container-platform/4.6/registry/securing-exposing-registry.html is not very clear about it. (haproxy.router.openshift.io/ip_whitelist might be configurable?)
haproxy.router.openshift.io/ip_whitelist
Anyway, I think it is nice if we can use this OpenShift feature not only for the pods that we start, but also for the bare-metal systems that we consume.
For now, we'll just keep rebuilding the images. When there is a decision to (not) open the registry, we'll consider how to proceed.
Metadata Update from @dkirwan: - Issue assigned to dkirwan
Metadata Update from @dkirwan: - Issue priority set to: None (was: Needs Review) - Issue tagged with: high-trouble, medium-gain
Metadata Update from @siddharthvipul1: - Issue priority set to: Next Meetings
As a (temporary?) workaround, we would like to deploy our own container registry in OpenShift (ceph-csi project). This needs some persistent storage, a PVC of 20GB should be sufficient to get us started.
PV created and bound to a PVC in the ceph-csi project. I'll get back to you tomorrow afternoon once I've had a chance to discuss opening up the image registry for access on Duffy nodes.
Hi @devos we've discussed exposing up the Openshift internal registry outside the cluster, and we've decided that we won't support this use-case at this time. Its best if a solution/workaround is found within the realm of tenant self service for the moment.
I would like to see the processes you put in place around security and maintenance (eg: image pruning), would be good to get documentation/automation to support this, as perhaps other tenants would be interested in this also in this feature.
When you figure out everything, it would be great if you could contribute to our docs here: https://github.com/centosci/ocp4-docs
Metadata Update from @dkirwan: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.