#10399 tracker issue for ostree native containers
Opened 6 months ago by walters. Modified 5 days ago

This is the releng tracker issue for https://fedoraproject.org/wiki/Changes/OstreeNativeContainer

There are multiple phases and goals. The first phase is shipping quay.io/fedora/coreos:$stream - this relates to https://github.com/coreos/fedora-coreos-pipeline/issues/359

We could also in parallel investigate other (rpm-)ostree based systems such as IoT and Silverblue etc. but particularly large desktop images will really depend on useful chunking and/or delta containers, which is ongoing work.

Ideally for Fedora 36 we land chunked images and ship quay.io/fedora/coreos and quay.io/fedora/silverblue etc. However, we may also focus on FCOS first.

The initial goal of this issue is really just awareness that this is coming.

It may take multiple Fedora major releases to fully transition, but the end state would be that all editions/spins have switched over to container images on the wire, and we would deprecate (mark read-only) the production Fedora ostree repository and not write new content there.


Metadata Update from @zlopez:
- Issue tagged with: change-ack, changes

6 months ago

Would the shipping be part of the FCOS release pipeline ? Then the ask here is really to get token and permissions to push to the fedora organization in quay.io ?

Would you also like to have the images on registry.fedoraproject.org ?

Would the shipping be part of the FCOS release pipeline ? Then the ask here is really to get token and permissions to push to the fedora organization in quay.io ?

That seems like the most obvious path, although it could also be pull based, i.e. the FCOS pipeline pushes to something like quay.io/fedora-coreos-testing/fcos:stable and there's something with credentials that watches for changes and mirrors from there to quay.io/fedora/coreos:stable.

Would you also like to have the images on registry.fedoraproject.org ?

I personally have no strong opinion on that; it seems like it should behave how other containers behave. If most of them are mirrored then let's mirror.

But this issue is about more than just FCOS note, because other things besides FCOS use rpm-ostree and would probably (maybe?) want this.

Would the shipping be part of the FCOS release pipeline ? Then the ask here is really to get token and permissions to push to the fedora organization in quay.io ?

That seems like the most obvious path, although it could also be pull based, i.e. the FCOS pipeline pushes to something like quay.io/fedora-coreos-testing/fcos:stable and there's something with credentials that watches for changes and mirrors from there to quay.io/fedora/coreos:stable.

We have bodhi currently doing something similar for layered images (OSBS builds and push the images to candidate-registry.fp.o and bodhi pushes the images to registry.fp.o when the update goes stable). Would the os container updates go through bodhi ?

Would you also like to have the images on registry.fedoraproject.org ?

I personally have no strong opinion on that; it seems like it should behave how other containers behave. If most of them are mirrored then let's mirror.

But this issue is about more than just FCOS note, because other things besides FCOS use rpm-ostree and would probably (maybe?) want this.

Yes, there has been some discussion to also move registry.fp.o to be hosted on quay.io (https://pagure.io/fedora-infrastructure/issue/10373) so It don't think it is super important to sort this out now.

In general, our compose artifacts should exist on our infrastructure, even if they're mirrored out externally. For all intents and purposes, that means things should be on registry.fp.o and mirrored to quay.io.

@walters As per the bz you said its almost ready, do you need anything from us? Also, what are your plans for building the images? As in like where are you going to build them? In koji?

At the current time I am focusing on driving this from the Fedora CoreOS side, where we have an integrated, container-native build and test flow. On this topic for example, https://github.com/coreos/fedora-coreos-pipeline/pull/457 merged.

I would love to pursue actually moving quay.io/coreos-assembler/fcos:testing-devel to quay.io/fedora/coreos:testing-devel. That seems like something that could be split out as a separate ticket? I think it'd literally just be "get quay.io robot account with permissions to push there from Fedora infra, change FCOS pipeline to use it".

For non-FCOS editions, anyone else can feel free to pick up those. I do want to emphasize though, it is today a first class operation to take an ostree commit and "encapsulate" it in a container image. See: https://coreos.github.io/rpm-ostree/container/#creating-base-images

So either the Koji-ostree integration could gain a configuration option to run that command, output a container image, and push it to a registry, or a separate controller process could run and watch the repo and synchronize the container. (Hmm, I could try to make this easier to do in an idempotent way in our tools)

I think for the container base images, it would be great if koji-ostree integration could gain that feature, that way we can use pungi to generate these images nightly and push it to the registries.

This is what we do currently for rpm based container base images.

I think for the container base images, it would be great if koji-ostree integration could gain that feature, that way we can use pungi to generate these images nightly and push it to the registries.

OCI container images are just layers of tarballs wrapped with JSON. One thing to bear in mind is that this process of "encapsulating" an ostree commit into an OCI container is hence basically "convert ostree commit into tar format and wrap it with JSON". It's not actually a "build" in any real sense (we'd never run gcc, we don't even run any scripts or solve any dependencies). It's just changing one data format into another.

Yes, for editions using koji-ostree to generate ostree commits, it could make sense to encapsulate the container as part of the same execution flow. But, I think it could also make sense to have it be a separate controller.

This is what we do currently for rpm based container base images.

I'd prefer to say e.g. "non-ostree" or "traditional" here. The container images we're generating using rpm-ostree definitely involve RPM. It just doesn't only involve RPM - it also involves ostree. Another way to say this is, these images aren't "ostree based" - ostree isn't the center of the universe. Neither is rpm. It's a hybrid system.

I raised this issue here: https://github.com/fedora-silverblue/issue-tracker/issues/275#issuecomment-1124160730

But it seems to have great overlap with the discussions here. With that being said if there's any way I can help I would be happy to.

It sounds like the first step (if in agreeance) is to get koji-ostree to push containers while it already generates ostree commits

Login to comment on this ticket.

Metadata