#360 Decide strategy for including container runtimes in Atomic Host
Closed 3 years ago Opened 3 years ago by jberkus.

Currently, we ship docker in the base image for Fedora Atomic Host. With the release of CRI-O, as well as the issues around docker versions, one possible solution is to ship only runc with the base Atomic Host, and make container runtimes installable as system containers. If we don't move docker to a system container, there's a strong argument to also ship CRI-O as part of the base image.

Arguments in favor of System Containers for CRs:

  • minimizes the base system image
  • helps deal with docker/containerd versioning issues
  • doesn't require users to ship container runtimes they don't want
  • consistent with our general modularity approach
  • puts CRI-O/docker on a separate upgrade cycle, which can help around kube/openshift updates
  • allows community to add other runtimes if they want (e.g. Rkt)

Arguments against using System Containers for CRs:

  • extra step (or two) for new user startup
  • total disk size for atomic + cri-o + docker will actually be larger
  • not sure if all things docker work as a system container
  • puts docker/CRI-O on a separate upgrade cycle, which could break around kernel updates

Discuss!


One bifurcation I see here is that larger scale systems do want decoupled host upgrades ("I want to yum update/rpm-ostree upgrade without upgrading Kubernetes/OpenShift").

But for smaller scale systems ("I have a home server") decoupling the container runtime feels like a lot of overhead. But eh, we can probably fix that.

I think my vote here is to focus on system containers for Kube/OpenShift first... which actually doesn't answer the question.

If we're going to default to containerized cri-o, I think we need to fix things upstream. Right now the upstream docs follow the (IMO anti-pattern) of installing build dependencies on a host and doing a sudo make install. https://github.com/kubernetes-incubator/cri-o

With the latest FAH rpm-ostree install cri-o seems to work fine FWIW.

I've been working on a writeup using the crio system container from https://github.com/projectatomic/atomic-system-containers.

Metadata Update from @dustymabe:
- Issue tagged with: meeting

3 years ago

I'd like to see us use System Containers for runtimes. While I agree that's different that what users currently expect it's due to us giving the expectation that there is a runtime and it is docker. Having runtime be something that is installed as a System Container as one of the first steps (or maybe part of first boot via parameter) seems sane to me.

We use atomic hosts in two different ways in production and our users follow two different usage patterns.

  1. as a kubernetes container runtime
  2. swarm cluster or standalone docker host

Pattern A
The user has short living clusters (of atomic hosts) and recreates often (compute intensive workloads) [These users expect to have the latest stable version when they create a new cluster and they don't necessarily care about upgrading. However, upgrading is a nice-to-have feature for them.]

Pattern B
The user has long running clusters for production services [these users want a relatively new version of docker (6 months old), upgrading is very important for them]

As service provider who offer atomic hosts I would be happy with running the runtimes in system containers. So +1 on all arguments in favor. Regarding the arguments against: Two steps to install a container runtime is reasonable. I think the space load is compensated by having more options and I don't think it is a big problem for usage on public/private clouds. The last two can be tricky but are also giving flexibility. However it would be nice to be able to select the version of the runtime eg atomic install cri-o:1.x or atomic install docker:17.09 even a newer release is available. It is more clear what you get instead of atomic install docker:latest.

Finally, having a docke-ce system container would be really nice to have.

When are we expecting the survey to close?

Earlier in the week @jberkus noted he was working on this and should have results late this week or early next.

So, we discussed this at the Fedora Atomic WG meeting:

https://meetbot.fedoraproject.org/fedora-meeting-1/2017-12-20/fedora_atomic_wg.2017-12-20-16.31.log.html

Our decision was to NOT remove docker from the base OStree in the Fedora 27/28 timeline, but to instead:

  • document and support superceding that version of docker with others using system containers, and
  • start an initiative around creating a "minimal" image for IoT and other use cases.

closing this based on result in previous comment. will revisit when necessary in the future.

Metadata Update from @dustymabe:
- Issue untagged with: meeting
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata