We will look into what it takes to bundle moby-engine, containerd and the related packages due to their unmaintainable dependency sprawl. This will have to be done gradually and carefully to avoid breaking things.

These are the packages that (recursively) depend on golang-github-docker-devel or compat-golang-github-moby-devel or containerd-devel and will also need to be bundled if we bundle docker and containerd.

$ sudo dnf repoquery --repo=rawhide --recursive --whatrequires golang-github-docker --whatrequires compat-golang-github-moby-devel --whatrequires containerd-devel --qf="%{source_name}" | grep -v none |sort | uniq

cadvisor
containerd
doctl
etcd
golang-github-beego
golang-github-bketelsen-crypt
golang-github-cockroachdb-cockroach
golang-github-containerd-aufs
golang-github-containerd-cri
golang-github-containerd-fuse-overlayfs-snapshotter
golang-github-containerd-imgcrypt
golang-github-containerd-nri
golang-github-containerd-stargz-snapshotter
golang-github-containerd-zfs
golang-github-deislabs-oras
golang-github-docker
golang-github-docker-cli
golang-github-docker-libkv
golang-github-docker-libnetwork
golang-github-docker-licensing
golang-github-docker-slim
golang-github-docker-swarmkit
golang-github-fsouza-dockerclient
golang-github-gohugoio-testmodbuilder
golang-github-google-certificate-transparency
golang-github-google-containerregistry
golang-github-google-monologue
golang-github-google-trillian
golang-github-istio-viper
golang-github-ory-dockertest
golang-github-prometheus
golang-github-quay-clair-4
golang-github-quay-claircore
golang-github-sagikazarmark-crypt
golang-github-skynetservices-skydns
golang-github-spf13-viper
golang-github-tonistiigi-fsutil
golang-github-tonistiigi-opentelemetry-contrib
golang-github-xordataexchange-crypt
golang-gocloud
golang-gvisor
golang-helm-3
golang-istio-pkg
golang-k8s-apiextensions-apiserver
golang-k8s-apiserver
golang-k8s-cloud-provider
golang-k8s-component-base
golang-k8s-controller-manager
golang-k8s-kube-aggregator
golang-k8s-kube-controller-manager
golang-k8s-kubectl
golang-k8s-kubelet
golang-k8s-kube-proxy
golang-k8s-kubernetes
golang-k8s-kube-scheduler
golang-k8s-legacy-cloud-providers
golang-k8s-pod-security-admission
golang-k8s-sample-apiserver
golang-opentelemetry-contrib
golang-opentelemetry-contrib-0.20
golang-sigs-k8s-apiserver-network-proxy
golang-sigs-k8s-application
golang-sigs-k8s-aws-iam-authenticator
golang-sigs-k8s-controller-runtime
golang-vitess
hugo

Do other packages that are deps of those to be bundled or removed?

Example, doctl depends on golang-github-digitalocean-godo. If doctl is bundled, golang-github-digitalocean-godo should be removed, that one is easy. But what happens if golang-github-digitalocean-godo is required by other package and doctl is bundled?

But what happens if golang-github-digitalocean-godo is required by other package and doctl is bundled?

Then, we'd have to keep it around, even if it's not used by doctl.

I just checked. It looks like the only other thing that depends on golang-github-digitalocean-godo is golang-github-prometheus, which would also have to be removed if we went through with this. In that case, you'd be able to get rid of golang-github-digitalocean-godo. There will likely be other packages that can be removed if we remove the packages from above.

Would the go-sig like to vote on whether bundling these packages is the right way forward? If there is agreement to do so, I can help coordinate.

I'm not against bundling, but would like to understand better why these packages suffer from "dependency sprawl".

Is it that they've too many non packaged dependencies? Is that they have strong dependencies that make other packages fail? Is it that deps don't follow semver?

There are many other packages that could be easily imported to Fedora if bundling would be enabled (terraform, packer, tailscale, sigstore, ...)*. That's why I want to understand it better, as we may need to change the documentation to add some reasoning or justification.

• I've been working locally on some of those packages and deps, but if bundling packages due to complex dependency trees is OK, some of them could be added to Fedora easily.

I've been working locally on some of those packages and deps, but if bundling packages due to complex dependency trees is OK, some of them could be added to Fedora easily.

From the Golang Packaging Guidelines:

===  Bundled or unbundled

At the moment golang projects packaged in Fedora SHOULD be unbundled by
default. It means projects are built from dependencies packaged in Fedora.

For some project it can be reasonable to build from bundled dependencies.
Every bundling needs a proper justification.

This leaves some room for interpretation, but I'd say more than 10 or maybe 15 new dependencies justifies bundling.

I'm not against bundling, but would like to understand better why these packages suffer from "dependency sprawl".

Is it that they've too many non packaged dependencies? Is that they have strong dependencies that make other packages fail? Is it that deps don't follow semver?

In order to fix golang-github-docker, golang-github-docker-cli, and golang-github-moby-buildkit's FTBFS which is caused by golang-containerd-stargz-snapshotter needing to be updated[^1], we'd need to package 25+ new dependencies.

[^1]: Technically, they're currently broken, because another packager pushed out a broken update for one of the dependencies, but that is not the main problem.

Note that the docker binary and the dockerd binary are already built from bundled deps (part of the moby-engine package, but golang-github-docker and golang-github-docker-cli exist so that other packages can depend on them. containerd and docker-proxy are not bundled. I don't think it's possible to build dockerd or docker from the deps we have in Fedora due to the dependency versions, but this split happened before I started maintaining the package. @eclipseo would know better.

Thanks for the extra info @gotmax23

I don't want to hijack the ticket, but I've some questions about bundling:

• Should each new bundled package first be discussed for acceptance or is it enough a maintainer preferring that way?

• Should each bundled package have a ticket, thread link or info that explains why it is being bundled?

• Should the maintainer try to unbundle the package as a mid-term goal?

Should each new bundled package first be discussed for acceptance or is it enough a maintainer preferring that way?

No, it does not really need to be discussed. I'm bringing this up here, because I don't own all of the affected packages and this has some larger ramifications. The package reviewer might encourage you to unbundle the package if possible, however.

Should each bundled package have a ticket, thread link or info that explains why it is being bundled?

No, but I would suggest leaving a comment in the specfile explaining the situation.

Should the maintainer try to unbundle the package as a mid-term goal?

I would recommend doing so, but that's not a requirement.

You also must create virtual provides denoting which dependency versions are bundled. See https://src.fedoraproject.org/rpms/cri-tools/pull-request/1#_2__90 and moby-engine for an example. In moby-engine, I keep the Provides in a separate source file. The specfile has Source5: provides.spec.inc and then %include %{SOURCE5} at the end of the preamble to "import" the contents of the second file into the specfile. The second approach makes the specfile much easier to read for packages that have a lot of dependencies, but it might confuse people or make the package review more difficult; rpm cannot parse the specfile unless both files are available.

It would be helpful if concurrently some updated documentation could be developed for the packaging guidelines specific to bundling - both to provide guidance and to try to create some sort of standard.

Perhaps investigating integrating bundling into go2rpm would be a good approach for improving our long-term management (provided some vigilance is maintained about not just bundling everything).