#312 Enabling CI/CD: Experiment with building Atomic Host out of a module

Created 18 days ago by dustymabe
Modified 12 hours ago

This started as an email to the atomic-devel list. Moving into a ticket for further discussion and tracking of experimentation of building Atomic Host out of a module. Summary of bullets from the email:

Here are some of the things we want to be able to do:

  • 1 Have a rigorous definition (including specific versions, buildroot) of
    what goes into an Atomic Host, including dependencies.

  • 2 Triggering the CI/CD pipeline based on a change to definition
    of what goes into Atomic Host.

  • 3 A way to revert a package in the Atomic Host compose to an earlier version.

  • 4 A place to store higher level tests along with rigorous definition
    of Atomic Host, including them being versioned.

  • 5 Landing multiple changes that need to land together to pass testing.

Out of that list we think we require that:

  • 1R The definition of what is composed into Atomic Host artifacts should
    include specific versions of packages, and all dependencies included.

  • 2R The definition of what is composed in an Atomic Host should be stored
    in a git repository so that changes can be detected easily. The
    CI/CD pipeline can be triggered off of changes to this reposiroty.

  • 3R A mechanism to make a future composed Atomic Host artifact, contain
    an earlier (in RPM NVR parlance) version of a package.

  • 4R The high level functional Atomic Host tests should live in the same
    git repository with the rigorous definition of what goes into an Atomic
    Host.

  • 5R A mechanism to tell the CI Pipeline that multiple dist-git repository
    changes (i.e. multiple changing RPMs) should be built and tested together.

Note - @walters also sent an email to a broader audience about modularity and Atomic Host that is worth a read.

@psabata, from modularity team, has volunteered to help us do a proof of concept on this

Edited 18 days ago by dustymabe
18 days ago

Metadata Update from @dustymabe:
- Issue tagged with: ci, host, modularity

13 days ago

Metadata Update from @dustymabe:
- Issue assigned to psabata

There's a lot of bugs to work through. I'm trying a build locally, and hitting a lot of variations of

https://github.com/cockpit-project/cockpit/issues/7510

But I'm a bit confused since some of the ones failing locally appear to be succeeding in koji; maybe newer RPM? One thing I haven't found yet is - the base-runtime etc. module is f26 only, is there a version of the host-and-platform somewhere?

Here's one that's currently very mysterious to me:
https://koji.fedoraproject.org/koji/buildinfo?buildID=955412
It looks like ostree's use of AC_USE_SYSTEM_EXTENSIONS is failing to define _GNU_SOURCE somehow.

There's other ones, like p11-kit fails with:

RPM build errors:
File not found: /builddir/build/BUILDROOT/p11-kit-0.23.7-3.module_55aa6181.x86_64/usr/share/man/man1/trust.1.gz

selinux-policy dies with:

libsepol.policydb_read: policydb version 31 does not match my version range 15-30

libxml2:

RPM build errors:
File not found: /builddir/build/BUILDROOT/libxml2-2.9.4-4.module_55aa6181.x86_64/usr/lib64/python3/site-packages/pycache/py*

etc.

Could you elaborate what exactly are you doing? Are you using mbs-build local for local tests?

The base-runtime module is indeed f26-only; it's being replaced by host and platform in f27. The atomic module depends on bootstrap:master for build, so that's the one mbs-build local will download for you -- beware, it's huge. Oh that and shim:master.

I can't comment on the build failures; I would need to dive into it some more. Sometimes mysterious things happen :/

I've updated some of the packages and made slight changes to the buildroot -- will kick off another build.

Should we track our findings here?

yes please

On another note, looking at the ostree rpm error I see:

error: Bad exit status from /var/tmp/rpm-tmp.Bvv1rE (%build)
    bogus date in %changelog: Sat Aug 25 2013 Colin Walters <walters@verbum.org> - 2013.5-3
    bogus date in %changelog: Sat Aug 25 2013 Colin Walters <walters@verbum.org> - 2013.5-2

are we building with a more strict version of the build tools in MBS vs normal koji?

are we building with a more strict version of the build tools in MBS vs normal koji?

Nope. These warnings are unrelated to the build failing and you have them in your passing builds, too.

Kicked off another build with @tdawson 's fixes. It will take a while to build & fail, of course. Feel free to watch the progress with mbs-build watch 806.

Ok, failures from the last build:

I need to leave early today and I'm traveling tomorrow so I'm not sure I can fix it all before the meeting but I'll try.

I have included fixes for (hopefully) everything listed above and kicked off another build. Watch the progress with mbs-build watch 812.

If it goes well, what's the next steps? Should I be pursuing new compose configuration in Fedora Infra or should OSTree images be created elsewhere?

Login to comment on this ticket.