One thing we can do with this is coherent versioning of the images and released tree commits. Right now the tree commits are versioned e.g. 23.X where X simply increments, but if we did one-commit-per-release we could have the OSTree commits match the image versioning. Where "image versioning" is currently just a date stamp.

The high level rationale I'm filing this now is that rpm-ostree deploy is introduced in F23, which allows a user to deploy a specific (old or new) commit. With that functionality, effort put into server side versioning becomes significantly more useful.

This makes sense to me.

The /testing/ ref though would continue at its current rate - and ideally go even faster. I would be a lot happier if we skipped the manual once-a-day RPM signing and did direct Koji -> /testing/.

What if we had a /devel/ ref that behaved that way, in addition to the signed-RPMs /testing/?

Replying to [comment:2 mattdm]:

What if we had a /devel/ ref that behaved that way, in addition to the signed-RPMs /testing/?

That'd be OK by me.

Another reason to do this is that it would make static delta management easier - we'd have fewer deltas, and people would be more likely to hit them.

We could still of course implement deltas for N-2 -> N and such, but it is subject to combinatorics the more variants one tries to support, which hurts more for once-a-day releases.

Whereas for development streams, the "archive-z2" model is okay enough.

Right now bodhi is responsible for these regular commits, which is decoupled from the 2-week-atomic workflow entirely. Until we have a central releng orchestration framework that will handle $EVERYTHING at whatever interval we desire, we'll have to tweak what we have in place now.

So, during an updates push we could have bodhi query to see if there is a new 2-week-atomic release, and then kick off the stable ostree compose. The testing composes would continue to churn along as they do, and if we want a branch with unsigned testing stuff from koji, then it doesn't really make sense to do that in bodhi. The downside here being there will be a slight lag between the images and the trees, since I'm assuming different people kickoff the twoweek & updates pushes on their own schedules.

For bodhi to query the latest 2-week-atomic release, it could potentially:
* Parse https://pagure.io/mark-atomic-bad/raw/master/f/good-builds.json
* Query datagrepper for the last atomic.twoweek.complete fedmsg. https://apps.fedoraproject.org/datagrepper/raw\?topic\=org.fedoraproject.prod.releng.atomic.twoweek.complete

Alternatively, we could move the stable ostree composes out of bodhi and into the push-two-week-atomic.py script (https://pagure.io/releng/blob/master/f/scripts/push-two-week-atomic.py), and have bodhi only do updates-testing composes.

Your last paragraph seems like the easiest to implement to me - we will just have to remember to watch out for critical security updates.

One additional reason to slow down the ostree commits to 2 weeks - it would allow us to put human-written content in the OSTree commit message, which so far we haven't used. But I could imagine doing so, and then we could display it on the client.

Colin's executive summary: Create a new additional ref:

fedora-atomic/f23/x86_64/batch/docker-host

Allow users to rebase to this who want it.

• Change the two-week-atomic.py script to generate this (new rpm-ostree run, need to sed the json for the new ref)
• then sed the kickstarts to pull from it (but still use a target ref of fedora-atomic/f23/x86_64/docker-host ?)
• Decide on a version numbering scheme...e.g. also use a datestamp? So ostree.version=23.20160106, need to pass this to rpm-ostree

Let's track the devel/ discussion in https://fedorahosted.org/rel-eng/ticket/6350

From an Atomic Host perspective, this is the highest priority ticket for us. Is this likely to be looked at in e.g. the next month?

Apologies for the massive delay on a lot of this, the Docker Layered Image Build has taken priority over my ability to work on this and it's been quite the journey (but initial phase is coming to a close and I'll be able to work on other things).

I will be revisiting this very soon (likely very shortly after Red Hat Summit 2016). I have plans to rewrite the current 2-week release as an Ansible module, this will allow for the release to be more flexible (at least that's the end goal). This is part of the [https://fedoraproject.org/wiki/Changes/ReleaseEngineeringAutomationWorkflowEngine | RelEng Automation Change]. If it becomes apparent very early on (I'm going to give myself a 2 day time limit once I start working on it) that this is going to not be feasible as a near-term goal (within the next month), then I will make the modifications as requested by Walters in Comment #7 to the two-week-release script to satisfy the need while the larger plan gets worked on. Long term the goal is to have this be a fully automated process that we could trigger based on any event that comes across the fedmsg bus that we see fit to trigger this action (CI or otherwise).

I'm open to feedback and look forward to others thoughts on the plan.

A quick update from a conversation Colin and I had last week. To fully clarify and summarize what we'd like to see:

• The delivered/highlighted output is a combination ostree compose and image build(s) based on that compose, done every two weeks

• Mirrors should contain the current image, the current compose and 3 prior composes (1 image and 4 ostree composes in total) - older composes may be pruned at will

• Anything done more frequently should be for dev/test only, need not be mirrored and can be pruned at will

I am willing to give $1000 (one thousand dollars) of my own personal money to someone who demonstrates commitment to implementing this ticket, and following through on related infrastructure in Fedora. Someone who posts or contributes to the upstream ostree list (or atomic-devel@), and codebase, and helps push forward things related to the OSTree side of Atomic Host release engineering, such as static deltas between these updates.

Example bug report type I'd like to see: https://bugzilla.gnome.org/show_bug.cgi?id=765701

Example implementation:

https://wiki.centos.org/SpecialInterestGroup/Atomic/Devel

See the Jenkins jobs for some examples of how the alpha/continuous branching flow is separated including generating separate images.

I forgot to update this, but I have a [https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org/thread/B73P6EDFVKZCMHJ75LBOWUMOJTI34N7N/ proposal out to the rel-eng mailing list] that we're currently hashing out now.

Hey Adam, what is the status of this? I remember discussing the proposal on the mailing list, but then it seems like it trailed off.

@mboddu thinks this work is done. Checking with @dustymabe

Perhaps this is the same ticket as https://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/us/297?kanban-status=146 ?

I consider this ticket to be done as we only update the stable ref when we do official releases. I'll close this.