#6125 single Atomic Host ostree repository
Closed: Fixed 2 years ago Opened 5 years ago by walters.

The way OSTree is designed, a repository can contain multiple branches. It's natural to have multiple states represented in a single repository, a good example being:

fedora-atomic/f21/x86_64/docker-host
fedora-atomic/f21/x86_64/updates-testing/docker-host

However, it's more convenient from the command line to go between multiple releases if they're part of the same repository, one can just do e.g.:

rpm-ostree rebase fedora-atomic/f22/x86_64/docker-host

If there is one repository per major release, it means that the remote name needs to be versioned, which is not the end of the world - it's just more awkward.

Something like:

diff --git a/scripts/buildbranched b/scripts/buildbranched
index 19b9363..157553f 100755
--- a/scripts/buildbranched
+++ b/scripts/buildbranched
@@ -28,7 +28,7 @@ SUBJECT='F-'$BRANCHED' Branched report: '$DATE' changes'
FROM="Fedora Branched Report rawhide@fedoraproject.org"
RSYNCPREFIX="sudo -u ftpsync"
ATOMIC=$(mktemp -d /tmp/atomici.${DIST}.$DATE.XXXX)
-ATOMICREPO="/mnt/koji/mash/atomic/$BRANCHED/"
-ATOMICDEST="/pub/fedora/linux/atomic/$BRANCHED/"
+ATOMICREPO="/mnt/koji/mash/atomic/repo/"
+ATOMICDEST="/pub/fedora/linux/atomic/repo/"

source ./scripts/build-functions.sh

Seems like it would start this, but it would need some compatibility code.


This is not something we can realistically do. As it is we do not manage things is a suitable way. we make a different tree for releases that we do not currently publish publically. It could be that we just do not have a proper understaing of the tooling, or it could be that teh tooling is lacking. we need ways to archive releases, we need to be able to prune and add in trees. The rawhide tree for instance has done nothing but grow since it was incepted. we really only need to keep the last two weeks of rawhide. with things setup as tehy are today, when we archive Fedora 21 we can hardlink the 21 atomic tree into the archive location and two weeks later remove it from the main mirror.

So what is our tooling to trim repos, tag repos, merge in different trees?

For pruning rawhide, see e.g.:

https://github.com/projectatomic/rpm-ostree-toolbox/commit/8ffdbb683cec93c983d88d499cbfc042c30639de

Writing this tooling really needs someone has the bandwidth to spend time on it consistently. I can help bootstrap it.

I think a path forward is to decide whether this library lives in rpm-ostree-toolbox (which is a big bag of stuff), or a new library is created. Some bits are in Luke's fedmsg-atomic-composer that could be share as well.

As far as archiving content between repositories, that's also quite possible to script.

However, if we want to simplify things significantly, we could pretty easily set things up so that only the last two commits to each release (including rawhide) are mirrored. I think it'd still make sense to keep the commits in an "archival" server for a more significant amount of time.

That would allow developers/testers to perform binary bisection, without impacting the main mirrors.

A patch to enable the composition of the fedora-atomic/f21/x86_64/updates-testing/docker-host OSTree branch.
0001-bodhi-Compose-f21-updates-testing-atomic-ostrees.patch

Replying to [comment:2 ausil]:

So what is our tooling to trim repos, tag repos, merge in different trees?

Tagging/merging trees is already solved by OSTree itself, right? Since we can just point rpm-ostree at the same repo with a treefile with a versioned ref, and everything gets "tagged" as expected. So then it's just a matter of pruning (and potentially archiving a repo with all commits?), which if it were exposed via cli would make things easy to integrate into the existing build{rawhide,branched} scripts.

Anything blocking Luke's patch?

I have applied and pushed it

One thing to note here is that with current OSTree, having separate repositories means that "static deltas" won't be easily available between them.

We may work to address that with future enhancements. But just like with git, the tooling is most strongly designed for working inside one repository.

Use case: Split off an archived branch (e.g. fedora 20 is EOL)

Let's say that we have a "production" repository an an "archive" repository. We want to move fedora-atomic/f20/x86_64/docker-host from production to archive.

{{{

Copy all of the history of the f20 branch from production to archive

ostree --repo=/path/to/archive pull-local --depth=-1 /path/to/production fedora-atomic/f20/x86_64/docker-host

Delete the branch from production

ostree --repo=/path/to/production refs --delete fedora-atomic/f20/x86_64/docker-host

Prune leftover objects from production

ostree --repo=path/to/production prune --refs-only

}}}

Metadata Update from @walters:
- Issue set to the milestone: Fedora 22 Beta

3 years ago

Metadata Update from @syeghiay:
- Issue close_status updated to: None

2 years ago

Metadata Update from @dustymabe:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata