#6984 create a unified ostree repo for fedora 27 and beyond
Closed: Fixed 4 years ago Opened 4 years ago by dustymabe.

Within the atomic working group we believe there are benefits to having a unified ostree repository (see discussion). Since we have not yet created an ostree repo for f27 I think now would be a good time for us to create the unified repo and let f27 publish into it. Then we move rawhide over to use that repo after we verify it is working.

I propose the repo go under https://kojipkgs.fedoraproject.org/atomic/host/. This would allow for us to also have another unified repo https://kojipkgs.fedoraproject.org/atomic/workstation/ as well.


Metadata Update from @mohanboddu:
- Issue tagged with: meeting

4 years ago

Also related to this https://github.com/ostreedev/ostree/pull/1033 will make it easy to create fedora/x86_64/atomic-host.

The whole reason we do not yet have a unified repo is due to a lack of tooling, experience and knowledge in how to manage it properly, I am all for doing it, however we will need a lot of assistance in order to do so.

The whole reason we do not yet have a unified repo is due to a lack of tooling, experience and knowledge in how to manage it properly, I am all for doing it, however we will need a lot of assistance in order to do so.

I'd like to help with closing the skills gap: https://pagure.io/releng/issue/6994

With me and @puiterwijk, I think we should have it covered for now. With help coming from @walters and @jlebon.

Note that the Atomic pieces of the fedora 27 compose will fail until we have an ostree repo. The sooner we resolve this ticket the quicker we can resolve that.

I had promised to put my thoughts on this in the ticket, sorry for the delay.

I think that we should first make clear on how we can clear old commits (from EOL'd Fedora releases etc) from such a repo, and then also get garbage collection to make sure that they get removed.
We should then probably also have tooling that can do this automatically, and archives older commits.

We also would need an idea of how this all works for releng.

Together with this, I think we should get the list of people that know ostree up from the current set.
We really should be getting clear SOPs and documentation on how to do all the ostree/atomic operations.

Big picture, the ostree model is really quite similar to registry.fedoraproject.org. The way we have the ostree repos today is a lot like if there was

  • registry23.fedoraproject.org
  • registry24.fedoraproject.org
  • registry25.fedoraproject.org

which would be clearly weird from the Docker/container user perspective. It's equally weird from the ostree design.

The point I'm trying to make here is that anything that applies at a conceptual level to registry.fedoraproject.org maps equally well to an ostree repo. (The tooling is different but we'll be working on that)

Well, in your analogy we have registry.fedoraproject.org/fedora23:latest, registry.fedoraproject.org/fedora24:latest and registry.fedoraproject.org/fedora25:latest for atomic.
And I am not saying we shouldn't get that to /fedora:23 etc (so a unified repository), in the end I'm in favor of that idea.

However, I do think that we need to get all of the parts figured out and documented (as said, archiving and garbage collection e.g.) and SOPs.
It's been reasonably often recently that we got literal commands from Dusty that I then ran instead of having a document where he can point to "You forgot to follow this part of the SOP, please do so".

Hmm? We don't have registry.fedoraproject.org/fedora23:latest.

# docker pull registry.fedoraproject.org/fedora26:latest
Trying to pull repository registry.fedoraproject.org/fedora26 ... 
manifest for registry.fedoraproject.org/fedora26:latest not found
# docker pull registry.fedoraproject.org/fedora:26
Trying to pull repository registry.fedoraproject.org/fedora ... 
sha256:311f2acf4b42a6c1c00cbffe902da9d34a684f127e39cd6b0558a18785390c3f: Pulling from registry.fedoraproject.org/fedora
727707040437: Downloading ...

And the :latest tag refers to the current major, which is all how docker was intended to work. Right?

@puiterwijk
I think that we should first make clear on how we can clear old commits (from EOL'd Fedora releases etc) from such a repo, and then also get garbage collection to make sure that they get removed.
We should then probably also have tooling that can do this automatically, and archives older commits.

I have identified needs for better pruning tooling around this as well. I discussed this with @walters earlier in the week and I opened this RFE to get more fine grained pruning.

@puiterwijk
We also would need an idea of how this all works for releng.
Together with this, I think we should get the list of people that know ostree up from the current set.
We really should be getting clear SOPs and documentation on how to do all the ostree/atomic operations.

I agree

@puiterwijk
However, I do think that we need to get all of the parts figured out and documented (as said, archiving and garbage collection e.g.) and SOPs.

I agree. I didn't realize how much of a gap existed in releng for having this information written down. Can we identify what SOPs we think we need?

1 - SOP for creating new ostree repo (should not be as needed after we have a unified repo
2 - SOP for archiving old release of Fedora (my preference would be that we create an archive repo to put the last commit from that release in)
3 - I don't think we need an SOP for pruning since we should have some automated process doing it.
3 - ??

This has been done now. We moved to a unified repo this morning. Here is an email with a summary of the changes that were made: https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org/thread/KLN5L33BIR3ZEHC5RIG4NXGO7LT6HBXJ/

This has been done now. We moved to a unified repo this morning. Here is an email with a summary of the changes that were made: https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org/thread/KLN5L33BIR3ZEHC5RIG4NXGO7LT6HBXJ/

Metadata Update from @dustymabe:
- Issue untagged with: meeting
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.

Metadata