this will take some coordination/discussion before we implement.

cc @puiterwijk @walters @pbrobinson @jlebon @sinnykumari @mohanboddu

note current rewrite/redirect rules are here: https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/download/files/httpd/dl.fedoraproject.org/rewrite.conf

If this is just on kojipkgs we can use symlinks if needed.

The plan seems fine to me.

We also need to adjust cloudfront? or switch to the other cloudfront endpoint?

If this is just on kojipkgs we can use symlinks if needed.
The plan seems fine to me.
We also need to adjust cloudfront? or switch to the other cloudfront endpoint?

right.. there are some other subsequent changes that we'd need to do (adjusting pungi configs, etc). We can try to enumerate those once the plan sounds good to everyone.

Thanks Dusty for initiating this, good to have a generic name of ostree repo.
+1 from me to make this change and follow-up changes wherever needed.

@puiterwijk - can we get your input on this?

This looks like an okay plan, but I'm not sure we want the kojipkgs.fp.o/atomic -> kojipkgs.fp.o/ostree redirect: nobody should be using kojipkgs.fp.o directly anymore, and this would be a way to find the people who are still pointing to kojipkgs directly.

unfortunately we'll need to keep kojipkgs.fp.o/atomic/repo working for a while. 😭 we discovered as we were investigating the mirroring options that our 29 SB installs are pointed at kojipkgs.fp.o/atomic/repo and not dl.fp.o/atomic/repo like we thought they were. We should be able to resolve this when we move to f30 and include it in the "upgrade/rebase instructions" to switch the remote.

Considering atomic host is going away I'd like to propose we move away from using atomic in the URLs we are using. This will take some brainstorming but here is what I propose:

delete koji/compose/ostree directory and contents (old stuff)
mv koji/compose/atomic koji/compose/ostree

Do we need anything other than koji/compose/atomic/repo/ content ? If not, we can just have koji/compose/atomic/repo/ moved into koji/compose/ostree/ and archive remaining data from
koji/compose/atomic/ ?

basically we are replacing atomic in the URL with ostree, which should handle atomic host, silverblue, iot, and coreos.

Just noticed that iot has its own separate ostree repo at https://kojipkgs.fedoraproject.org/compose/iot/repo/ . I couldn't find out if it gets synced somewhere from /compose/iot/repo/ for production use. It's definitely does not get synced to koji/atomic/repo/ .

Wanted to bring this up so that we make changes accordingly.

Do we need anything other than koji/compose/atomic/repo/ content ? If not, we can just have koji/compose/atomic/repo/ moved into koji/compose/ostree/ and archive remaining data from
koji/compose/atomic/ ?

From what I understand we only need koji/compose/atomic/repo/ content. The rest of the files under koji/compose/atomic/ are old and can be archived.

Just noticed that iot has its own separate ostree repo at https://kojipkgs.fedoraproject.org/compose/iot/repo/ . I couldn't find out if it gets synced somewhere from /compose/iot/repo/ for production use. It's definitely does not get synced to koji/atomic/repo/ .

Until this point IoT has been handled separately. This plan does not reconcile that, but I believe in the future it wouldn't be hard to bring IoT under the unified repo structure.

Do we need anything other than koji/compose/atomic/repo/ content ? If not, we can just have koji/compose/atomic/repo/ moved into koji/compose/ostree/ and archive remaining data from
koji/compose/atomic/ ?

From what I understand we only need koji/compose/atomic/repo/ content. The rest of the files under koji/compose/atomic/ are old and can be archived.

Just noticed that iot has its own separate ostree repo at https://kojipkgs.fedoraproject.org/compose/iot/repo/ . I couldn't find out if it gets synced somewhere from /compose/iot/repo/ for production use. It's definitely does not get synced to koji/atomic/repo/ .

Until this point IoT has been handled separately. This plan does not reconcile that, but I believe in the future it wouldn't be hard to bring IoT under the unified repo structure.

Thanks Dusty! sounds good then

@walters @jlebon @mohanboddu Before we start working on the changes, would like to know does the requested change in this ticket looks ok to you ?

It sounds good to me; only comment I'd add is that ideally it's easy to undo if something goes wrong. I think that'd just be rolling back the ansible for the redirects?

Feels like we may want to stage things; do the redirects or the repo move first instead of both at once?

WFM too! I'm not very familiar with how everything is wired up, but I like the general idea of removing atomic from the URLs.

ok patrick set us up a cloudfront CDN and a top level redirect known as https://ostree.fedoraproject.org. We will be using this in the future with a config like:

[remote "fedora"]
url=https://ostree.fedoraproject.org
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-primary
contenturl=mirrorlist=https://ostree.fedoraproject.org/mirrorlist

Given that result we will remove the dl.fp.o/ostree/repo -> kojipkgs.fp.o/ostree/repo redirect from this plan as we will use https://ostree.fedoraproject.org instead.

This seems like an okay plan although it requires some changes to pungi configs and nightly scripts as @dustymabe mentioned.

Thanks everyone for the input! We will now identify/work on list of changes to be done and then co-ordinate with infra team to make this happen.

Created an etherpad listing possible changes which will be needed - https://public.etherpad-mozilla.org/p/fedora_atomic_ostree_repo_rename . This is to co-ordinate changes we will be doing during ostree repo move.

Commit f1741cff relates to this ticket

We have successful Fedora-29-updates compose with updated ostree repo and corresponding changes in place - https://kojipkgs.fedoraproject.org/compose/updates/Fedora-29-updates-20190208.0/
FAH artifacts from above compose has new ostree config containing contneturl. Also, existing Silverblue and FAH updates fine with current setting.
Looks to me that ostree repo migration has been done successfully.
Thank you all!

Maybe we can close this issue as fixed.

The only thing I think that hasn't been done here is that we didn't clean up things that we didn't need any longer:

• The only directory under koji/compose/ostree should be repo.
• The koji/atomic directory should be archived.

Can we get all of those contents moved somewhere else (with a $date) and open a ticket to have them deleted in the next few months?

We can just reopen this one to track the things that @dustymabe mentioned above.

I can do these things.

What do you want me to do with the other directories under koji/compose/ostree? Just delete? Or archive for a while?

and how long to do we want to archive them for?

I can do these things.

❤️

What do you want me to do with the other directories under koji/compose/ostree? Just delete? Or archive for a while?

archive just for safety

and how long to do we want to archive them for?

2 months would be safe - let's open a new ticket to tell us to delete it.

Done. No need for a ticket, I have set a task to remove this in 2 months. I also moved them to a dir called "archive-until-2019-04-12" in case we are looking later.

:slot_machine:

Thanks @kevin ! Is the archived dir accessible publicly or kept private? Don't see it at https://kojipkgs.fedoraproject.org/compose/

I put it on the top level, which is not publicly served. Do we need it public?

Do we need it public?

I personally don't think so.

I put it on the top level, which is not publicly served. Do we need it public?

Not needed, I asked it to make sure that I am not missing anything. Thanks!

FYI, I am going to remove this later today. Shout if we need it for some reason.