#1878 Write sync script for centos-10 using reposync
Merged 2 months ago by kevin. Opened 4 months ago by jrichardson.
fedora-infra/ jrichardson/ansible centos_10_sync  into  main

@@ -0,0 +1,21 @@ 

+ #!/bin/sh

+ 

+ #Sync using reposync instead of rsync until available

+ SERVER=composes.stream.centos.org

+ TOPDIR=stream-10/production/latest-CentOS-Stream/compose

+  

+ ## mirror CentOS-10

+  

+ DESTDIR=/mnt/fedora/app/fi-repo/centos/centos-10

+  

+ mkdir -p ${DESTDIR}

+  

+ for basearch in aarch64 ppc64le s390x x86_64

+ do

+     for repo in BaseOS AppStream CRB

+     do

+             mkdir -p ${DESTDIR}/${repo}

+         cd ${DESTDIR}

+         reposync --download-metadata --disablerepo=* --repofrompath=$repo,https://$SERVER/$TOPDIR/$repo/$basearch/os --enablerepo=$repo

+     done

+ done 

\ No newline at end of file

rebased onto 1fe5e31c4657fdb574acab3d1565ea7d84526eda

4 months ago

rebased onto 1fe5e31c4657fdb574acab3d1565ea7d84526eda

4 months ago

rebased onto 344e4dab73683334672efad8ae1691ba361d8f58

4 months ago

rebased onto 344e4dab73683334672efad8ae1691ba361d8f58

4 months ago

The https:// part is in both the SERVER variable, and down below in the --repofrompath argument. That will result in it being doubled up, i.e. https://https://composes.stream.centos.org. Removing it from the SERVER variable would match the centos-9s-sync script.

I think we should just use a directory of centos-10 (drop the -stream suffix) for simplicity.

A couple of points on these for loops:

  • There is no i386 repo to sync, so we can remove that.
  • As-is this won't work because the i variable will get overwritten in the inner loop. You have to use a different variable like j. I would recommend using more descriptive variables, like basearch and repo.
  • It's unnecessary to use the echo command substitution because you can just pass the strings to the for loop, e.g. for repo in BaseOS AppStream CRB.

We will need to reference the basearch and repo directories separately in the URL. If we set those variables in the for loops as described above, the last two flags should be --repofrompath=$repo,https://$SERVER/$TOPDIR/$repo/$basearch/os/ --enablerepo=$repo.

rebased onto 74ade68f2b57869259b779f0de8467b5f85263cb

4 months ago

rebased onto 91a7e7f4e13f2d4272a799b4573bf7da0b8f8a88

4 months ago

rebased onto 91a7e7f4e13f2d4272a799b4573bf7da0b8f8a88

4 months ago

rebased onto 3ee83ac456f16116e058c8fa36dcc9edb4aa3b5f

3 months ago

rebased onto 3ee83ac456f16116e058c8fa36dcc9edb4aa3b5f

3 months ago

This has a small syntax error I missed in my first review of this. It's missing the do.

Similar to the missing do in the outer for loop, the inner for loop is missing a done.

rebased onto 0a6054a0bbe52eb06ebdf340929ed8223485d343

3 months ago

rebased onto d249a6e37d8ed399cd9811535705bdf854f682b7

3 months ago

rebased onto d249a6e37d8ed399cd9811535705bdf854f682b7

3 months ago

LGTM, thanks for making those adjustments.

rebased onto aa005b6a3576feadf243e4aaf2c81d925a6972da

3 months ago

rebased onto aa005b6a3576feadf243e4aaf2c81d925a6972da

3 months ago

This looks just fine... I do wonder about dropping the -stream name tho. Won't that confuse people and make them think this is some kind of weird centos linux 10?
We do sync centos linux 8 without the stream name (and with for stream 8), so dropping the stream seems confusing to me.

Thoughts?

I do wonder about dropping the -stream name tho. Won't that confuse people and make them think this is some kind of weird centos linux 10?

That was a change I asked @jrichardson to make. Since there is only one distro from the CentOS project with version number 10, I don't think it would be confusing. Anyone looking this closely at the infrastructure implementation details will know that the official distro name is still "CentOS Stream 10". I regularly see people online refer to "CentOS 9" as shorthand for "CentOS Stream 9", because there is only one distro from the CentOS Project with version number 9.

We do sync centos linux 8 without the stream name (and with for stream 8), so dropping the stream seems confusing to me.

We don't sync CentOS Linux 8 anymore. I could see the confusion if we were trying to repurpose the centos-8 directory for CentOS Stream 8, but that's not being suggested. As an aside, if we were going to be 100% consistent, the centos-8 sync directory should have been named centos-linux-8.

Really what you've uncovered here is me inching closer to my secret plan to eventually drop the Stream suffix officially, similar to how Fedora dropped it's Core suffix. The whole reason we came up with the Stream suffix is because we needed to differentiate between two variants of version 8, the classic downstream rebuild and the new upstream model that can accept contributions. That need no longer exists, so longer term my goal is to get rid of the useless suffix. For now I'm happy to just encode the name in fewer places when it isn't necessary.

Since it's just the local directory name, I don't think it makes too much difference either way, but dropping the -stream would be my preference.

rebased onto 768b6b0

2 months ago

rebased onto 768b6b0

2 months ago

Pull-Request has been merged by kevin

2 months ago

So, I set it up and ran it... and it ran ok, but didn't do what was intended.

centos-10:
AppStream
BaseOS
CRB

centos-10/AppStream:
Packages
repodata

centos-10/AppStream/Packages:
389-ds-base-3.0.1-2.el10.aarch64.rpm
389-ds-base-3.0.1-2.el10.ppc64le.rpm
389-ds-base-3.0.1-2.el10.s390x.rpm
389-ds-base-3.0.1-2.el10.x86_64.rpm

ie, it's just putting them all in Packages/ under the repos, not split by arches.

I think you need to use -a and make some more dirs... happy to provide more info. :)

Metadata