PR#2070: Update centos-10-sync script to rsync - fedora-infra/ansible

fedora-infra / ansible

#2070 Update centos-10-sync script to rsync

Merged 7 months ago by kevin. Opened 7 months ago by jrichardson.

fedora-infra/ jrichardson/ansible centos_10_sync into main

Update centos-10-sync script to rsync

james02135 • 7 months ago

decfe3e

file modified

+16 -19

		`@@ -1,21 +1,18 @@`
		`#!/bin/sh`

		`- #Sync using reposync instead of rsync until available`
		`- SERVER=composes.stream.centos.org`
		`- TOPDIR=stream-10/production/latest-CentOS-Stream/compose`
		`-`
		`- ## mirror CentOS-10`
		`-`
		`- DESTDIR=/mnt/fedora/app/fi-repo/centos/centos-10`
		`-`
		`- mkdir -p ${DESTDIR}`
		`-`
		`- for basearch in aarch64 ppc64le s390x x86_64`
		`- do`
		`- for repo in BaseOS AppStream CRB`
		`- do`
		`- mkdir -p $DESTDIR/$repo/$basearch`
		`- cd $DESTDIR/$repo/$basearch`
		`- reposync --download-metadata --disablerepo=* --repofrompath=os,https://$SERVER/$TOPDIR/$repo/$basearch/os/ --enablerepo=os`
		`- done`
		`- done`
		`\ No newline at end of file`
		`+ ## Set up variables we use`
		`+ RSYNC='/usr/bin/rsync'`
		`+ RS_OPT="-avSHP --numeric-ids"`
		`+ RS_DEADLY="--delete --delete-delay --delay-updates"`
		`+ CENT_EXCLUDES="--exclude=isos --exclude=kickstart --exclude=images --exclude=debug"`
		`+`
		`+ SERVER=centos-master01.iad2.fedoraproject.org`
		`+`
		`+ ## Mirror CentOS-10-Stream`
		`+`
		`+ RSYNC_DESTDIR=/mnt/fedora/app/fi-repo/centos/centos-10/`
		`+ RSYNC_MOD="CentOS-Stream-Ref/10-stream/"`
		`+`
		`+ mkdir -p ${RSYNC_DESTDIR}`
		`+`
		`+ ${RSYNC} ${RS_OPT} ${RS_DEADLY} ${CENT_EXCLUDES} ${SERVER}::${RSYNC_MOD} ${RSYNC_DESTDIR}`
		`\ No newline at end of file`

jrichardson commented 7 months ago

Signed-off-by: james02135 james02135@hotmail.com

zuul commented 7 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/71743d9197c749c79c78b4ed70b5c216

fi-ansible-lint-diff : SUCCESS in 3m 54s
fi-yamllint-diff : SUCCESS in 2m 33s

rebased onto c775045

7 months ago

zuul commented 7 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/a98ad59695b74516a4bd0c672b6e833b

fi-ansible-lint-diff : SUCCESS in 4m 08s
fi-yamllint-diff : SUCCESS in 2m 25s

rebased onto 9170bf5

7 months ago

rebased onto 9170bf5

7 months ago

zuul commented 7 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/fa746406f3fd41c197a3ed95f781d3b5

fi-ansible-lint-diff : SUCCESS in 3m 54s
fi-yamllint-diff : SUCCESS in 2m 25s

zlopez commented on line 30 of roles/batcave/files/centos-10-sync 7 months ago

Why the server change?

rebased onto 40c0408

7 months ago

rebased onto 40c0408

7 months ago

zuul commented 7 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/894885856a8e48e6acb97fe89f6ef952

fi-ansible-lint-diff : SUCCESS in 3m 46s
fi-yamllint-diff : SUCCESS in 2m 27s

carlwgeorge commented on line 34 of roles/batcave/files/centos-10-sync 7 months ago

Let's keep the same DESTDIR as before, /mnt/fedora/app/fi-repo/centos/centos-10.

carlwgeorge commented on line 28 of roles/batcave/files/centos-10-sync 7 months ago

I think we can get away with excluding the debug directory as well, since we don't build against those.

carlwgeorge commented on line 30 of roles/batcave/files/centos-10-sync 7 months ago

The title of this PR is a bit misleading. The overall purpose is to switch from compose content (composes.stream.centos.org) to mirrored content (centos-master01.iad2.fedoraproject.org), matching how we do it for CentOS 9. reposync vs rsync is just an implementation detail.

jrichardson commented 7 months ago

I think we can get away with excluding the debug directory as well, since we don't build against those.

The title of this PR is a bit misleading. The overall purpose is to switch from compose content (composes.stream.centos.org) to mirrored content (centos-master01.iad2.fedoraproject.org), matching how we do it for CentOS 9. reposync vs rsync is just an implementation detail.

My fault, I should have titled it better

rebased onto 0d30d6b

7 months ago

jrichardson commented 7 months ago

Changes made

zuul commented 7 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/34557f0fcc1f4e57b007ac78b180b89e