While staging F30 Beta, I used this script to rsync the compose to the mirrors which internally uses compose-partial-copy.
But it missed rsyncing some variants totally and some arches under the variants. The missed one's are:
Variant - arches - location Cloud - aarch64, x86_64 - /pub/fedora/linux/releases/test/30_Beta/Cloud/ Container - aarch64, x86_64 - /pub/fedora/linux/releases/test/30_Beta/Container/ Container - s390x - /pub/fedora-secondary/releases/test/30_Beta/Container/ Labs - armhfp - /pub/alt/releases/test/30_Beta/Labs/ Spins - armhfp - /pub/fedora/linux/releases/test/30_Beta/Spins/
The script possibly miss some other's as well, we just didn't have those artifacts. For example, for this compose we didn't had Container/ppc64le images, dont know if it would have rsynced it properly or not, but thought of mentioning that as well.
More missing stuff:
Labs - i386/images - /pub/fedora-secondary/releases/test/30_Beta/Labs/i386/ Labs - x86_64/images - /pub/alt/releases/test/30_Beta/Labs/x86_64/ Server - aarch64/images - /pub/fedora/linux/releases/test/30_Beta/Server/aarch64/ Server - armhfp/images - /pub/fedora/linux/releases/test/30_Beta/Server/armhfp/ Workstation - armhfp/images - /pub/fedora/linux/releases/test/30_Beta/Workstation/armhfp/
Fixed all of them by:
sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Cloud/aarch64/ /pub/fedora/linux/releases/test/30_Beta/Cloud/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Cloud/x86_64/ /pub/fedora/linux/releases/test/30_Beta/Cloud/x86_64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Container/aarch64/ /pub/fedora/linux/releases/test/30_Beta/Container/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Container/x86_64/ /pub/fedora/linux/releases/test/30_Beta/Container/x86_64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Container/s390x/ /pub/fedora-secondary/releases/test/30_Beta/Container/s390x/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Labs/armhfp/ /pub/alt/releases/test/30_Beta/Labs/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Spins/armhfp/ /pub/fedora/linux/releases/test/30_Beta/Spins/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Labs/i386/images/ /pub/fedora-secondary/releases/test/30_Beta/Labs/i386/images/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Labs/x86_64/images/ /pub/alt/releases/test/30_Beta/Labs/x86_64/images/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Server/aarch64/images/ /pub/fedora/linux/releases/test/30_Beta/Server/aarch64/images/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Server/armhfp/images/ /pub/fedora/linux/releases/test/30_Beta/Server/armhfp/images/ sudo -u ftpsync rsync -avhH /pub/alt/stage/30_Beta-1.8/Workstation/armhfp/images/ /pub/fedora/linux/releases/test/30_Beta/Workstation/armhfp/images/
I think I narrowed down the issue, its skipped rsyncing all the images artifacts.
images
Please upgrade python3-productmd on the machine where the script is running to latest version. The partial copy figures out what to copy based on metadata in the compose, and if it encounters a problem with images.json, it will skip images.
I tested this on stage composer with older productmd, and it had the same result due to vmdk being unrecognized. After updating to 1.21 the copy would work correctly.
vmdk
It probably makes sense to bail out completely instead of silently copying only part of the data.
PR #94 changes the script to report an error.
@lsedlar Thanks, I updated python3-productmd to python3-productmd-1.21-1 on the box. And I will also update the compose-utils once its built and will verify this next time when I have to run the staging again (probably for f30 GA).
@mohanboddu can we close this ticket?
Yup and done.
Metadata Update from @mohanboddu: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
This happened again, so I fixed it manually again
sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Cloud/aarch64/ /pub/fedora/linux/releases/33/Cloud/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Cloud/x86_64/ /pub/fedora/linux/releases/33/Cloud/x86_64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Container/aarch64/ /pub/fedora/linux/releases/33/Container/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Container/armhfp/ /pub/fedora/linux/releases/33/Container/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Container/x86_64/ /pub/fedora/linux/releases/33/Container/x86_64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Server/armhfp/ /pub/fedora/linux/releases/33/Server/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Server/aarch64/ /pub/fedora/linux/releases/33/Server/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Spins/armhfp/ /pub/fedora/linux/releases/33/Spins/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Workstation/armhfp/ /pub/fedora/linux/releases/33/Workstation/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Workstation/aarch64/ /pub/fedora/linux/releases/33/Workstation/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Cloud/ppc64le/ /pub/fedora-secondary/releases/33/Cloud/ppc64le/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Cloud/s390x/ /pub/fedora-secondary/releases/33/Cloud/s390x/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Container/ppc64le/ /pub/fedora-secondary/releases/33/Container/ppc64le/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Container/s390x/ /pub/fedora-secondary/releases/33/Container/s390x/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Spins/aarch64/ /pub/fedora-secondary/releases/33/Spins/aarch64/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Labs/armhfp/ /pub/alt/releases/33/Labs/armhfp/ sudo -u ftpsync rsync -avhH /pub/alt/stage/33_RC-1.2/Labs/x86_64/ /pub/alt/releases/33/Labs/x86_64/
These are the versions that I am using
python3-productmd-1.27-1.fc32.noarch and compose-utils-0.1.42-1.fc32.noarch
python3-productmd-1.27-1.fc32.noarch
compose-utils-0.1.42-1.fc32.noarch
Metadata Update from @mohanboddu: - Issue status updated to: Open (was: Closed)
Login to comment on this ticket.