From 6685c828730bc7356516f91623507b0556ac9a7d Mon Sep 17 00:00:00 2001 From: Kevin Fenzi Date: Aug 25 2022 23:16:03 +0000 Subject: rework the openh264 sop Signed-off-by: Kevin Fenzi --- diff --git a/docs/source/sop_generating_openh264_composes.rst b/docs/source/sop_generating_openh264_composes.rst index e31737a..175a455 100644 --- a/docs/source/sop_generating_openh264_composes.rst +++ b/docs/source/sop_generating_openh264_composes.rst @@ -9,13 +9,20 @@ Description =========== Openh264 repos are a special case and we need to generate the composes for it in a different way. -We use ODCS to generate the private compose and send the repos to Cisco to publish them on their CDN. +We use ODCS to generate the private compose and send the rpms to Cisco to publish them on their CDN. +We publish the repodata on our side. .. warning:: We do not have all the appropriate legal rights to distribute these packages, so we need to be extra carefull to make sure they are never distributed via our build system or websites Action ====== +Permissions needed +------------------ + +You will need some ODCS permissions in order to request private composes and composes from tags. +You can set this in infra/ansible in inventory/group_vars/odcs in the odcs_allowed_clients_users variable. See other releng users entries for format. + Get the odcs token ------------------ @@ -25,19 +32,30 @@ Run the odcs-token.py under ``scripts/odcs/`` from pagure releng repository to g :: - $ python odcs-token.py + $ ./odcs-token.py + +Make sure rpms are written out with the right signature +------------------------------------------------------- + +:: + + $ koji write-signed-rpm eb10b464 openh264-2.2.0-1.fc38 + +Where the key for that branch is listed, then the open264 package and version. Generate a private odcs compose ------------------------------- -With the token generated in the privious step, generate the odcs private compose +With the token generated above, generate the odcs private compose :: - $ python odcs-private-compose.py + $ python odcs-private-compose.py `koji_tag`: fxx-openh264 (Openh264 builds are tagged to fxx-openh264 tags where `xx` represents the fedora release) +`signingkeyid`: The short hash of the key for this Fedora branch. + The composes are stored under ``/srv/odcs/private/`` dir on ``odcs-backend-releng01.iad2.fedoraproject.org`` Pull the compose to your local machine @@ -80,42 +98,14 @@ Please follow the following commands to make the necessary tar files to send to $ cp -rv 32-openh264/compose/Temporary/*/*/*/*/*rpm 32-rpms/ # Copy debuginfo rpms $ cp -rv 32-openh264/compose/Temporary/*/*/*/*/*/*rpm 32-rpms/ - # Remove src.rpm - $ rm 32-rpms/*src.rpm + # copy the src.rpm + $ cp -rv 32-openh264/compose/Temporary/*/*/*/*/*src.rpm 32-rpms/ $ cd 32-rpms # Create the tar file with the rpms $ tar -cJvf ../fedora-32-openh264-rpms.tar.xz *rpm We need to send this tar file to Cisco along with the list of rpms in each tarball. -Signing of repodata -^^^^^^^^^^^^^^^^^^^ - -Sync the compose from your local machine to bodhi-backend01.iad2.fedoraproject.org. As sigul is available there, you can sign the repodata on that box. - -Create a working dir in the home dir on bodhi-backend01 - -:: - - $ ssh bodhi-backend01.iad2.fedoraproject.org - $ mkdir openh264-20200813 - -Now from you local machine scp the compose dir to bodhi-backend01 - -:: - - $ scp -rv 32-openh264/ bodhi-backend01.iad2.fedoraproject.org:/home/fedora/mohanboddu/openh264-20200813 - -Go back to bodhi-backend01 and run the following commands - -:: - - $ ssh bodhi-backend01.iad2.fedoraproject.org - # Sign the repodata - $ for repodata in $(find ~/openh264-20200813/32-openh264/ -name repomd.xml); do sigul sign-data fedora-32 $repodata -o $repodata.asc; done - # Change the perms of the the signed repodata - $ find ~/openh264-20200813/32-openh264/ -name repomd.xml.asc | xargs chmod 664 - Syncing the compose to sundries01 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -125,13 +115,6 @@ Once we get a confirmation from Cisco that the rpms are updated on their CDN, ve $ curl -I http://ciscobinary.openh264.org/openh264-2.1.1-1.fc32.x86_64.rpm -Then pull the signed repodata compose to your local machine - -:: - - $ cd openh264-20200813/32-openh264 - $ rsync -avhHP bodhi-backend01.iad2.fedoraproject.org:/home/fedora/mohanboddu/openh264-20200825/32-openh264/ . - Now push these composes to **sundries01.iad2.fedoraproject.org** and **mm-backend01.iad2.fedoraproject.org** On sundries01 we need to sync to a directory that is owned by *apache*, so first we sync to the home directory on sundries01. Same with mm-backend01 as the directory is owned by *root*. @@ -171,3 +154,25 @@ On mm-backend01 $ cd openh264-20200825 $ sudo rsync -avhHP 32-openh264/compose/Temporary/ /srv/codecs.fedoraproject.org/openh264/32/ + +Extra info +^^^^^^^^^^ + +Normally that should be it, but in some cases you may want to push things out faster than normal, +and here's a few things you can do to do that: + +On mm-backend01.iad2.fedoraproject.org you can run: + +:: + + # sudo -u mirrormanager /usr/local/bin/umdl-required codecs /var/log/mirrormanager/umdl-required.log + +This will have mirrormanager scan the codecs dir and update it if it's changed. + +On batcave01.iad2.fedoraproject.org you can use ansible to force all the proxies to sync the codec content from sundries01: + +:: + + # nsible -a '/usr/bin/rsync --delete -a --no-owner --no-group sundries01::codecs.fedoraproject.org/ /srv/web/codecs.fedoraproject.org/' proxies + +Mirrorlist servers should update every 15min.