From f7e20b5dfd683fd5c415a4ae721fc170ea058e71 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Mar 02 2022 14:53:31 +0000 Subject: Update systemd docs to reflect new systemd RPM process The hyperscale systemd CentOS RPM spec now pulls tarballs directly from the staging repo on pagure. We also switched to the flat dist-git layout so we can sync from the main rawhide branch using git instead of doing it manually. This commit updates the documentation to reflect those process changes. --- diff --git a/docs/internal/systemd.md b/docs/internal/systemd.md index a42e198..e8d2283 100644 --- a/docs/internal/systemd.md +++ b/docs/internal/systemd.md @@ -20,7 +20,7 @@ Steps: * Get a Fedora account [here](https://accounts.fedoraproject.org/user/daandemeyer/) if you don't have one already. * Fork [centos-sig-hyperscale/systemd](https://pagure.io/centos-sig-hyperscale/systemd). -* Check out the latest branch there. It should be the last one alphabetically in the form of "fb-v". +* Check out the latest Hyperscale branch there. It should be the last one alphabetically in the form of "hs-v". If you want to backport a patch to a stable systemd version that does not yet have a branch available in the staging repo, open an issue [here](https://pagure.io/centos-sig-hyperscale/systemd/issues) to ask the maintainers to create a new branch for that version. * If you think your PR from upstream will apply cleanly, you can download it and `git am ` it directly. Otherwise you can use @@ -32,13 +32,7 @@ Steps: * Once merged, one of the systemd Hyperscale maintainers will update the RPM spec and publish a new version of systemd for Hyperscale with the patches. -# Contributing to the RPM spec - -Not yet available - -[systemd Hyperscale RPM sources repo](https://git.centos.org/rpms/systemd/tree/c8s-sig-hyperscale) - -# New Version Updates in the SIG +# Contributing to the RPM spec / New Version Updates in the SIG Make sure you're [onboarded](https://sigs.centos.org/hyperscale/internal/onboarding/) onto the SIG before following these steps. @@ -51,20 +45,10 @@ newer CentOS version, rarely will changes on the older branch fail on the newer Steps: -* Upload the new version tarball to CentOS. - * Download the desired version's tarball from [systemd-stable](https://github.com/systemd/systemd-stable/tags). Move - it to the RPM sources repo's `SOURCES/` directory (you'll need this for later). - * Use the [centos-git-common's lookaside_upload script](https://git.centos.org/centos-git-common/blob/master/f/lookaside_upload) - to upload the tarball. For example: `lookaside_upload -f ./SOURCES/systemd-250.3.tar.gz -n systemd -b c8s-sig-hyperscale`. -* Update `.systemd.metadata` in the RPM sources repo with the new version and hash. - * For example, the [c8s-sig-hyperscale .systemd.metadata](https://git.centos.org/rpms/systemd/blob/c8s-sig-hyperscale/f/.systemd.metadata) - contains the sha1sum of the tarball, and the location of the tarball relative to the RPM sources repo root. -* Clean up all of the patches that have been merged into the latest version of systemd. - * When you drop patches, delete the `PatchXYZ:` line and delete the corresponding file from the `SOURCES/` directory. - * Take [lines 98-162 of this example commit](https://git.centos.org/rpms/systemd/blob/4f096da11ccb2b9dc2a13ca184ad3ae23ca68fa1/f/SPECS/systemd.spec) - from systemd 249.4-2.12. If we're updating to systemd 250.3, Patch0007-Patch0023 can be dropped since they are merged - in 250. If there are PRs that are unmerged, like Patch0024, we should keep them on the list. We may be able to drop - more patches, the Fedora ones specifically, in the next step. +* Clone the CentOS RPM sources [repo](https://git.centos.org/rpms) + * Checkout the c8s-sig-hyperscale branch +* Add the Fedora RPM sources [repo](https://src.fedoraproject.org/rpms/systemd) as a git remote + * `git remote add fedora https://src.fedoraproject.org/rpms/systemd && git fetch fedora` * Sync against changes from [Fedora Rawhide](https://src.fedoraproject.org/rpms/systemd/commits/rawhide). * The gist of this step is to figure out when we last synced from Fedora Rawhide, and get a diff between the latest Rawhide changes and the last time we synced. Have the Fedora Rawhide repo checked out and pulled to the latest @@ -75,37 +59,64 @@ Steps: we use for Hyperscale (e.g. 249.4-2 vs. 249.4-2.1). * If we look for the Fedora Rawhide commit corresponding to that changelog line, it will take us to [46a408102a0cfb5692d4625a33025811cab0343f](https://src.fedoraproject.org/rpms/systemd/c/46a408102a0cfb5692d4625a33025811cab0343f?branch=rawhide). - * Do a `git diff 46a408102a0cfb5692d4625a33025811cab0343f` to get all the changes from the latest commit to the last - time we synced to CentOS Hyperscale. At this point, you can drop Fedora patches where necessary in order to match - Rawhide. Most of the work will be applying the spec/source changes we want by hand to the CentOS Hyperscale RPM sources - repo, taking care not to add anything we don't want (e.g. firewalld stuff). -* Add new backports to the specfile. - * We usually have a few patches from systemd main that we want backported to the latest release. We should have also - backported them all to the Hyperscale systemd staging repo (above). Now we need to turn those into patch files and - update the specfile. I usually keep it to one patch file per PR to make it easy to annotate and remove later. - * Take [fb-v250.3 from the staging repo](https://pagure.io/centos-sig-hyperscale/systemd/tree/fb-v250.3) as an example. - Commits 033135218df105bb1b79d78c56fe72fdb240a528 through abf3f6ecbf785439730d04307ccaf9c550697ef0 belong to [PR #21183](https://github.com/systemd/systemd/pull/21183). - We can run `git format-patch 033135218df105bb1b79d78c56fe72fdb240a528..abf3f6ecbf785439730d04307ccaf9c550697ef0 --stdout > /path/to/systemd-centos-rpm/SOURCES/221183-cherrypicked.patch` - to generate a patch file with all of the desired commits. - * Update the CentOS Hyperscale RPM specfile with a number, and the name of the patch file you just created. There - is a comment in the specfile that tells you how to number patches according to upstream PRs vs. downstream, etc. - * You should be keeping the patches in relatively the same order that they are in the staging repo otherwise you - might have a bad time if they get applied out of order. - * Don't forget to use the `facebook` flag where necessary to gate out FB-only patches. -* Update the version/release, and changelog. + * `git diff 46a408102a0cfb5692d4625a33025811cab0343f..fedora/rawhide` will get us all the changes from the latest + commit to the last time we synced to CentOS Hyperscale. To apply these changes to the Hyperscale branch, run + `git apply -3 < $(git diff 46a408102a0cfb5692d4625a33025811cab0343f..fedora/rawhide)`. This will apply the changes and + generate merge conflicts when the upstream changes conflict with our own changes. +* Update the source commits, version/release and changelog. + * At the top of the specfile, update the `hs_commit` macro to point to the latest commits for the Hyperscale and + Facebook branches for the current release on the staging [repo](https://pagure.io/centos-sig-hyperscale/systemd) + respectively. + * For the v250 release, these would be the latest commits on the hs-v250.3 and hs+fb-v250.3 branches respectively. * At the top of the specfile, update the `Version` and `Release` values. This should match Fedora, except that we also append a dot number value to indicate which iteration of the Hyperscale release we're on. * At the bottom of the specfile under `changelog`, add an entry about these changes. +* Update the `sources` file with the new tarball names and SHA512 hashes. + * For example, the c8s-sig-hyperscale [sources](https://git.centos.org/rpms/systemd/blob/c8s-sig-hyperscale/f/sources) + contains the SHA512 hashes along with the names of the tarballs. * Git commit all these changes. Don't push just yet. * If you push at this point it's not the end of the world, but we haven't confirmed the build succeeds yet! * At this point it's also a good idea to ask for a sanity check from the other Hyperscale maintainers. You can do a `git show` and paste the output to the [CentOS pastebin](https://pastebin.centos.org/). -* Create an RPM by following the instructions below for creating a src.rpm, then build locally with `mock` or do a -scratch build in CBS. +* Run the following commands to download the tarballs for the Hyperscale and Facebook branches: + * `spectool --define "_sourcedir $PWD" --define "%facebook 1" -g systemd.spec` + * `spectool --define "_sourcedir $PWD" -g systemd.spec` +* Build locally with `mock` * One common reason for the RPM build failing is patch files between the staging repo and Fedora not playing nicely together. If this is the case, you can try using `git am ` to apply the Fedora patches to the staging repo and iron out build failures there. Rearrange patches as needed in the staging repo or specfile to get things to fit. * Other reasons for failure are dependencies. This you'll need to figure out from logs. +* Do a scratch build in CBS + * You can use the source RPM generated by running `mock` to do the scratch build. It'll be located at + `/var/lib/mock/centos-stream-8-x86_64/root/builddir/build/SRPMS`. +* Upload the new version tarballs to CentOS. + * Calculate the SHA512 hash of a tarball using sha512sum. + * Check if the tarball has not already been uploaded [here](https://git.centos.org/sources/systemd/) + * Upload the tarball to the CentOS lookaside cache using the following curl command: + + ``` + curl https://git.centos.org/sources/upload.cgi \ + --fail \ + --cert ~/.centos.cert \ + --form "name=systemd" \ + --form "branch=" \ + --form "sha512sum=" \ + --form "file=@" \ + --progress-bar + ``` + + An example invocation might look as follows: + + ``` + curl https://git.centos.org/sources/upload.cgi \ + --fail \ + --cert ~/.centos.cert \ + --form name=systemd \ + --form branch=c8s-sig-hyperscale \ + --form sha512sum=5b9ec28102538bc3dcb632ee16389ff20dccf4b723186f6ae2da119a1809d84db0d8bcecf9b75c5e2da8427f5543e1da281bbed1a154e529d8a82ea5128c465c \ + --form file=@./systemd-249.4.tar.gz \ + --progress-bar + ``` * All updates are in and we have a successful build? Push your changes and get started with testing! # Build Commands