#75 Update multi package update instructions
Merged 2 years ago by oturpe. Opened 2 years ago by oturpe.
fedora-docs/ oturpe/package-maintainer-docs multi-build-update  into  main

Update multi package update instructions
Otto Urpelainen • 2 years ago  
@@ -64,128 +64,6 @@ 

  (on the now hopefully fixed package),

  and so forth.

  

- [#multiple_packages]

- === Multiple Packages

- 

- Some updates require changes in multiple related packages,

- e.g. if a library is upgraded to a new major version

- and packages using it need to be rebuilt.

- 

- Fedora has the concept of using side-tags for these situations,

- which means the builds are done "on the side"

- and do not affect packages out of the side-tag,

- nor are they available for installing

- until the side-tag is merged.

- 

- Packagers can create side-tags on their own,

- allowing them to build disruptive components in isolation

- and submit the builds in a side-tag

- as one update in Bodhi,

- to be tested and subsequently merged into the main distribution.

- 

- Side-tags are cleaned up 30 days after creation, or 14 days if they

- have not been used at all. Make sure and use your side tag before then.

- 

- [#creating_a_side_tag]

- ==== Creating a side-tag

- 

- To create a side-tag for building packages for Rawhide,

- the easiest way is to be

- in the checked out `rawhide` branch of one of the packages

- and issue the following command:

- 

- ....

- fedpkg request-side-tag

- ....

- 

- Alternatively, you can specify the parent/base tag

- from which to create the side-tag, e.g.:

- 

- [subs="attributes+"]

- ....

- fedpkg request-side-tag --base-tag f{MAJOROSVER}-build

- ....

- 

- 

- NOTE: Side-tags for particular Fedora releases are based off its respective build tag.

- I.e. if you wanted to create a side-tag for Rawhide

- while it's ramping up for Fedora {MAJOROSVER},

- the parent tag to choose would be `f{MAJOROSVER}-build`.

- 

- This will tell you the commands to

- 

- . build a package in the specific side-tag and

- . wait for the respective build root to be recreated, e.g.:

- 

- [subs="attributes+"]

- ....

- $ fedpkg request-side-tag --base-tag f{MAJOROSVER}-build

- Side tag 'f{MAJOROSVER}-build-side-7863' (id 7863) created.

- Use 'fedpkg build --target=f{MAJOROSVER}-build-side-7863' to use it.

- Use 'koji wait-repo f{MAJOROSVER}-build-side-7863' to wait for the build repo to be generated.

- ....

- 

- The latter is important

- if any builds depend on previous ones in the side-tag.

- Use `koji wait-repo --build <package-nvr> <side-tag>` to ensure

- that the respective build is available

- in the build root

- for subsequent builds.

- 

- [#bodhi_update_for_builds_in_a_side_tag]

- ==== Bodhi update for builds in a side-tag

- 

- When you're done building all packages you want in a side-tag,

- you have to submit them as an update to Bodhi

- before they can be made available generally

- to be installed and built upon.

- 

- In the _Create New Update_ form in Bodhi,

- choose the _Use Side-Tag_ drop-down

- to create an update from the latest package builds in the respective tag:

- 

- image:Bodhi-builds-from-side-tag.png[Bodhi-builds-from-side-tag.png,title="Bodhi-builds-from-side-tag.png"]

- 

- As with single packages, tests will be run

- whose result affects if the update can be moved to stable.

- The difference is that tests have to succeed

- (or be waived)

- for all builds in the update.

- If you have to update the list of builds,

- e.g. to fix problems found during testing,

- edit the update

- and refresh the list of builds using the 🔃 (refresh) button:

- 

- image:Bodhi-builds-refresh-from-side-tag.png[Bodhi-builds-refresh-from-side-tag.png,title="Bodhi-builds-refresh-from-side-tag.png"]

- 

- The web interface only works if you are the creator of the side-tag.

- If you are a proven packager submitting an update for a side-tag,

- you currently need to use the bodhi cli:

- 

- ....

- $ bodhi updates new --from-tag --notes "whatever" --user <FAS>

- ....

- 

- Once the update moves to stable,

- the builds will be tagged to the main tag of the release,

- i.e. are available for the general public.

- 

- [adding_removing_builds_to_from_a_side_tag]

- ==== Adding/Removing builds to/from a side-tag

- As a packager,

- you can add or remove builds from your side-tag

- using these commands:

- 

- ....

- $ koji tag <side-tag> <nvr>

- $ koji untag <side-tag> <nvr>

- ....

- 

- This can be used to remove a build that made a test fail

- or to add a build that was originally missed.

- If you add or remove a build from a side-tag,

- you will have to refresh the corresponding update in Bodhi.

- 

  [#later_branched_and_stable_releases]

  == Later Branched and stable releases

  
@@ -296,83 +174,6 @@ 

  is marked as _stable_

  and reaches the _updates_ repository.

  

- [#updating_inter_dependent_packages]

- === Updating inter-dependent packages

- 

- If an update you wish to submit would cause a dependency issue of any kind

- (a strict package dependency error,

- or simply another package failing to operate correctly)

- if updated alone,

- you must not submit the package as a single-package update.

- You must always collect all inter-dependent or related packages together

- into a single multi-package update,

- such that no user will face problems

- if they install all the packages in the update together.

- 

- For example:

- if you maintain a package `libfoo`

- which the package `bar` depends on,

- and you need to update `libfoo`,

- you should check that `bar` continues to function correctly

- with the updated version of `libfoo`.

- If it does not,

- you must ensure the appropriate changes are made to `bar`,

- and include the updated `bar` in your update

- along with the updated `libfoo`.

- 

- The `fedpkg` tool does not handle multi-package updates.

- You can create an update with multiple packages

- with the https://bodhi.fedoraproject.org[Bodhi web application],

- or with the `bodhi` command line tool.

- You can pass as many package names as you like to `bodhi updates new` command

- to create a new multi-package update,

- or use `bodhi updates edit` to edit an existing update.

- 

- It is possible you will run into problems with permissions

- when trying to add builds of packages

- you do not have commit privileges for

- to an update,

- or trying to add a build

- for a package you do have privileges for

- to someone else's update.

- If you encounter a situation like this,

- you should contact the https://docs.pagure.org/releng/[release engineering] team

- or a proven packager

- for help.

- 

- You may need a _buildroot override_

- to complete a multi-package update successfully.

- For instance in the case described above,

- you may need to rebuild `bar` against the new `libfoo` package

- and submit both packages together

- as a multi-package update.

- However, in the normal course of events,

- you would not be able to build another package against your new `libfoo` build

- until it reached the stable state.

- To resolve this dilemma,

- you can request a buildroot override,

- which causes the `libfoo` build

- to be included in the buildroot for a short time

- in order to get the `bar` package build done.

- 

- You can request a buildroot override with bodhi:

- This would submit a buildroot override

- with a duration of two days.

- Buildroot overrides are usually granted within 15-30 minutes of submission.

- If you submit an override request with the bodhi tool,

- it will suggest a command

- that will let you monitor when the package appears in the buildroot,

- so you can fire your dependent build at the appropriate time:

- 

- [subs="attributes+"]

- ....

- koji wait-repo f{MAJOROSVER}-build --build=(name-version-release)

- ....

- 

- You can also request buildroot overrides from the https://bodhi.fedoraproject.org/overrides/new[Bodhi web application].

- 

- The https://fedoraproject.org/wiki/Bodhi/BuildRootOverrides[buildroot override instructions] explain the buildroot override process in more detail.

- 

  [#handling_feedback_from_automated_tests]

  === Handling feedback from automated tests

  
@@ -521,6 +322,148 @@ 

  

  For more on the Fedora development process, see https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle[Fedora Release Life Cycle].

  

+ [#multiple_packages]

+ == [[updating_inter_dependent_packages]] Multi-package updates

+ 

+ Some updates require changes in multiple related packages.

+ For example, if you maintain `bar` which depends on `libfoo`,

+ and you need to update `libfoo`,

+ you should check that `bar` still works with the updated `libfoo`.

+ If it does not, you must ensure the appropriate changes are made to `bar`,

+ and include the updated `bar` in your update along with the updated `libfoo`.

+ 

+ === Side tags

+ 

+ Fedora has the concept of using side tags for these situations,

+ which means the builds are done "on the side"

+ and do not affect packages out of the side tag,

+ nor are they available for installing until the side tag is merged.

+ 

+ Packagers can create side tags on their own,

+ allowing them to build disruptive components in isolation

+ and submit the builds in a side tag as one update in Bodhi,

+ to be tested and subsequently merged into the main distribution.

+ 

+ Side tags are cleaned up 30 days after creation,

+ or 14 days if they have not been used at all.

+ Make sure and use your side tag before then.

+ 

+ [#creating_a_side_tag]

+ ==== Creating a side tag

+ 

+ The easiest way to create a side tag

+ is to be in the checked out  branch (e.g. `rawhide`) of one of the packages

+ and issue the following command:

+ 

+ ----

+ fedpkg request-side-tag

+ ----

+ 

+ Alternatively, the following variants allow specifying the Fedora release or the base tag for the side tag:

+ 

+ [subs="attributes+"]

+ ----

+ fedpkg --release f{MAJOROSVER} request-side-tag

+ fedpkg request-side-tag --base-tag f{MAJOROSVER}-build

+ ----

+ 

+ `fedpkg` will tell you the commands to build a package in the specific side tag

+ and to wait for the respective build root to be recreated:

+ 

+ [subs="attributes+"]

+ ----

+ $ fedpkg request-side-tag

+ Side tag 'f{NEXTOSVER}-build-side-7863' (id 7863) created.

+ Use 'fedpkg build --target=f{NEXTOSVER}-build-side-7863' to use it.

+ Use 'koji wait-repo f{NEXTOSVER}-build-side-7863' to wait for the build repo to be generated.

+ ----

+ 

+ The latter is important if any builds depend on previous ones in the side tag.

+ Use `koji wait-repo --build <package-nvr> <side-tag>` to ensure

+ that the respective build is available in the build root for subsequent builds.

+ 

+ [#bodhi_update_for_builds_in_a_side_tag]

+ ==== Bodhi update for builds in a side tag

+ 

+ When you're done building all packages you want in a side tag,

+ you have to submit them as an update to Bodhi

+ before they can be made available generally to be installed and built upon.

+ 

+ In the _Create New Update_ form in Bodhi,

+ choose the _Use Side-Tag_ drop-down

+ to create an update from the latest package builds in the respective tag:

+ 

+ image:Bodhi-builds-from-side-tag.png[Bodhi-builds-from-side-tag.png,title="Bodhi-builds-from-side-tag.png"]

+ 

+ As with single packages, tests will be run

+ whose result affects if the update can be moved to stable.

+ The difference is that tests have to succeed (or be waived) for all builds in the update.

+ If you have to update the list of builds,

+ e.g. to fix problems found during testing,

+ edit the update and refresh the list of builds using the 🔃 (refresh) button:

+ 

+ image:Bodhi-builds-refresh-from-side-tag.png[Bodhi-builds-refresh-from-side-tag.png,title="Bodhi-builds-refresh-from-side-tag.png"]

+ 

+ The web interface only works if you are the creator of the side tag.

+ If you are a proven packager submitting an update for a side tag,

+ you currently need to use the bodhi cli:

+ 

+ ----

+ bodhi updates new --from-tag --notes "whatever" --user <FAS>

+ ----

+ 

+ Once the update moves to stable,

+ the builds will be tagged to the main tag of the release,

+ i.e. are available for the general public.

+ 

+ [#adding_removing_builds_to_from_a_side_tag]

+ ==== Adding/Removing builds to/from a side tag

+ As a packager, you can add or remove builds from your side tag using these commands:

+ 

+ ----

+ koji tag <side-tag> <nvr>

+ koji untag <side-tag> <nvr>

+ ----

+ 

+ This can be used to remove a build that made a test fail

+ or to add a build that was originally missed.

+ If you add or remove a build from a side tag,

+ you will have to refresh the corresponding update in Bodhi.

+ 

+ [#removing_side_tag]

+ ==== Removing a side tag

+ To conserve build system resources,

+ side tags that are not needed anymore should be removed.

+ This can be achieved with `fedpkg` as follows:

+ 

+ ----

+ fedpkg remove-side-tag <side-tag>

+ ----

+ 

+ === Bodhi multi package updates

+ 

+ You can also create an update with multiple packages directly in Bodhi,

+ either with the https://bodhi.fedoraproject.org[Bodhi web application],

+ or with the `bodhi` command line tool.

+ You can pass multiple package names to `bodhi updates new` command

+ to create a new multi-package update,

+ or use `bodhi updates edit` to edit an existing update.

+ 

+ It is possible you will run into problems with permissions

+ when you are not the maintainer of the package you are trying to add to an update,

+ or when you are trying to add packages to an update created by somebody else.

+ If you encounter a situation like this,

+ you should contact the https://docs.pagure.org/releng/[release engineering] team

+ or a proven packager for help.

+ 

+ [#buildroot_overrides]

+ === Buildroot overrides

+ 

+ Yet another way to build multiple packages together are

+ https://fedoraproject.org/wiki/Bodhi/BuildRootOverrides[buildroot overrides].

+ Unlike side tags, buildroot overrides are not isolated

+ and thus should be only used in special cases where side tags do not suffice.

+ 

  [#security_updates]

  == Security updates

  

Package Update Guide contained strange instructions
recommending using side tags only for Rawhide and early branched,
and buildroot overrides for late branched and stable.
Actually, side tags should be preferred in every case.
Merged the instructions together to new section "Multi Package Updates"
that prefers side tags in all cases.

Resolves #33

rebased onto 85b9f2810b6c2b122a3770c41434e2d727e2f62d

2 years ago

@churchyard I think you have asked for this more than once.
Please take a look.

This method was included in the previous version, but I am not sure.
Would it be simpler and safer to just say "use side tags",
even if there may be some cases where this method would suffice?

I think what you have here now is fine... someday we may retire buildroot overrides entirely and we can remove that then...

Technical: Could we keep the updating_inter_dependent_packages anchor somewhere in this document?

I would consider fedpkg --release {BRANCH} request-side-tag more familiar to packagers. WDYT?

That might be out of scope here. Ignore the comment.

I think the override section is as good as it gets. They exist and should only be used when side tags do not suffice. +1

Technical: Could we keep the updating_inter_dependent_packages anchor somewhere in this document?

Good point. Another AsciiDoc nut to crack.
Turns out this syntax allows multiple anchors for one section:

[#multiple_packages]
== [[updating_inter_dependent_packages]] Multi-package updates

rebased onto 7a9aab8

2 years ago

I would consider fedpkg --release {BRANCH} request-side-tag more familiar to packagers. WDYT?

That might be out of scope here. Ignore the comment.

I edited that section a bit regardless.
It now covers the case of creating a side tag for non-rawhide better,
and also lists that simplex syntax.

I did further minor formatting changes,
and added a short section about removing unused side tags.
I think this is ready for merge now.
I will self-merge if nobody else does that.
Maybe @kevin can do that, as he is in the committer group?

Pull-Request has been merged by oturpe

2 years ago
Metadata