#203 How do I get correct changelogs as a downstream when mirroring Git repos?
Opened 3 years ago by ngompa. Modified 3 years ago

With @decathorpe's build of rust-async-io using rpmautospec, I attempted to do my typical "downstream package build" workflow for this. Unfortunately, one very big problem is that I can't divine the changelogs from Git at all. This makes maintaining the historical information of Fedora packages properly almost impossible unless I switch from Git to importing SRPMs, which I don't want to do.

What am I supposed to do?


Note this also has implications for Fedora->RHEL as well, since it becomes possible to accidentally lose the history...

What's downstream for Rust packages? I assumed there's no problem here since they're not in EPEL.

You could just preprocess things with "rpmautospec process-distgit" before doing downstream builds or if preprocessing locally is not an option you could fetch the SRPM files from koji (scratch that, I see now that this is not what you want to do.)

If you need only the changelog, rpmautospec generate-changelog … would do that for you.

So, I'm looking at extending my internal obsctl tooling (yes, still planned to be open sourced, I promise!) to handle the rpmautospec workflow, and that means wiring up the correct stuff to leverage it for building in the Open Build Service.

What I want to avoid is a chance for it going wrong when I am not talking to Fedora infrastructure.

My workflow currently is:

  1. Mirror package from src.fedoraproject.org
  2. Create downstream branch (e.g. internal)
  3. Do adaptations for our use (e.g. add support for building on Ubuntu, tweaks for EL8, custom flags, etc.)
  4. Verify in CI, then submit to internal OBS for production build on tagging a commit to release

Since OBS doesn't work off Git natively, I need to wire up things so I can process packages properly before releasing to the build system.

This seems to work as intended, though it's a bit odd. Is this essentially bumping the release for every commit?

ngompa@Belldandy-Slimbook ~/t/rust-async-io (internal)> rpmautospec generate-changelog
* Thu Jul 15 2021 Neal Gompa <ngompa13@gmail.com> 1.6.1-1
- new version

* Thu Jul 15 2021 Neal Gompa <ngompa13@gmail.com> 1.6.0-3
- Revert "Use SPDX identifiers"

* Thu Jul 15 2021 Neal Gompa <ngompa13@gmail.com> 1.6.0-2
- Use SPDX identifiers

* Thu Jul 08 2021 Fabio Valentini <decathorpe@gmail.com> 1.6.0-1
- Update to version 1.6.0; Fixes RHBZ#1977770

* Wed Apr 28 2021 Fabio Valentini <decathorpe@gmail.com> - 1.4.1-1
- Update to version 1.4.1.
- Fixes RHBZ#1953161

* Fri Apr 16 2021 Fabio Valentini <decathorpe@gmail.com> - 1.4.0-1
- Update to version 1.4.0.
- Fixes RHBZ#1950433

* Wed Jan 27 2021 Fedora Release Engineering <releng@fedoraproject.org> - 1.3.1-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild

* Wed Jan 06 2021 Fabio Valentini <decathorpe@gmail.com> - 1.3.1-1
- Initial package

I guess I'll need to have a wrapper for tag-releases for packages using rpmautospec.

Since OBS doesn't work off Git natively, I need to wire up things so I can process packages properly before releasing to the build system.

For my understanding, OBS consumes just SRPMS? We implemented something like this for scratch builds in fedpkg/rpkg, here are the relevant PRs that landed in rpkg meanwhile:

https://pagure.io/rpkg/pull-request/548
https://pagure.io/rpkg/pull-request/557

Condensed: check if rpmautospec features are used and if so, preprocess the spec file and roll an SRPM from it.

This seems to work as intended, though it's a bit odd. Is this essentially bumping the release for every commit?

Yes, it works as intended, what FESCo asked from us to essentially avoid having to care about historical builds (which was the reason for the git tags in the prototype).

On IRC, you mentioned you wanted something which only added the plain version without release to the changelog entries for the commits where the version is bumped. Did I get this correctly?

With rpkg-1.62-6 or newer, you should be able to fedpkg srpm and import that one to OBS. Does that solve your problem?

I've also opened https://pagure.io/fedora-infra/rpmautospec/issue/206 -- that should make it possible to add "downstream only commits" without changing the release number.

Since OBS doesn't work off Git natively, I need to wire up things so I can process packages properly before releasing to the build system.

For my understanding, OBS consumes just SRPMS? We implemented something like this for scratch builds in fedpkg/rpkg, here are the relevant PRs that landed in rpkg meanwhile:

https://pagure.io/rpkg/pull-request/548
https://pagure.io/rpkg/pull-request/557

Condensed: check if rpmautospec features are used and if so, preprocess the spec file and roll an SRPM from it.

OBS cannot consume SRPMs directly. It exclusively consumes Dist-Git-style "flat" trees of spec, patches, and sources (with the restriction that file names cannot start with . and no directories are permitted).

For example: https://build.opensuse.org/package/show/openSUSE:Factory/hello

This seems to work as intended, though it's a bit odd. Is this essentially bumping the release for every commit?

Yes, it works as intended, what FESCo asked from us to essentially avoid having to care about historical builds (which was the reason for the git tags in the prototype).

On IRC, you mentioned you wanted something which only added the plain version without release to the changelog entries for the commits where the version is bumped. Did I get this correctly?

Yeah. Because OBS rewrites the Release field, it's not useful for us to declare the version-release on each changelog entry because the value that OBS generates will almost certainly not match what rpmautospec does.

OBS cannot consume SRPMs directly. It exclusively consumes Dist-Git-style "flat" trees of spec, patches, and sources (with the restriction that file names cannot start with . and no directories are permitted).

How is this flat tree submitted to OBS, does one upload a tarball of it? Sorry for asking what must be a newbie question, but I've never worked with it.

For example: https://build.opensuse.org/package/show/openSUSE:Factory/hello

This seems to work as intended, though it's a bit odd. Is this essentially bumping the release for every commit?

Yes, it works as intended, what FESCo asked from us to essentially avoid having to care about historical builds (which was the reason for the git tags in the prototype).

On IRC, you mentioned you wanted something which only added the plain version without release to the changelog entries for the commits where the version is bumped. Did I get this correctly?

Yeah. Because OBS rewrites the Release field, it's not useful for us to declare the version-release on each changelog entry because the value that OBS generates will almost certainly not match what rpmautospec does.

Ahh. If my guess above is somehow valid, we could add parameters to support your use case: to not prepend the header defining %autorelease (as OBS will replace the Release: %autorelease … line anyway) and to only add version info to the changelog entries where the version was changed.

Log in to comment on this ticket.

Metadata