#9801 Allow build_from_srpm for fNN-coreos-continuous Koji target
Opened 5 months ago by jlebon. Modified 12 days ago

  • Describe the issue

We're trying to set up Packit to do continuous Koji builds in the fNN-coreos-continuous tags for some of our upstream repos, but Packit is hitting a policy violation when submitting to Koji:

Result | ActionNotAllowed: policy violation (build_from_srpm)

https://koji.fedoraproject.org/koji/taskinfo?taskID=52957745
https://github.com/packit/packit-service/issues/842

I think this policy makes sense for the production/release targets, though for the fNN-coreos-continuous tags, we'd like more flexibility. These aren't builds that we would submit to Bodhi or expose to users. They're strictly for use by CoreOS developers and CI.

  • When do you need this? (YYYY/MM/DD)

Not urgent, though ideally within a week or so?

  • When is this no longer needed or useful? (YYYY/MM/DD)

No end time in sight.

  • If we cannot complete your request, what is the impact?

We wouldn't be able to iterate on our build process and tools as fast as we'd like while still supporting multi-arch.


We are currently in freeze for F33 release, so I don't see this happening until after that is done unless there is a freeze exception.

We are also needing to track down disk usage from new features to see how much disk space is expected to be used per day, how long that data needs to be kept until it is free, and how much growth over time is expected. Could you provide some rough estimates for that.

Metadata Update from @smooge:
- Issue tagged with: medium-gain, medium-trouble, ops

5 months ago

One way to "go around" this would be to push the update to dist-git and build from there - which would increase the volume of data even further (there would be a new tarball in lookaside for every branch push). I presume such solution is unacceptable here.

@smooge once F33 is out, what would be the process to progress here further? FESCO?

I think @smooge has not idea and it should be addressed to @kevin
[sorry I did this without looking to see why this was addressed to me. I was mainly triaging tickets that meeting and putting things in so kevin and mohan could focus on thinking.]

So, I am fine doing this, but I wish we had a way to make sure any builds made this way never ever went to users via other tags. :(

One idea @mohanboddu had was perhaps changing the dist tag for that tag? Then these builds would have a different dist tag we could check against and not allow being tagged anywhere else?

Thoughts?

One idea @mohanboddu had was perhaps changing the dist tag for that tag? Then these builds would have a different dist tag we could check against and not allow being tagged anywhere else?

Hmm yeah, that makes sense to me. Like a .fXX.coreos.ci dist tag or something.

Packit offers hooks so we could just sed that into the spec file before it starts the build. Though... is there a way to embed this in the canonical spec file so that it's conditional on the Koji target itself? That way it's less error-prone and will work for manual fedpkg build --target=fXX-coreos-continuous too.

So, f33 is out now. :)

Circling back to this, does anyone know of a way to have spec file conditionals based on the Koji target? I guess the way we could do this is have a special RPM in the buildroot which installs an RPM macro, though that seems heavyweight.

I'm personally not aware of koji setting any env vars during build which would enable your package to figure out where it's being built. I was checking if koji/fedpkg support setting with/without conditionals via CLI but they don't :/

I can see three feasible options:
1. do what you suggest, have an RPM in the buildroot which defines RPM macros so that you can change the %release tag - yes, it's heavy, but this is how module builds are implemented anyway
2. have 2 spec files: one for production builds, one for the CI builds and try to keep them in sync
3. do this via packit and sed the %release, again, as you point out

We can set dist in koji tags extras info... just pick a dist and tell us what tag and we can set it.

Look for example at the eln tag:

koji taginfo eln
Tag: eln [22492]
Arches: 
Groups: appliance-build, build, livecd-build, livemedia-build, srpm-build
Tag options:
  mock.new_chroot : 0          [f35]
  mock.package_manager : 'dnf' [f35]
Inheritance:
  0    .... f35 [37068]
➜  ansible git:(main) koji taginfo eln-build
Tag: eln-build [22493]
Arches: i686 x86_64 aarch64 ppc64le s390x
Groups: appliance-build, build, livecd-build, livemedia-build, srpm-build
Tag options:
  mock.new_chroot : 0          [f35]
  mock.package_manager : 'dnf' [f35]
  rpm.macro.dist : ('%{!?distprefix0:%{?distprefix}}%{expand:%{lua:for i=0,9999 do '
 'print("%{?distprefix" .. i .."}") '
 'end}}.eln%{eln}%{?with_bootstrap:~bootstrap}')
  rpm.macro.eln : 110
  rpm.macro.rhel : 10
This tag is a buildroot for one or more targets
Current repo: repo#2976002: 2021-02-11 00:22:41.386753+00:00
Targets that build from this tag:
  eln-rebuild
  eln
  eln-candidate
Inheritance:
  0    .... eln [22492]
  5    .... f35-build [37078]

We can set dist in koji tags extras info... just pick a dist and tell us what tag and we can set it.

Ohh, that's great. And sorry looking back now I realize that's what you were suggesting in https://pagure.io/releng/issue/9801#comment-696921, but I mistakenly understood you meant doing this in the spec file directly.

So yes, that looks like a great solution. Can we have the dist tag for fNN-coreos-continuous be e.g. .fc%{fedora}.coreos.ci? And then is there more work needed to have packages with that dist tag never somehow end up in Bodhi, or is that filter already on by default?

We don't have any filter currently. Ideally we would just have something that would tell us if one of those packages got into an update compose, but I am not sure where we could put that test. ;(
I guess we could look at bodhi, but not sure how easy it will be to do there.

Hmm, I wonder if we could use Koji ACL rules to do something pretty close. E.g. a rule which prevents packages tagged with fNN-coreos-continuous from being untagged, and another rule which prevents a package to be tagged in both fNN-coreos-continuous and fNN-updates. Would that be enough to prevent someone from accidentally submitting a Bodhi update for that package?

Login to comment on this ticket.

Metadata
Boards 1
Ops Status: Backlog