If we want to run CI on pull requests, we first need to scratch-build what's in those pull requests.
However, "forks/" namespace doesn't seem to be supported by Koji:
$ koji build --scratch f35 'git+https://src.fedoraproject.org/forks/msrb/rpms/abrt-java-connector.git#a6d02a57cf83759323315e05939b290c454b5839' Created task: 63687131 Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=63687131 Watching tasks (this may be safely interrupted)... 63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): free 63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): free -> FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs 0 free 0 open 0 done 1 failed 63687132 buildSRPMFromSCM (/forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs 63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839) failed
FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs
Would it be possible to add src.fedoraproject.org:/forks to the list of allowed SCMs? Enabling it for scratch-builds would be enough.
src.fedoraproject.org:/forks
Thanks :wink:
To my knowledge this is not possible currently.
koji has a list of allowed scms, but if we add forks to that it means people could make official builds from forks and then delete them or make changes that are no longer tracked. ;(
I've filed a koji RFE to ask for seperate allowed scms for scratch builds: https://pagure.io/koji/issue/2757
Metadata Update from @kevin: - Issue close_status updated to: Upstream - Issue status updated to: Closed (was: Open)
Metadata Update from @msrb: - Issue status updated to: Open (was: Closed)
Reopening as the upstream issue seems to be fixed now :balloon: :tada: :)
I would be great if we could enable this in https://koji.fedoraproject.org.
We will update this after f35 is out the door. :)
Metadata Update from @kevin: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-trouble, medium-gain, ops, unfreeze
Thanks! :)
Hum. I'm happy to add this, but the option is pretty confusing. I guess we need to enable policy on the hub in order to do this? and move it from builders, or at least have both?
@tkopecek How can we configure this in fedora koji? I looked at the upstream commits, but it's confusing to me if you have a min to help. ;)
You've to either disable builder-side configs and use policy instead (preferred way as it is unified and changes are limited to hub). Or leave everything as is and just add policy for this one type. Both types of policies can work in the same time (mostly for backward compatibility). For the first case:
all kojid.confs
allowed_scms_use_config=False allowed_scms_use_policy=True
policy at hub:
build_from_scm = # allow scratch build for anything from anywhere bool scratch :: allow # replace original kojid.confs lines - allow [use_common] srpm_build_command match scm_type GIT GIT+SSH && match scm_host pkgs.fedoraproject.org :: allow fedkg sources # is srpm_build_command is not used, default "make sources" is used match scm_type GIT GIT+SSH && match scm_host git.fedorahosted.org:/git/spin-kickstarts.git :: allow all :: deny
"match scm_type GIT GIT+SSH" is not needed if you don't want to limit the protocols.
Please double check in stage - I'm writing this from the head. For details: https://docs.pagure.org/koji/defining_hub_policies/#actions build_from_scm https://docs.pagure.org/koji/access_controls/#allowed-scms
build_from_scm
[backlog refinement] Policy needs to be created for koji. We just need to find spare cycles to do it.
[backlog refinement] Still waiting for spare cycles to create the koji policy.
I added the policy in https://pagure.io/fedora-infra/ansible/pull-request/1237 This will only enable the policy for staging koji, so we can test it first.
I forgot to update. This change is merged for the staging. Could you test it there to see if this works?
Metadata Update from @zlopez: - Issue priority set to: Waiting on Reporter (was: Waiting on Assignee)
@msrb Do you have time to test this on staging?
@zlopez Yes -- thanks! Sorry, I completely missed the previous comment.
Just to make sure I am doing the right thing -- this is the stage we are talking about, right? https://koji.stg.fedoraproject.org/koji/ :)
Yes. thats the place.
So... I don't know how to authenticate in staging :/
$ koji --topurl=https://kojipkgs.stg.fedoraproject.org/ --server=https://koji.stg.fedoraproject.org/kojihub --weburl=https://koji.stg.fedoraproject.org/koji build --scratch f39 'git+https://src.fedoraproject.org/forks/msrb/rpms/satyr.git#b78337aee68d146640d41abdc7814681944d3ff8' 2023-03-01 19:07:55,449 [ERROR] koji: (gssapi auth failed: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://koji.stg.fedoraproject.org/kojihub/ssllogin) Use following documentation to debug kerberos/gssapi auth issues. https://docs.pagure.org/koji/kerberos_gssapi_debug/ 2023-03-01 19:07:55,451 [ERROR] koji: GSSAPIAuthError: unable to obtain a session (gssapi auth failed: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://koji.stg.fedoraproject.org/kojihub/ssllogin) Use following documentation to debug kerberos/gssapi auth issues. https://docs.pagure.org/koji/kerberos_gssapi_debug/
staging has it's own account system: https://accounts.stg.fedoraproject.org You need to make an account there, or reset your password and then 'fkinit --staging' to get a stg kerberos ticket.
Works! https://koji.stg.fedoraproject.org/koji/taskinfo?taskID=96029461
Thank you ;)
Nice, so the next step is to move it to production. Let me prepare a PR for that, but we should wait till the freeze is over before merging it.
The PR is now open. Let's wait with merge till after freeze.
I'm a bit confused if that does exactly what we want.
I think part of my confusion is that we still have a bunch of policy on kojid.conf side on the builders.
Perhaps we could consolidate all that onto the hub config?
we defintely only want to let scratch builds use forks, not official builds.
There is a PR opened for this that should do what we want
Commit 8d99c12e relates to this ticket
I've pushed that out and after a few tweaks it seems to be working. ;)
So, I think this is all done, please let us know if you have any further issues with it.
Metadata Update from @kevin: - Issue close_status updated to: Fixed with Explanation - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.