#9728 [rfe] src.fedoraproject.org:/forks/... is not in the list of allowed SCMs
Closed: Fixed with Explanation 10 months ago by kevin. Opened 3 years ago by msrb.

Describe what you would like us to do:


If we want to run CI on pull requests, we first need to scratch-build what's in those pull requests.

However, "forks/" namespace doesn't seem to be supported by Koji:

$ koji build --scratch f35 'git+https://src.fedoraproject.org/forks/msrb/rpms/abrt-java-connector.git#a6d02a57cf83759323315e05939b290c454b5839'
Created task: 63687131
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=63687131
Watching tasks (this may be safely interrupted)...
63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): free
63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): free -> FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs
  0 free  0 open  0 done  1 failed
  63687132 buildSRPMFromSCM (/forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839): FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs

63687131 build (f35, /forks/msrb/rpms/abrt-java-connector.git:a6d02a57cf83759323315e05939b290c454b5839) failed

FAILED: BuildError: src.fedoraproject.org:/forks/msrb/rpms/abrt-java-connector.git is not in the list of allowed SCMs

Would it be possible to add src.fedoraproject.org:/forks to the list of allowed SCMs? Enabling it for scratch-builds would be enough.

Thanks :wink:

When do you need this to be done by? (YYYY/MM/DD)

No rush. We have a workaround, but it can be fragile (building SRPMs locally and then submitting them to Koji).


To my knowledge this is not possible currently.

koji has a list of allowed scms, but if we add forks to that it means people could make official builds from forks and then delete them or make changes that are no longer tracked. ;(

I've filed a koji RFE to ask for seperate allowed scms for scratch builds:
https://pagure.io/koji/issue/2757

Metadata Update from @kevin:
- Issue close_status updated to: Upstream
- Issue status updated to: Closed (was: Open)

3 years ago

Metadata Update from @msrb:
- Issue status updated to: Open (was: Closed)

2 years ago

Reopening as the upstream issue seems to be fixed now :balloon: :tada: :)

I would be great if we could enable this in https://koji.fedoraproject.org.

We will update this after f35 is out the door. :)

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: low-trouble, medium-gain, ops, unfreeze

2 years ago

Hum. I'm happy to add this, but the option is pretty confusing. I guess we need to enable policy on the hub in order to do this? and move it from builders, or at least have both?

@tkopecek How can we configure this in fedora koji? I looked at the upstream commits, but it's confusing to me if you have a min to help. ;)

You've to either disable builder-side configs and use policy instead (preferred way as it is unified and changes are limited to hub). Or leave everything as is and just add policy for this one type. Both types of policies can work in the same time (mostly for backward compatibility). For the first case:

all kojid.confs

allowed_scms_use_config=False
allowed_scms_use_policy=True

policy at hub:

build_from_scm =
    # allow scratch build for anything from anywhere
    bool scratch :: allow
    # replace original kojid.confs lines - allow [use_common] srpm_build_command
    match scm_type GIT GIT+SSH && match scm_host pkgs.fedoraproject.org :: allow fedkg sources
    # is srpm_build_command is not used, default "make sources" is used
    match scm_type GIT GIT+SSH && match scm_host git.fedorahosted.org:/git/spin-kickstarts.git :: allow
    all :: deny

"match scm_type GIT GIT+SSH" is not needed if you don't want to limit the protocols.

Please double check in stage - I'm writing this from the head. For details:
https://docs.pagure.org/koji/defining_hub_policies/#actions build_from_scm
https://docs.pagure.org/koji/access_controls/#allowed-scms

[backlog refinement]
Policy needs to be created for koji. We just need to find spare cycles to do it.

[backlog refinement]
Still waiting for spare cycles to create the koji policy.

I added the policy in https://pagure.io/fedora-infra/ansible/pull-request/1237 This will only enable the policy for staging koji, so we can test it first.

I forgot to update. This change is merged for the staging. Could you test it there to see if this works?

Metadata Update from @zlopez:
- Issue priority set to: Waiting on Reporter (was: Waiting on Assignee)

a year ago

@msrb Do you have time to test this on staging?

@zlopez Yes -- thanks! Sorry, I completely missed the previous comment.

Just to make sure I am doing the right thing -- this is the stage we are talking about, right?
https://koji.stg.fedoraproject.org/koji/
:)

Yes. thats the place.

So... I don't know how to authenticate in staging :/

$ koji --topurl=https://kojipkgs.stg.fedoraproject.org/ --server=https://koji.stg.fedoraproject.org/kojihub  --weburl=https://koji.stg.fedoraproject.org/koji build --scratch f39 'git+https://src.fedoraproject.org/forks/msrb/rpms/satyr.git#b78337aee68d146640d41abdc7814681944d3ff8'
2023-03-01 19:07:55,449 [ERROR] koji: (gssapi auth failed: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://koji.stg.fedoraproject.org/kojihub/ssllogin)
Use following documentation to debug kerberos/gssapi auth issues. https://docs.pagure.org/koji/kerberos_gssapi_debug/
2023-03-01 19:07:55,451 [ERROR] koji: GSSAPIAuthError: unable to obtain a session (gssapi auth failed: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://koji.stg.fedoraproject.org/kojihub/ssllogin)
Use following documentation to debug kerberos/gssapi auth issues. https://docs.pagure.org/koji/kerberos_gssapi_debug/

staging has it's own account system: https://accounts.stg.fedoraproject.org
You need to make an account there, or reset your password and then 'fkinit --staging' to get a stg kerberos ticket.

Nice, so the next step is to move it to production. Let me prepare a PR for that, but we should wait till the freeze is over before merging it.

The PR is now open. Let's wait with merge till after freeze.

I'm a bit confused if that does exactly what we want.

I think part of my confusion is that we still have a bunch of policy on kojid.conf side on the builders.

Perhaps we could consolidate all that onto the hub config?

we defintely only want to let scratch builds use forks, not official builds.

There is a PR opened for this that should do what we want

Commit 8d99c12e relates to this ticket

I've pushed that out and after a few tweaks it seems to be working. ;)

So, I think this is all done, please let us know if you have any further issues with it.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

10 months ago

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog
Related Pull Requests
  • #1365 Merged 10 months ago