Issue #6361: All write access on src.fedoraproject.org requires membership in packager group - fedora-infrastructure

https://src.fedoraproject.org/fork/petasis/rpms/fossil

Just hit this issue.

petasis commented 5 years ago

Me also. I want to make a PR to the fossil package, I have forked it here:

But I have been trying a whole day to understand why cloning with ssh does not work.
Now, I found this... :-)

There is progress on this, it's pending review at:
- https://pagure.io/fedpkg/pull-request/225
- https://pagure.io/rpkg/pull-request/322

PRs are merged. Fixed now?

@abitrolly the PR are merged but I don't think they have been released yet :)

I can sell you releasemeter that measures lag between feature merges and releases. =)

dustymabe commented 5 years ago

There is progress on this, it's pending review at:
- https://pagure.io/fedpkg/pull-request/225
- https://pagure.io/rpkg/pull-request/322

i took a gander at the PRs but it's not obvious.. what exactly is the fix?

We are allowing HTTPS-based pushing, for which users do not need ssh access to our systems, but instead use OpenID Connect/OAuth2 access tokens to authenticate pushes.
The PRs contain code to rpkg/fedpkg to retrieve and provide said tokens to git actions.

psss commented 5 years ago

Is there a plan to release a new package which we could test the functionality with? Are there any extra config steps required from the user to make this working? Thanks.

Everything is in place on the server end now, we are just waiting for another fedpkg/rpkg release with the changes.

@cqi is there any plan for a fedpkg release soon? Thanks.

cqi commented 5 years ago

Yes. There is a plan to release rpkg and fedpkg in this month.

Plan with tests could get it gets automatically released once all of them pass. )

filbranden commented 5 years ago

+1 I just hit this too.

https://docs.pagure.org/rpkg/releases/1.55.html
https://docs.pagure.org/fedpkg/releases/1.34.html

This is an essential feature and needs to be rolled about as a matter of priority. The poor workflow created without this is horrific and prevents efficient contribution.

franzh commented 5 years ago

+1 hit this as well,
If non packagers are not allowed to push code to src.fedoraproject.org, please disable the fork option, thanks!

cqi commented 5 years ago

rpkg-1.55 and fedpkg-1.34 have been released which support this. Please have a try.

rpkg-1.55 and fedpkg-1.34 have been released which support this. Please have a try.
https://docs.pagure.org/rpkg/releases/1.55.html
https://docs.pagure.org/fedpkg/releases/1.34.html

These needs to be rebuilt for EPEL please, so those who use RHEL and CentOS etc. can use them.

cqi commented 5 years ago

These needs to be rebuilt for EPEL please, so those who use RHEL and CentOS etc. can use them.

There is. https://bodhi.fedoraproject.org/updates/?packages=rpkg&status=testing

The reason I didn't mention that yet is because there is current a issue with pagure preventing non packagers from making forks.... but sure if you already have an existing fork it might work.

The reason I didn't mention that yet is because there is current a issue with pagure preventing non packagers from making forks.... but sure if you already have an existing fork it might work.

With the updated packages from 'epel-testing', the issue remains when attempting to clone a repo or fork of a repo.

Edited 5 years ago by kathenas

@kathenas with the new packages, non-packagers can clone their forks with the -a ("anonymous") flag, which will clone over HTTPS, and you can now push via that same clone URL as well.
So please make sure to clone your fork with the -a flag.

@kathenas with the new packages, non-packagers can clone their forks with the -a ("anonymous") flag, which will clone over HTTPS, and you can now push via that same clone URL as well.
So please make sure to clone your fork with the -a flag.

The on fork page 'Source GIT URLs' are misleading and fail, no matter what.

You can successfully clone a fork via HTTPS, doing:

fedpkg clone --anonymous forks/<your_fas_name>/rpms/<project_name>

Edited 5 years ago by kathenas

Could you maybe give us the full URLs?

Could you maybe give us the full URLs?

At the moment I am playing/testing: https://src.fedoraproject.org/fork/kathenas/rpms/liferea

Clone done with:

fedpkg clone --anonymous forks/kathenas/rpms/liferea

Below command will fail:

fedpkg clone --anonymous https://src.fedoraproject.org/forks/kathenas/rpms/liferea.git

Then we get to the fun stuff...

Make a change to the spec file.

Commit the above change.

'fedpkg push' will prompt for FAS username and password and display the following through that element.

[philwyett@ks-kenobi liferea]$ fedpkg push
Could not execute gitcred_get: init() got an unexpected keyword argument 'printfd'
Could not execute gitcred_get: init() got an unexpected keyword argument 'printfd'
Could not execute gitcred_erase: init() got an unexpected keyword argument 'printfd'
Could not execute gitcred_erase: init() got an unexpected keyword argument 'printfd'
fatal: Authentication failed for 'https://src.fedoraproject.org/forks/kathenas/rpms/liferea.git/'
Could not execute push: Command '['git', '-c', 'credential.helper=/usr/bin/fedpkg gitcred', '-c', 'credential.useHttpPath=true', 'push']' returned non-zero exit status 128
[philwyett@ks-kenobi liferea]$

It should not be asking for a userna,e/password.
However, this issue is known and is caused by fedpkg missing a requires on python-openidc-client >= 0.6.0.
So if you grab the new python-openidc-client from updates-testing, it should work.

https://src.fedoraproject.org/fork/kathenas/rpms/liferea

It should not be asking for a userna,e/password.
However, this issue is known and is caused by fedpkg missing a requires on python-openidc-client >= 0.6.0.
So if you grab the new python-openidc-client from updates-testing, it should work.

Added the app and FAS token grab worked.

Now able to fork, commit and PR.

Fork:

Commit:

https://src.fedoraproject.org/fork/kathenas/rpms/liferea/c/cc75359ff77bda4b405d956ef2185e34c16c2b69?branch=master

PR:

https://src.fedoraproject.org/rpms/liferea/pull-request/2

Now fantastic and contribution is made simple. :-D

Today. @vstinner (not a packager) was surprised by the fact the ha was able to fork a repo, but not to push to his fork.

I propose, as well as @franzh did: if non packagers are not allowed to push code to src.fedoraproject.org forks, please disable the fork option. This is really a confusing setup.

Thank You.

@churchyard This is in fact fixed, we just haven't announced it yet.

They definitely should have been able to push to their fork via fedpkg push (provided fedpkg is new enough and they are using a graphical session that has a browser running).

I'll try and get things alined to announce this as soon as I can...

The procedure is documented in https://fedoraproject.org/wiki/CI/Pull_Requests for the moment, but we'll need to document it elsewhere and announce it for sure :)

Also, the documentation doesn't work on Fedora 29, where fedpkg uses Python 3.

@churchyard do you have a recent enough version of fedpkg and python3-openidc-client installed?
And what does "doesn't work" mean?

https://fedoraproject.org/w/index.php?title=CI%2FPull_Requests&type=revision&diff=525804&oldid=523133

"deson't work" in the most recent comment relates to the documentation that mentions python2-openidc-client. I'm working on updating it to work both on F29+ and F28-.

I don't need this HTTP pushing for myself. But to answer your question, I have python3-openidc-client-0.6.0-3.20180605gitcd8d91c.fc29.noarch

Just wanted to report that this is still not working on Fedora 29 (I'm not a packager).

All required packages are installed, as per https://fedoraproject.org/wiki/CI/Pull_Requests instructions:

$ rpm -q python3-openidc-client python3-rpkg fedpkg                                      
python3-openidc-client-0.6.0-3.20180605gitcd8d91c.fc29.noarch
python3-rpkg-1.57-6.fc29.noarch
fedpkg-1.36-2.fc29.noarch

$ sudo dnf update python3-openidc-client fedpkg python3-rpkg --enablerepo=updates-testing
Last metadata expiration check: 0:11:48 ago on Tue 19 Feb 2019 01:35:01 PM -03.
Dependencies resolved.
Nothing to do.
Complete!

I cloned from my own fork with fedpkg clone -a forks/mopsfelder/rpms/qemu command and section [credential] in .git/config is like this:

[credential]
    helper = /usr/bin/fedpkg gitcred
    useHttpPath = true

When I try to push with git push command:
- it opens browser at https://id.fedoraproject.org/openidc/Authorization;
- I enter my FAS credential and it authenticates correctly and shows You can close this window and return to the CLI message;
- then it still asks for username and password on the terminal:
Username for 'https://src.fedoraproject.org/forks/mopsfelder/rpms/qemu.git':
Password for 'https://mopsfelder@src.fedoraproject.org/forks/mopsfelder/rpms/qemu.git':
- and lastly the authentication error message:
fatal: Authentication failed for 'https://src.fedoraproject.org/forks/mopsfelder/rpms/qemu.git/'

@churchyard @pingou Am I missing anything here?

All packagers I have mentored so far used external PRs rather than this, so I assume this is till broken (or at least overcomplicated). However as a packager I have no way to test this other than creating artificial FAS accounts.

It definitely works ok here. There's no need to make other fas accounts to test, just clone with -a (https) and it will use the token method.

@mopsfelder Do you have any global ~/.git/config that could be getting in the way?

If you do a 'kinit mopsfelder@FEDORAPROJECT.ORG' first does it work?

@kevin

I don't have ~/.git/config but I do have ~/.gitconfig and there is no [credential] section in ~/.gitconfig.

I tried with kinit mopsfelder@FEDORAPROJECT.ORG but it didn't improve the behaviour, i.e.: git push prompted for the username/password as before and authentication failed.

I could see a Kerberos ticket was created for me with klist command.

Edited 5 years ago by mopsfelder

I kevin was referring to the .git/config file within the git repo/clone (the ~/.gitconfig is a configuration file for git across all git repos, so you probably don't want that setting there).

@kevin @pingou

Thanks for clarifying, Pierre.

Here is the content of .git/config after a fresh clone of the fork with fedpkg clone -a forks/mopsfelder/rpms/qemu command:

➜ cat .git/config 
[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = https://src.fedoraproject.org/forks/mopsfelder/rpms/qemu.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
    remote = origin
    merge = refs/heads/master
[credential]
    helper = /usr/bin/fedpkg gitcred
    useHttpPath = true
[bz]
    default-tracker = bugzilla.redhat.com
    default-product = Fedora
    default-version = rawhide
    default-component = qemu
[sendemail]
    to = qemu-owner@fedoraproject.org

Is [credential] section as expected? Anything wrong popping eyes out?

Is it possible to verify on server side if there is anything wrong with my user settings? I'm running out of ideas on my side.

I did appreciate your help until now!

scfc commented 5 years ago

Just as another data point: It doesn't work for me either. Without a kinit token, in a repository cloned with fedpkg clone -a forks/scfc/rpms/python-sqlalchemy, git push opens a page in Firefox that says the "transaction expired":

If I click on "Try to login again", I get shown that page once more (!), then after clicking again on "Try to login again", I get a login screen with "None wants to use your Fedora Account System (FAS) credentials":

Logging in gets to me this page:

This happens regardless whether I'm logged into FAS in my browser or not.

With kinit scfc@FEDORAPROJECT.ORG, I immediately get a "400 - Bad request/Invalid transaction id":

(If someone wants to debug this interactively with me on IRC, please propose a time convenient for you.)

Are either of you blocking cookies or using extensions that would block cookies?

Can you try:

a) Allow cookies from *.fedoraproject.org

b) If you have such cookies now, can you delete them and try and push again?

scfc commented 5 years ago

I block one specific non-Fedora domain (in three forms: "http://domain.com/", "http://www.domain.com/", "https://www.domain.com/"), but allow all other cookies. I deleted all cookies with "fedora" in their name, and nothing changed.

But then I tried accessing the URL in a different Firefox profile (also via a proxy on another IP if that matters), et voilà, it worked the first time and I could push successfully to my fork.

I'll try to pinpoint whatever difference between the two profiles causes that, but until then I'll just set the environment variable $BROWSER (https://docs.python.org/2/library/webbrowser.html) to launch the "working" profile.

Thanks!

@kevin

I cleaned up all cookies, explicitly allowed cookies from [*.]fedoraproject.org in Chrome but no luck.

Tried other browser by exporting BROWSER environment variable to firefox but got no luck too.

Edited 5 years ago by mopsfelder

@scfc Glad it worked for you!

Can you please share the output of rpm -q python3-openidc-client python3-rpkg fedpkg so I can compare with mine? Also, can you share the version of your working browser too?

Thank you.

scfc commented 5 years ago

@mopsfelder: I use Fedora 28, so fedpkg-1.36-2.fc28.noarch, python2-rpkg-1.57-6.fc28.noarch (instead of python3-rpkg) and python3-openidc-client-0.6.0-1.20180605gitcd8d91c.fc28.noarch. My working browser is Firefox 65.0 (firefox-65.0-4.fc28.x86_64).

Note that I had to authorize only once; since then (so far) I can just happily fedpkg push, so it is worth to persist :-).

I would create a new Firefox profile, write a shell wrapper à la firefox -P new_profile "$@", then set $BROWSER to the shell wrapper and try that.

@mopsfelder Is this working for you now? I'd try the new profile or perhaps even a new local user? That should rule out local user config...

@kevin thanks for the suggestion.

I tried the following today:

changed my FAS password
created new fork: https://src.fedoraproject.org/fork/mopsfelder/rpms/python-sqlalchemy
created new local user on my laptop
cloned the new fork (logged in as new user)

All things fresh. Same issue: fatal: Authentication failed for 'https://src.fedoraproject.org/forks/mopsfelder/rpms/python-sqlalchemy.git/'

I'm inclined to think there might be something wrong with my FAS account or something else on server side.

@mopsfelder You did clone the fork with 'fedpkg clone -a' right?

@kevin Thanks for reaching out.

For the record, some days ago I changed my password to a new one without the special character $ and it magically worked today, i.e.: I was able to git push to my own fork.

Could that be client code trying to sanitize or expand the $ or server not liking it at all?

Shouldn't be... but glad you got it working!

How does this look for an announcement:

Subject: announcing https pushing to dist-git/src.fedoraproject.org                                                     

Greetings.

I'm happy to announce that you can now use https to push commits                                                        
to src.fedoraproject.org. You will need to use 'fedpkg clone -a'                                                        
and have a session with a running browser to do the initial
authentication, but after that everything should be transparent.                                                        

Users who are not part of the 'packager' group can now use
this to push to forks of projects to create pull requests.

Please see https://fedoraproject.org/wiki/Infrastructure/HTTPS-commits                                                  
for more details.

We hope this support will allow more community members to
contribute along with allowing more flexability to maintainers
who don't always have ssh access.

zbyszek commented 5 years ago

Hooray!

Subject: announcing https pushing to dist-git/src.fedoraproject.org

I'd change this to "announcing pushing to dist-git/src.fedoraproject.org for non-packagers".
The fact that it's over https is something of a low-level detail, and what this enables is more interesting.

Also s/https/HTTPS/g.

sobek commented 5 years ago

I would change:
flexability to flexibility
ssh to SSH