This is a formal proposal following the discussion on devel mailing list [1].
Accordingly to Ben Cotton [2] there are currently 2.453 users in the packager group and well over a thousand of them have not submitted a single koji build for over a year. Noggin web interface counts them to 2.000 (maybe because of a limited query?) [3].
packager
I've wrote a simple script [4] which queries src.fedoraproject.org and datagrepper to identify packagers which have not showed any activities in pagure or in any Fedora activity monitored through datagrepper. To my count, src.fedoraproject.org shows 1.787 users in the packager group and 104 users have no activity within Fedora through the last year [5].
If my guesses are correct, that means that (2453 - 1787 =) 666 packagers have never interacted with src.fedoraproject.org. In total we have (666 + 104=) 770 users with commit access to our repositories which show no activity. We don't know if they lost interest in Fedora or passed away or something else. The most important thing is that we don't know if the email address they provided upon registeration is still valid and still managed by them.
There are some service providers that free email adresses if they're not used for some time. That means that anyone can claim them. If a packager account was using one of those released email, anyone can reset the former user password and gain commit as a packager and possibly push malware into Fedora sources. That applies also to custom domains emails and possibly to email from organizations (luckily, gmail seems to never release older emails since they're bounded to Google accounts).
My proposal is to periodically run a check on packager activities to show if they're still engaged in Fedora. If no activity is detected within a year, try to contact them by their account email and if no reply is received after a proper delay (1 month) proceed by removing them from the packager group. Not that I'm not proposing to delete their account, just remove privileges. If the user shows up again after some time they can still use their old account (but in order to regain packager privileges we should find a way to make them prove their really who they claim to be).
Also, I'd like the current "Unresponsive maintainer policy" to be enhanced so that aside orphaning the unresponsive user packages, we also remove them from the packager group (not sure if it is already done by the process).
In my opinion this will enhance security and also avoid most of the "Unresponsive maintainer" requests to be handled, by identifing in advance problematic users.
[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/R3T4LB2OSGWUOSV5CMB6GZ6FAQ4AWB6U/ [2] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/6MXPV6UWY5SCHUB6ZS5S6R22BLNLMQR7/ [3] https://accounts.fedoraproject.org/group/packager/ [4] https://mattia.fedorapeople.org/inactive-packagers/find_inactive_packagers.py [5] https://mattia.fedorapeople.org/inactive-packagers/inactive_users.txt
If the user shows up again after some time they can still use their old account (but in order to regain packager privileges we should find a way to make them prove their really who they claim to be).
I think we should specify how to initiate the procedure here: my suggestion on the mailing list was to file a ticket in sponsors (https://pagure.io/packager-sponsors/new_issue). The default text there would need to be updated to mention restoration of a dormant account.
I do not think that we need to specify the rest of the procedure. I don't think that such cases will be very common, and we can use different strategies. E.g. if the user has another well-known mail address, we could just send a simple query there. Or if the user is a RedHat employee who moved to other tasks but now wants to come back to packaging, we can easily check their status internally. Also, if the user has 2FA enabled, we can probably restore the account without further ado. In the worst-case scenario where somebody is actually highjacking an account, we'll at least have a public log of activity.
proceed by removing them from the packager group.
I think this is a very bad idea. We shouldn't offend people. If you remove "packager" status, this user will probably leave Fedora, because they will need to be sponsored again. Some people are waiting for sponsorship for months.
Maintainers are the main value of the distribution. We shouldn't offend and forcing them to leave Fedora.
If no activity is detected within a year, try to contact them by their account email and if no reply is received after a proper delay (1 month)
And you make life easier for potential hackers. They will simply copy this email and send it to all Fedora contributors. Some of them will follow the link and the hackers will get a lot of real working accounts, but most will simply reject them as phishing attempts.
Accordingly to Ben Cotton [2] there are currently 2.453 users in the packager group and well over a thousand of them have not submitted a single koji build for over a year.
Some maintainers don't have recent commits or Koji builds because other Fedora contributors maintain their packages.
To my count, src.fedoraproject.org shows 1.787 users in the packager group and 104 users have no activity within Fedora through the last year
You don't need to be logged into src.fedoraproject.org or account.fedoraproject.org to maintain packages. You can simply make commits and send them to Bodhi using CLI tools.
As I also mentioned on the devel thread, this is not true. You need to log into src.fedoraproject.org at least once to sync group memberships to it (including "packager"), and you need to set your SSH key in accounts.fedoraproject.org. Without doing both those things, pushes to dist-git repos are rejected. So if both systems don't know a user, they aren't even yet able to push changes even if they wanted to.
As I also mentioned on the devel thread, this is not true. You need to log into src.fedoraproject.org at least once to sync group memberships to it (including "packager"), and you need to set your SSH key in accounts.fedoraproject.org.
Yes, but only once. Later you can use CLI tools.
proceed by removing them from the packager group. I think this is a very bad idea. We shouldn't offend people. If you remove "packager" status, this user will probably leave Fedora, because they will need to be sponsored again. Some people are waiting for sponsorship for months. Maintainers are the main value of the distribution. We shouldn't offend and forcing them to leave Fedora.
So, a user finally get into the packager group. Cool. Then they doesn't show any activity as a packager in a year (checked by activity in pagure), nor any activity in Fedora globally in a year (checked by datagrepper), nor they find some time to reply to an email checking their status. They just want to be in the packager group because "it's cool" and they're going to be offended if we remove them from packagers? Is there another solution to avoid having possibly compromised accounts with commit privileges being active forever in the distribution?
If no activity is detected within a year, try to contact them by their account email and if no reply is received after a proper delay (1 month) And you make life easier for potential hackers. They will simply copy this email and send it to all Fedora contributors. Some of them will follow the link and the hackers will get a lot of real working accounts, but most will simply reject them as phishing attempts.
My idea is to ask the user to show any activity in Fedora. Looking at datagrepper we can re-check the status and act accordingly. No links to click in an email. Or just ask them to login into noggin or pagure and confirm their email address.
Accordingly to Ben Cotton [2] there are currently 2.453 users in the packager group and well over a thousand of them have not submitted a single koji build for over a year. Some maintainers don't have recent commits or Koji builds because other Fedora contributors maintain their packages.
They should still show any activity in datagrepper. Or find some time to read a yearly email.
To my count, src.fedoraproject.org shows 1.787 users in the packager group and 104 users have no activity within Fedora through the last year You don't need to be logged into src.fedoraproject.org or account.fedoraproject.org to maintain packages. You can simply make commits and send them to Bodhi using CLI tools.
Yes, but even if they commit via CLI, pagure shows their activity. Just look at https://src.fedoraproject.org/api/0/user/{yourusername}/activity/stats and you'll see it counts commit done via CLI.
https://src.fedoraproject.org/api/0/user/{yourusername}/activity/stats
Then they doesn't show any activity as a packager in a year (checked by activity in pagure), nor any activity in Fedora globally in a year (checked by datagrepper), nor they find some time to reply to an email checking their status.
Maybe other Fedora contributors maintain their packages. Do you want to delete such users from Fedora?
They just want to be in the packager group because "it's cool" and they're going to be offended if we remove them from packagers?
If we remove them from packagers, they will need to deal again with getting sponsored procedure.
Is there another solution to avoid having possibly compromised accounts with commit privileges being active forever in the distribution?
I don't remember such precedents in the entire history of Fedora.
Instead of removing, we can simply lock such accounts. If the maintainer returns, they can log in again and confirm account unlock with email.
Then they doesn't show any activity as a packager in a year (checked by activity in pagure), nor any activity in Fedora globally in a year (checked by datagrepper), nor they find some time to reply to an email checking their status. Maybe other Fedora contributors maintain their packages. Do you want to delete such users from Fedora?
No. Probably I wasn't clear enough translating my though into English. I don't propose to remove their account or lock it down (which would be better in a security viewpoint, but I knew this would have encountered severe opposition), just remove the association with the packager group.
They just want to be in the packager group because "it's cool" and they're going to be offended if we remove them from packagers? If we remove them from packagers, they will need to deal again with getting sponsored procedure. Is there another solution to avoid having possibly compromised accounts with commit privileges being active forever in the distribution? I don't remember such precedents in the entire history of Fedora. Instead of removing, we can simply lock such accounts. If the maintainer returns, they can log in again and confirm account unlock with email.
The problem is that if their email is "compromised" (meaning that maybe it was not used for a long period of time and someone else claimed it), this will have no effect at all to the security.
Also, consider that I'm not talking to look only to pagure activity, but also to look at datagrepper activity. This should identify only users which really left Fedora, for one reason or another.
We can (should) also add a check about activity in the mailing lists to be 1000% sure.
If we remove users from the packager group, we need to figure out what to do with their packages as well. I suppose we could remove the user from all the packages, as long as we log what we did, so if they come back immediately after we do it, we have a way to revert.
I've tried to develop my script a little more and uploaded to https://pagure.io/find-inactive-packagers
It now:
If a user is the main POC for a package, removing the users from the packager group will need to orphan their packages. This is no different than the already in place "Unresponsive maintainer policy". In fact, I was thinking it would be just simpler to open a unresponsive maintainer ticket for inactive packagers: if they react to the ticket, datagrepper will see their activity in Bugzilla and the next script run will filter out those users from being removed.
We can just use the "Unresponsive maintainer policy" by adding to it:
[1] https://mattia.fedorapeople.org/inactive-packagers/find_inactive_packagers_2022_02_13.log
Google doesn't allow you to register an account with a name previously deleted for any reason. Other public mail hostings should have the same policy.
a returning user doesn't need to go through the sponsoring process, they can file a ticket as suggested by @zbyszek and have their identity verified
Looks better, but what's to stop a hacker from filling a ticket and taking back control over packages? Do we have a plan, how how the returning maintainer can prove that they are the same user?
Given the amount of discussion still happening on this, could we keep discussing on list and leave this ticket for updating the proposal and voting?
The discussion on the mailing list has died down. There was a lot of useful input in the thread. @mattia, how should we proceed here? Is the procedure described in the initial proposal + https://pagure.io/fesco/issue/2759#comment-781091 ready for approval?
I think the idea to use the Unresponsive maintainer process is not feasible, because on a later thought I recognized that it is not relevant for packagers which aren't main admin for any package. So, I think a separate policy is needed.
There are several things to tweak, but overall my final proposal would be as follow: 1- run the script every two months; it will provide two lists, 'inactive packagers' and 'still inactive packagers'. 2- send an email to the inactive packagers asking them to show any activity, so that the next run will not list them as still inactive. 3- remove the still inactive packagers from the packagers group (and orphan any package where they're primary admins). 4- returning users that may have been removed from the packagers group can re-apply to the group by filing a ticket into the packager-sponsors repo.
Apart from step 1, I think the other steps should be handled mostly manually. For example, sending the email through the devel mailing list to all inactive packagers may cause the user to reply to the sender, rather to the mailing list, so the script will not see the user activity at the next run. Or, at the contrary, a user may reply "I'm not interested anymore, remove me from the packager group" and the script will count that as activity and list out the user from the next run results...
I think this is fine. We'll have a bunch of manual work the first time this is done, but then it should slow down to a trickle.
I have the following idea though: use a fesco ticket in 2. We can script this, and it'll leave a public trail, maybe also allowing other people to make comments or suggest another way to contact the packager.
3- remove the still inactive packagers from the packagers group (and orphan any package where they're primary admins).
And to be clear, also remove them from co-maintaining / commit on any package too right?
The first run would then be like ~275 fesco tickets. Thats quite the firehose.
So, that means a packager has to do some activity every 4 months or be nagged to reply to the script? Perhaps this new proposal should move back to the list for feedback again?
3- remove the still inactive packagers from the packagers group (and orphan any package where they're primary admins). And to be clear, also remove them from co-maintaining / commit on any package too right?
Yes, right. I should be able to write down a PR against fesco-docs with all the formalities in the next days. I'll link it here when done.
I have the following idea though: use a fesco ticket in 2. We can script this, and it'll leave a public trail, maybe also allowing other people to make comments or suggest another way to contact the packager. The first run would then be like ~275 fesco tickets. Thats quite the firehose.
ahem... the last time I run the script (16/02 see https://mattia.fedorapeople.org/inactive-packagers/ ) it detected 875 packagers inactive in the last year...
However, I think the idea of a Fesco ticket is the right one. We can tag the username in the ticket, so that the user will receive the communication through the correct email (specified in their profile); I think I can make the script check the user activity on pagure.io just like src.fp.org, so if they reply to the ticket the script will detect their activity. However, the ticket will need to be handled manually, for example in the case where the user replies "go ahead and remove me from the packager group".
So, this workflow will have to open over 800 tickets (only for the first run) on fesco tracker... is it ok?
No, the first script run will list the user with no packaging activity, nor any activity in BZ, Bodhi and mailing lists, in the last 12 months. We will try to contact them. Then we will run the script again after 2 more months and have a list of packagers that didn't reply / didn't show any activity again.
That means a 12 + 2 months period of inactivity and no reply to the ping.
Pull request for FESCo-docs: https://pagure.io/fesco/fesco-docs/pull-request/61
Ok folks, this seems to be somehow forogtten. I am restarting the week counter from now and please vote. I will mention this in today's meeting.
Metadata Update from @churchyard: - Issue tagged with: meeting
+1
(I think that once we actually try to make an implementation, there might be some things we learn and the wording might need some tweaks, but this version is good enough.)
I agree.
Metadata Update from @churchyard: - Issue untagged with: meeting
I'm +1 with the proviso that we wait until we have the scripts implemented to add this policy and announce it.
One slight possible change: instead of a returning packager opening up a packager-sponsors ticket to regain their packager group membership, how about they instead re-open the closed fesco ticket pinging them? Or would that confuse the scripting too much?
Also, it might be nice to post again to devel this current proposal, since I think it's changed since it was last discussed there.
TBH this seems too confusing from the user side. Opening a ticket is straightforward IMO.
Ack, it's good to announce those things via as many channels as possible :). Last week a collegue was surprised that package gets removed from the distribution if it doesn't build for two Fedora versions...
I posted back in the thread asking folks to look at this proposal.
Metadata Update from @sgallagh: - Issue tagged with: stalled
For reference, this was Message-ID: <20220419155117.7cba5ln4cdfp3lny@logain.scrye.com>. There have been no replies to that unfortunately.
Message-ID: <20220419155117.7cba5ln4cdfp3lny@logain.scrye.com>
https://pagure.io/fesco/fesco-docs/pull-request/61
I re-read this again, and I think we should go ahead with the proposal. Even if it's not perfect, we shouldn't let perfect be the enemy of good. Having a policy is certainly better than the current state where we have hundreds of dead accounts. I'll attach it to the next meeting agenda.
@mhayden, @music this is from before you joined FESCo, please see the linked PR.
Metadata Update from @zbyszek: - Issue untagged with: stalled - Issue tagged with: meeting
I was waiting for the new FESCo members to sit in their chairs before asking if there's something else I should do to move ahead this proposal...
So, to recap: - the PR to FESCo docs is at https://pagure.io/fesco/fesco-docs/pull-request/61 - the script to detect inactive packagers is at https://pagure.io/find-inactive-packagers (I've developed it a little more, so it can now auto open tickets to a pagure repository to ping inactive packagers)
I'm not sure if I'll be able to attend at the next FESCo meeting, though... I can try to quit early from work, if needed.
The returning part is still a bit fuzzy I think...
"Providing that they can prove their identity through other methods than the email used in their account, they can open a ticket in the link:https://pagure.io/packager-sponsors/new_issue/[packager-sponsors] tracker and ask for their status to be restored."
Who do they prove this to? In the ticket? Some of the things we allow for identiy when people can't login to their account anymore: can sign a email with gpg key associated with the account can use the ssh private key associated with the public key in their account (ie, can login to fedorapeople) if Red Hat employee, internal verification tool. etc.
Some of those are very hard to do/prove in a ticket. ;(
"Users whose account is protected with 2FA are always considered verified."
I'm not sure what this means? If you have a otp enrolled and can login with it, thats sufficient proof of identity?
I don't think we should write the verification method in the policy. Can't this just be some Releng SOP that gets adjusted over time? Should we just say "The details of the procedure are determined by Releng" ?
The returning part is still a bit fuzzy I think... "Providing that they can prove their identity through other methods than the email used in their account, they can open a ticket in the link:https://pagure.io/packager-sponsors/new_issue/[packager-sponsors] tracker and ask for their status to be restored." Who do they prove this to? In the ticket? Some of the things we allow for identiy when people can't login to their account anymore: can sign a email with gpg key associated with the account can use the ssh private key associated with the public key in their account (ie, can login to fedorapeople) if Red Hat employee, internal verification tool. etc. Some of those are very hard to do/prove in a ticket. ;(
My idea is that they can open a ticket to start the quick procedure for being re-admitted into packager group... I mean, to get in touch with someone. How that is achieved is still to be determined.
"Users whose account is protected with 2FA are always considered verified." I'm not sure what this means? If you have a otp enrolled and can login with it, that's sufficient proof of identity?
I'm not sure what this means? If you have a otp enrolled and can login with it, that's sufficient proof of identity?
This is something that needs to be tweaked. My original idea was: if a user asks to be re-admitted into the packager group and they have 2FA enabled in their account, we can consider them verified. At a later re-think, this is not true, because (currently) we don't when 2FA was enabled, so if the account was hijacked the 2FA could be have been enabled by someone else. I think we should change this to be something like "accounts with 2FA enabled are always considered safe and are not subject to periodic check". This should also encourage users with commit privileges to use a safer authentication method by enabling 2FA on their account. But, currently, the data fasclient provides doesn't show if a user has 2FA enabled, so my script cannot exclude those accounts from the check.
This was discussed during today's fesco meeting: * ACTION: mattia so submit another version taking into account comments in the ticket and on the pull request (zbyszek, 17:16:42)
I've updated the policy proposal as: - step one (identify inactive packagers and open tickets) one week before Beta freeze - step two (remove inactive packagers and orphan packages) one week after final release
I've also tweaked the 2FA point (grammar mistakes apart) by just saying that "in future users with 2FA enabled may be exempted from the check" so encouraging them to enable 2FA.
This ticket will be discussed during today's FESCo meeting (2022-07-12 17:00 UTC in #fedora-meeting).
Due to the circumstances we did not, in fact, discuss this ticket during today's meeting. FESCo members, please review the latest update to the proposed policy, and vote in ticket.
Consider votes that were cast in previous comments invalidated by the changes to the proposed policy that were submitted in the meantime.
Metadata Update from @decathorpe: - Issue untagged with: meeting
Pull request for FESCo-docs: https://pagure.io/fesco/fesco-docs/pull-request/61 please review the latest update to the proposed policy, and vote in ticket.
please review the latest update to the proposed policy, and vote in ticket.
+1 here also.
I offered some last-minute nit-picks that affect style rather than substance.
With or without those suggestions:
This seems APPROVED (+4,0,-0)
Metadata Update from @churchyard: - Issue tagged with: document it, pending announcement
Announced:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/6VAM5N54Z4AJQ7XVYFSZZFC6QTG26P7H/
Metadata Update from @decathorpe: - Issue close_status updated to: Accepted - Issue status updated to: Closed (was: Open)
Metadata Update from @decathorpe: - Issue untagged with: pending announcement
Login to comment on this ticket.