NOTE
If your issue is for security or deals with sensitive info please mark it as private using the checkbox below.
Investigate why https://src.fedoraproject.org/fork/music/rpms/python-setuptools-gettext/diff/rawhide..pyproject (or any similar “create a PR” URL on that project) results in an error 500.
Note that this does not seem to have anything to do with authentication, cannot be resolved by clearing cookies, and is reproducible with curl https://src.fedoraproject.org/fork/music/rpms/python-setuptools-gettext/diff/rawhide..pyproject.
curl https://src.fedoraproject.org/fork/music/rpms/python-setuptools-gettext/diff/rawhide..pyproject
N/A; this prevents creating PR’s on that project, but it can be (inconveniently) worked around by offering patches or mentioning forks in Bugzilla.
I do not know if this is a general issue or not; I have only seen it on that project.
See also https://pagure.io/fedora-infrastructure/issue/12291#comment-944557.
This seems to be the problem:
[Sat Nov 16 20:46:13.388338 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] [2024-11-16 20:46:13,387] ERROR in app: Exception on /fork/music/rpms/python-setuptools-gettext/diff/rawhide..pyproject [GET] [Sat Nov 16 20:46:13.388391 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] Traceback (most recent call last): [Sat Nov 16 20:46:13.388399 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app [Sat Nov 16 20:46:13.388406 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] response = self.full_dispatch_request() [Sat Nov 16 20:46:13.388412 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request [Sat Nov 16 20:46:13.388419 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] rv = self.handle_user_exception(e) [Sat Nov 16 20:46:13.388426 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception [Sat Nov 16 20:46:13.388457 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] reraise(exc_type, exc_value, tb) [Sat Nov 16 20:46:13.388482 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise [Sat Nov 16 20:46:13.388506 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] raise value [Sat Nov 16 20:46:13.388521 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request [Sat Nov 16 20:46:13.388527 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] rv = self.dispatch_request() [Sat Nov 16 20:46:13.388542 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request [Sat Nov 16 20:46:13.388559 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] return self.view_functions[rule.endpoint](**req.view_args) [Sat Nov 16 20:46:13.388566 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/pagure/ui/fork.py", line 1797, in new_request_pull [Sat Nov 16 20:46:13.388585 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] requestrepo = pygit2.Repository(requestrepopath) [Sat Nov 16 20:46:13.388592 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib64/python3.6/site-packages/pygit2/repository.py", line 1210, in __init__ [Sat Nov 16 20:46:13.388607 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] path_backend = init_file_backend(path) [Sat Nov 16 20:46:13.388618 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] _pygit2.GitError: Repository not found at /srv/git/repositories/requests/rpms/python-setuptools-gettext.git [Sat Nov 16 20:46:13.388665 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] [Sat Nov 16 20:46:13.388990 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] 2024-11-16 20:46:13,388 [ERROR] pagure.flask_app: Error while processing request [Sat Nov 16 20:46:13.389014 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] Traceback (most recent call last): [Sat Nov 16 20:46:13.389021 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app [Sat Nov 16 20:46:13.389028 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] response = self.full_dispatch_request() [Sat Nov 16 20:46:13.389035 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request [Sat Nov 16 20:46:13.389065 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] rv = self.handle_user_exception(e) [Sat Nov 16 20:46:13.389072 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception [Sat Nov 16 20:46:13.389080 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] reraise(exc_type, exc_value, tb) [Sat Nov 16 20:46:13.389086 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise [Sat Nov 16 20:46:13.389093 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] raise value [Sat Nov 16 20:46:13.389109 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request [Sat Nov 16 20:46:13.389121 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] rv = self.dispatch_request() [Sat Nov 16 20:46:13.389141 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request [Sat Nov 16 20:46:13.389161 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] return self.view_functions[rule.endpoint](**req.view_args) [Sat Nov 16 20:46:13.389170 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib/python3.6/site-packages/pagure/ui/fork.py", line 1797, in new_request_pull [Sat Nov 16 20:46:13.389189 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] requestrepo = pygit2.Repository(requestrepopath) [Sat Nov 16 20:46:13.389202 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] File "/usr/lib64/python3.6/site-packages/pygit2/repository.py", line 1210, in __init__ [Sat Nov 16 20:46:13.389215 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] path_backend = init_file_backend(path) [Sat Nov 16 20:46:13.389232 2024] [wsgi:error] [pid 1394715:tid 139985291089664] [remote 10.3.163.74:49708] _pygit2.GitError: Repository not found at /srv/git/repositories/requests/rpms/python-setuptools-gettext.git
Poking around on the server end a bit, /srv/git/repositories/requests/rpms/python-setuptools-gettext.git is there but does not seem to be a valid git repo. The equivalent directories for other packages are valid git repos. Not sure what's happened, if it's got corrupted or what.
/srv/git/repositories/requests/rpms/python-setuptools-gettext.git
The dir seems to be missing objects and refs subdirectories, compared to others.
objects
refs
Metadata Update from @zlopez: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: medium-gain, medium-trouble, ops, src.fp.o
@music Could you try to delete the fork and create it again?
It's not issue with the fork, the same is happening in the https://src.fedoraproject.org/rpms/python-setuptools-gettext
I wonder what we can do about that?
I'd say we need to write a little script to check every dir and find ones that aren't valid git repos (running git status in the dir will tell you), and somehow restore the broken ones from...somewhere else? That's the tricky part. I don't know if we have copies of the repos anywhere else :/
git status
Hum. This works:
[adamw@xps13a build]$ fedpkg co python-setuptools-gettext Cloning into 'python-setuptools-gettext'... remote: Enumerating objects: 77, done. remote: Counting objects: 100% (77/77), done. remote: Compressing objects: 100% (70/70), done. remote: Total 77 (delta 42), reused 13 (delta 5), pack-reused 0 Receiving objects: 100% (77/77), 9.27 KiB | 4.63 MiB/s, done. Resolving deltas: 100% (42/42), done.
so there must be a working copy of the repo somewhere, I guess. That brings up various questions about what this broken repo on the server end is, obviously...
oh, is this the git repo that backs the package's pull requests, rather than the repo for the package itself? I forget that everything's a git repo to Pagure.
OK, after a bit of fiddling about I got a working check script, here's the survey results:
[root@pkgs01 rpms][PROD-IAD2]# for i in `ls -d *.git`; do cd $i; git rev-parse --abbrev-ref HEAD > /dev/null || echo "$i broken"; cd ..; done fatal: not a git repository (or any of the parent directories): .git aubit4gl.git broken fatal: not a git repository (or any of the parent directories): .git kf6-bluez-qt.git broken fatal: not a git repository (or any of the parent directories): .git libsv.git broken fatal: not a git repository (or any of the parent directories): .git python3-pyserial.git broken fatal: not a git repository (or any of the parent directories): .git python-setuptools-gettext.git broken fatal: not a git repository (or any of the parent directories): .git qatzip.git broken fatal: not a git repository (or any of the parent directories): .git sherlock-project.git broken fatal: not a git repository (or any of the parent directories): .git unzboot.git broken
edit: in good news, running the same check script on the directory that contains the actual package repos, none of those are bad.
@kevin can we restore these from backups, maybe? not sure we'd have anywhere else to restore them from.
We have grokmirror backups. I can check those, but likely if they stopped being git repos it wouldn't back them up anymore (but might have the last one).
To clarify, these are all requests repos? If there's no pull requests (how could there be with broken requests repos), can we just create a new empty one there?
Yeah, the ones I found are request repos. The scenario I'm worried about is that they were valid repos for some time, and so maybe some pull requests do/did exist, but then they got broken somehow - so if we recreate them as empty repos we'll erase those PRs from history. Can we check the backups and see if it looked like they contained anything at the last time they were valid repos, if they ever were?
I suppose it might also be possible to check in pagure's db if there's any trace of existing prs for any of these packages...
It is possible. ;)
So, I looked at grokmirror backups. None of these projects have a requests repo backed up.
I looked in the db, and I could only find one PR against any of these packages:
id | uid | title | project_id | branc h | project_id_from | remote_git | branch_ from | commit_start | commit_stop | ini tial_comment | user_id | assignee_id | merge_status | status | closed_by_id | closed_at | date_created | updated_on | last_updated | private | allow_rebase ----+----------------------------------+------------------------------------+------------+------ ---+-----------------+-----------------------------------------------------------------+-------- -----+------------------------------------------+------------------------------------------+---- -------------+---------+-------------+--------------+--------+--------------+------------------- ---------+----------------------------+----------------------------+---------------------------- +---------+-------------- 1 | 5fd5d2e980cc4a8696c497f0659b9083 | add doc package with QCH API Docs | 66396 | rawhi de | | https://src.fedoraproject.org/forks/loise/rpms/kf6-bluez-qt.git | rawhide | a8d464ec284ca4119e1e5d47dea8ee000e33b7cb | a8d464ec284ca4119e1e5d47dea8ee000e33b7cb | | 4138 | | | Merged | 4138 | 2024-01-10 12:36:5 8.216813 | 2024-01-10 12:35:33.035806 | 2024-01-10 12:36:59.536595 | 2024-01-10 12:36:59.538805 | f | f
it was merged...and it shows up on the interface?
https://src.fedoraproject.org/rpms/kf6-bluez-qt/pull-requests?status=Merged
No 500 there? perhaps because it was merged?
Anyhow, I think it should be ok to just recreate these repos?
sure, sounds reasonable to me.
Metadata Update from @kevin: - Issue assigned to kevin
I'll try and do this in coming days.
Log in to comment on this ticket.