Learn more about these different git repos.
Other Git URLs
When creating a repo through pagure (webui or api call) that should be created in repospanner region, but that repospanner itself is down/unreachable, pagure still creates the repo in DB and so visible in WebUI. That means that there is no way to even force to create it correctly when repospanner is down, and the task to create the repo in pagure worker isn't kept. That means that there is currently no way to heal that state ?
Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: Traceback (most recent call last): Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 238, in trace_task Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: R = retval = fun(*args, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 416, in __protected_call__ Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: return self.run(*args, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/pagure/lib/tasks_utils.py", line 36, in decorated_function Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: return function(self, session, *args, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/pagure/lib/tasks.py", line 267, in create_project Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: ignore_existing_repo, Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/pagure/lib/git.py", line 2612, in create_project_repos Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: project, region, templ, ignore_existing, repotype Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/pagure/lib/git.py", line 2544, in _create_project_repo Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: regioninfo["admin_cert"]["key"], Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/requests/api.py", line 108, in post Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: return request('post', url, data=data, json=json, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: response = session.request(method=method, url=url, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: resp = self.send(prep, **send_kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: r = adapter.send(request, **kwargs) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 415, in send Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: raise ConnectionError(err, request=request) Mar 27 12:27:14 git.dev.lon1.centos.org celery[382]: ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))
Impossible then to create the repo through api, as it exists (from Pagure side), so goes 500 : https://git.dev.centos.org/sig-core/repospanner_down. And if repospanner is available again, of course repospanner itself answers 404 as nothing was ever created :
[Wed Mar 27 12:35:20.250088 2019] [:error] [pid 25397] Traceback (most recent call last): [Wed Mar 27 12:35:20.250115 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app [Wed Mar 27 12:35:20.250146 2019] [:error] [pid 25397] response = self.full_dispatch_request() [Wed Mar 27 12:35:20.250154 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request [Wed Mar 27 12:35:20.250161 2019] [:error] [pid 25397] rv = self.handle_user_exception(e) [Wed Mar 27 12:35:20.250167 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception [Wed Mar 27 12:35:20.250173 2019] [:error] [pid 25397] reraise(exc_type, exc_value, tb) [Wed Mar 27 12:35:20.250179 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1473, in full_dispatch_request [Wed Mar 27 12:35:20.250186 2019] [:error] [pid 25397] rv = self.preprocess_request() [Wed Mar 27 12:35:20.250192 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1666, in preprocess_request [Wed Mar 27 12:35:20.250198 2019] [:error] [pid 25397] rv = func() [Wed Mar 27 12:35:20.250204 2019] [:error] [pid 25397] File "/usr/lib/python2.7/site-packages/pagure/flask_app.py", line 354, in set_request [Wed Mar 27 12:35:20.250210 2019] [:error] [pid 25397] flask.g.branches = sorted(flask.g.repo_obj.listall_branches()) [Wed Mar 27 12:35:20.250216 2019] [:error] [pid 25397] GitError: Error received from repoSpanner: 404
So what's the way to ensure that pagure_worker fixes those request and doesn't create in pagure the repo if it wasn't able to really init it in repospanner region ? Next question is : how to solve that (I'll open a new RFE to allow deleting a project through api, as I have hundreds of repos in that state for now)
Metadata Update from @pingou: - Issue tagged with: bug
I confirm that following patch (https://pagure.io/pagure/c/a3b1fe3ec6dcec2b46b60f96b1c84f30468cebda.patch) solves this. I disabled repospanner, tried to create project and go directly "connection refused" and so no project/repo was created in pagure (Expected). Thank you ! I guess it will be merge soon and so available in next release. Feel free to change status on that issue when it will be merged (or ask me to even close it once it landed in the tree)
Metadata Update from @pingou: - Issue assigned to pingou - Issue set to the milestone: 5.5
Commit 15a6649 fixes this issue
Log in to comment on this ticket.