#166 testdays instance is exploding
Closed: Fixed None Opened 8 years ago by tflink.

I was following up on a TODO from QECamp and started looking at the testdays instance at http://testdays.fedorainfracloud.org/

Now that htttpd is running, it is throwing 500s when I try to import a new testday or access the one that's in there already. The httpd logs show the following traceback when I try to import a new job:

[Thu Feb 25 18:01:49.167259 2016] [:error] [pid 23740] /usr/lib/python2.7/site-packages/fedora/client/wiki.py:51: UserWarning: The Wiki client is deprecated.  Please use resources recommended by upstream instead: https://www.mediawiki.org/wiki/API:Client_code#Python
[Thu Feb 25 18:01:49.167283 2016] [:error] [pid 23740]   "The Wiki client is deprecated.  Please use resources "
[Thu Feb 25 18:01:50.310674 2016] [:error] [pid 23740] [resultsdb_api.py:58] 2016-02-25 18:01:50 WARNING Received HTTP failure status code 404 for request: http://127.0.0.1/resultsdb_api/api/v1.0/jobs/1
[Thu Feb 25 18:01:50.310905 2016] [:error] [pid 23740] [remote 75.70.15.227:27801] mod_wsgi (pid=23740): Exception occurred processing WSGI script '/usr/share/testdays/testdays.wsgi'.
[Thu Feb 25 18:01:50.310938 2016] [:error] [pid 23740] [remote 75.70.15.227:27801] Traceback (most recent call last):
[Thu Feb 25 18:01:50.310959 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
[Thu Feb 25 18:01:50.310995 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     return self.wsgi_app(environ, start_response)
[Thu Feb 25 18:01:50.311007 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
[Thu Feb 25 18:01:50.311025 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     response = self.make_response(self.handle_exception(e))
[Thu Feb 25 18:01:50.311034 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
[Thu Feb 25 18:01:50.311049 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     reraise(exc_type, exc_value, tb)
[Thu Feb 25 18:01:50.311057 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
[Thu Feb 25 18:01:50.311071 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     response = self.full_dispatch_request()
[Thu Feb 25 18:01:50.311081 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
[Thu Feb 25 18:01:50.311094 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     rv = self.handle_user_exception(e)
[Thu Feb 25 18:01:50.311103 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
[Thu Feb 25 18:01:50.311117 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     reraise(exc_type, exc_value, tb)
[Thu Feb 25 18:01:50.311125 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
[Thu Feb 25 18:01:50.311153 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     rv = self.dispatch_request()
[Thu Feb 25 18:01:50.311163 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
[Thu Feb 25 18:01:50.311177 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     return self.view_functions[rule.endpoint](**req.view_args)
[Thu Feb 25 18:01:50.311186 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/testdays/controllers/main.py", line 244, in show_event
[Thu Feb 25 18:01:50.311203 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     results = preparse_results(event)
[Thu Feb 25 18:01:50.311211 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/testdays/controllers/main.py", line 64, in preparse_results
[Thu Feb 25 18:01:50.311225 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     results = RDB_API.get_job(event.resultsdb_job_id)['results']
[Thu Feb 25 18:01:50.311234 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/resultsdb_api.py", line 110, in get_job
[Thu Feb 25 18:01:50.311248 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     self.__raise_on_error(r)
[Thu Feb 25 18:01:50.311256 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]   File "/usr/lib/python2.7/site-packages/resultsdb_api.py", line 60, in __raise_on_error
[Thu Feb 25 18:01:50.311270 2016] [:error] [pid 23740] [remote 75.70.15.227:27801]     '%s (HTTP %s)' % (r.json()['message'], r.status_code), r)
[Thu Feb 25 18:01:50.311297 2016] [:error] [pid 23740] [remote 75.70.15.227:27801] ResultsDBapiException: u'Job not found (HTTP 404)'

I see a similar error when I try to access the one testday which is already in the app. Triage what's going on and propose a fix


We don't have any project defined in Phab for testdays, should we create one? Adding to #infrastructure for the moment.

For some reason, the resultsdb's DB was absolutely empty. I have manually created a resultsdb job for the existing testday, and tried to create a new testday via the web interface - everything works fine.

Leaving opened, as @tflink and I are trying to investigate a little bit.

Metadata Update from @tflink:
- Issue tagged with: infrastructure

6 years ago

Login to comment on this ticket.

Metadata