#683 mbs-build local fails with 'ERROR - Could not process message handler. See the traceback.'
Closed 6 years ago Opened 6 years ago by karsten.

module-build-service-1.3.27-2.fc26 can't do local builds anymore:

PoolThread-twisted.internet.reactor-0 - module_build_service - ERROR - Could not process message handler. See the traceback.
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/consumer.py", line 218, in process_message
      further_work = handler(conf, session, msg) or []
    File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/modules.py", line 206, in wait
      dependencies, tag = _get_deps_and_tag()
    File "/usr/lib/python2.7/site-packages/module_build_service/utils.py", line 67, in inner
      return function(*args, **kwargs)
    File "/usr/lib/python2.7/site-packages/module_build_service/scheduler/handlers/modules.py", line 181, in _get_deps_and_tag
      pdc_session, build.mmd(), strict=True)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 423, in get_module_build_dependencies
      session, modified_dep, strict=strict)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 319, in _get_recursively_required_modules
      session, modified_dep, modules, strict)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 319, in _get_recursively_required_modules
      session, modified_dep, modules, strict)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 319, in _get_recursively_required_modules
      session, modified_dep, modules, strict)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 282, in _get_recursively_required_modules
      module_info = get_module(session, variant_dict, strict)
    File "/usr/lib/python2.7/site-packages/module_build_service/pdc.py", line 192, in get_module
      retval = session['unreleasedvariants/'](page_size=1, **query)
    File "<string>", line 1, in <lambda>
    File "/usr/lib/python2.7/site-packages/beanbag/namespace.py", line 136, in fn
    *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 102, in call
      return self.make_request(path, verb, kwargs, body)
    File "/usr/lib/python2.7/site-packages/beanbag/url_v1.py", line 151, in make_request
      r = self.session.request(verb, path, params=params, data=ebody)
    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
      resp = self.send(prep, **send_kwargs)
    File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
      r = adapter.send(request, **kwargs)
    File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 473, in send
      raise ConnectionError(err, request=request)
  ConnectionError: ('Connection aborted.', BadStatusLine("''",))
  2017-09-14 14:23:26,580 - PoolThread-twisted.internet.reactor-0 - module_build_service - DEBUG - Done with wait: MBSModule, 2
  2017-09-14 14:23:26,584 - PoolThread-twisted.internet.reactor-0 - module_build_service.scheduler - DEBUG - stop_condition checking <ModuleBuild X11-base, id=1, stream=master, version=20170905093318, state 'failed', batch 0, state_reaso
  2017-09-14 14:23:26,584 - PoolThread-twisted.internet.reactor-0 - module_build_service - INFO - Scheduling shutdown.

module-build-service-1.3.26-3.fc26 works with the same module


That's more likely some temporary networking issue when talking to PDC.

I don't think that's the problem here. I've switched between the 2 mbs versions several times, 1.3.26 works every time, 1.3.27 never works

Ah - I see now the ConnectionError: ('Connection aborted.', BadStatusLine("''",)) error.

I think @jkaluza is right, and that the underlying problem is an ugly bug in the requests/urllib3 stack. See this reproducer.

If you try again, does the error go away?

One way for us to fix this in MBS is to turn off session keepalives in MBS's use of the pdc client lib.

Metadata Update from @ralph:
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata