#7798 mdapi connections error
Opened 3 months ago by cverna. Modified 2 months ago

When testing the newly deployed mdapi on OpenShift (mdapi.fp.o) with fedora-packaging indexing I get a lot of connection errors.

On the client side (fedora-packages making requests to mdapi) :

Exception in thread Thread-26:                                                                                                                                                                                     
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner                                                                                                                                         
    self.run()                                                                                                                                                                                                     
  File "/usr/lib64/python2.7/threading.py", line 757, in run                                                                                                                                                       
    self.__target(*self.__args, **self.__kwargs)                                                                                                                                                                   
  File "/usr/lib/python2.7/site-packages/fedoracommunity/pool.py", line 33, in run                                                                                                                                 
    result = func(item)                                                                                                                                                                                            
  File "/usr/lib/python2.7/site-packages/fedoracommunity/search/index.py", line 365, in io_work                                                                                                                    
    package = self.construct_package_dictionary(package)                                                                                                                                                           
  File "/usr/lib/python2.7/site-packages/fedoracommunity/search/index.py", line 288, in construct_package_dictionary                                                                                               
    package['sub_pkgs'] = list(self.get_sub_packages(package))                                                                                                                                                     
  File "/usr/lib/python2.7/site-packages/fedoracommunity/search/index.py", line 306, in get_sub_packages                                                                                                           
    response = local.http.get(url)                                                                                                                                                                                 
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 537, in get                                                                                                                                   
    return self.request('GET', url, **kwargs)                                                                                                                                                                      
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 524, in request                                                                                                                               
    resp = self.send(prep, **send_kwargs)                                                                                                                                                                          
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 637, in send                                                                                                                                  
    r = adapter.send(request, **kwargs)                                                                                                                                                                            
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 498, in send                                                                                                                                  
    raise ConnectionError(err, request=request)                                                                                                                                                                    
ConnectionError: ('Connection aborted.', BadStatusLine("''",))  

On the server side (mdapi in OpenShift)

2019-05-15 11:31:22,850 [ERROR] aiohttp.server: Unhandled exception
Traceback (most recent call last):
  File "/usr/lib64/python3.7/site-packages/aiohttp/web_protocol.py", line 411, in start
    await resp.write_eof()
  File "/usr/lib64/python3.7/site-packages/aiohttp/web_response.py", line 596, in write_eof
    await super().write_eof(body)
  File "/usr/lib64/python3.7/site-packages/aiohttp/web_response.py", line 401, in write_eof
    await self._payload_writer.write_eof(data)
  File "/usr/lib64/python3.7/site-packages/aiohttp/http_writer.py", line 136, in write_eof
    self._write(chunk)
  File "/usr/lib64/python3.7/site-packages/aiohttp/http_writer.py", line 67, in _write
    raise ConnectionResetError('Cannot write to closing transport')
ConnectionResetError: Cannot write to closing transport

I am wondering if the connection are not getting killed by the proxies, but I don't know enough about how the proxies works to be sure.


What host(s) are the connections using, resolving to what?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

3 months ago

So the connections are against mdapi.fedoraproject.org and it resolves to either proxy110.phx2.fedoraproject.org (10.5.126.9) or proxy101.phx2.fedoraproject.org (10.5.126.8).

@cverna out of curiosity, is this still happening?

I have not tried again since last time. I wanted to try it locally to see if I could reproduce the error.

Login to comment on this ticket.

Metadata