#1196 uploading 1.9GB SRPM failed
Closed: Fixed 3 months ago by schlupov. Opened 4 months ago by praiskup.

2020-01-14 04:26:17,867 [ERROR][/usr/lib/python3.7/site-packages/flask/app.py:1761|app:log_exception] Exception on /api_3/build/create/upload [POST]
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1303, in read
    read = self._read(to_read)
OSError: Apache/mod_wsgi request data read error: Partial results are valid but processing is incomplete.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 110, in wrapper
    return f(self, stream, *args, **kwargs)
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 222, in _parse_multipart
    form, files = parser.parse(stream, boundary, content_length)
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 531, in parse
    return self.cls(form), self.cls(files)
  File "/usr/lib/python3.7/site-packages/werkzeug/datastructures.py", line 409, in __init__
    for key, value in mapping or ():
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 529, in <genexpr>
    form = (p[1] for p in formstream if p[0] == 'form')
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 488, in parse_parts
    for ellt, ell in self.parse_lines(file, boundary, content_length):
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 431, in parse_lines
    for line in iterator:
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1126, in make_line_iter
    for item in _iter_basic_lines():
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1102, in _iter_basic_lines
    new_data = next(_iter, '')
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1045, in _make_chunk_iter
    item = _read(buffer_size)
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1305, in read
    return self.on_disconnect()
  File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 1275, in on_disconnect
    raise ClientDisconnected()
werkzeug.exceptions.ClientDisconnected: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/share/copr/coprs_frontend/coprs/rest_api/__init__.py", line 44, in error_router
    return original_handler(e)
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/share/copr/coprs_frontend/coprs/views/misc.py", line 276, in decorated_function
    return f(*args, **kwargs)
  File "/usr/share/copr/coprs_frontend/coprs/views/apiv3_ns/__init__.py", line 94, in file_upload_wrapper
    if "json" in flask.request.files:
  File "/usr/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/usr/lib/python3.7/site-packages/werkzeug/utils.py", line 73, in __get__
    value = self.func(obj)
  File "/usr/lib/python3.7/site-packages/werkzeug/wrappers.py", line 571, in files
    self._load_form_data()
  File "/usr/lib/python3.7/site-packages/flask/wrappers.py", line 168, in _load_form_data
    RequestBase._load_form_data(self)
  File "/usr/lib/python3.7/site-packages/werkzeug/wrappers.py", line 385, in _load_form_data
    mimetype, content_length, options)
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 205, in parse
    content_length, options)
  File "/usr/lib/python3.7/site-packages/werkzeug/formparser.py", line 117, in wrapper
    chunk = stream.read(1024 * 64)
OSError: Apache/mod_wsgi request data read error: Input is already in error state.

$ copr build ping test.src.rpm
Uploading package test.src.rpm
|################################| 3.2 GB 9.1 MB/s eta 0:00:00Request is not in JSON format, there is probably a bug in the API code.

Metadata Update from @praiskup:
- Issue assigned to schlupov

4 months ago

This seems to be a regression. It is affecting projects that I was previously building in copr too:

copr-cli build @dotnet-sig/dotnet /home/omajid/fedora-git/dotnet-2-1/dotnet2.1-2.1.511-1.fc29.src.rpm
Uploading package /home/omajid/fedora-git/dotnet-2-1/dotnet2.1-2.1.511-1.fc29.src.rpm
 |################################| 1.3 GB 875.8 kB/s eta 0:00:00Request is not in JSON format, there is probably a bug in the API code.

This is preventing me from pushing updates to the existing package https://copr.fedorainfracloud.org/coprs/g/dotnet-sig/dotnet/package/dotnet2.1/

Does anyone have any idea about how long this might take to fix? Any ETA when a fix might be live? I can't upload src.rpms with sizes under 500MB either.

I'm working on it, unfortunately still not sure where might be the problem.

Today we reproduced the problem, this is in logs:

[Wed Feb 05 16:46:46.289018 2020] [wsgi:error] [pid 8959:tid 140578759104256] [client 213.175.37.10:8784] mod_wsgi (pid=8959): Request data write error when proxying data to daemon process: Broken pipe.
[Wed Feb 05 16:47:12.652863 2020] [authz_core:error] [pid 3547:tid 140578868143872] [client 192.47.255.254:10400] AH01630: client denied by server configuration: /usr/share/copr/coprs_frontend/application
[Wed Feb 05 16:48:12.638551 2020] [authz_core:error] [pid 3755:tid 140579187844864] [client 192.47.255.254:10602] AH01630: client denied by server configuration: /usr/share/copr/coprs_frontend/application
[Wed Feb 05 16:48:42.776158 2020] [wsgi:error] [pid 8959:tid 140578759104256] [client 213.175.37.10:8784] mod_wsgi (pid=8959): Request data write error when proxying data to daemon process: Broken pipe.
[Wed Feb 05 16:48:42.776262 2020] [wsgi:error] [pid 8959:tid 140578759104256] [client 213.175.37.10:8784] Truncated or oversized response headers received from daemon process '127.0.0.1': /usr/share/copr/coprs_frontend/application

TimeOut for the post request has been raised. Same problem as in issue #1228.

Metadata Update from @schlupov:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Metadata
Related Pull Requests
  • #1252 Last updated 2 months ago