#12850 db01.rdu3.fedoraproject.org timeout (postgresql?) when running git push to my rpms/haruna fork
Closed: Duplicate a month ago by kevin. Opened a month ago by rathann.

Describe what you would like us to do:


Fix the following issue:

$ git push -u rathann ffmpeg8 --force
The pre-push script can't check private branches. Push operation continues.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 384 bytes | 384.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
remote:     return fn()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
remote:     return _ConnectionFairy._checkout(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
remote:     fairy = _ConnectionRecord.checkout(pool)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
remote:     rec = pool._do_get()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 138, in _do_get
remote:     self._dec_overflow()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
remote:     compat.reraise(exc_type, exc_value, exc_tb)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 129, in reraise
remote:     raise value
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 135, in _do_get
remote:     return self._create_connection()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
remote:     return _ConnectionRecord(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in __init__
remote:     self.__connect(first_connect_check=True)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
remote:     connection = pool._invoke_creator(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
remote:     return dialect.connect(*cargs, **cparams)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
remote:     return self.dbapi.connect(*cargs, **cparams)
remote:   File "/usr/lib64/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
remote:     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
remote: psycopg2.OperationalError: could not connect to server: Connection timed out
remote:     Is the server running on host "db01.rdu3.fedoraproject.org" (10.16.163.41) and accepting
remote:     TCP/IP connections on port 5432?
remote: 
remote: 
remote: The above exception was the direct cause of the following exception:
remote: 
remote: Traceback (most recent call last):
remote:   File "hooks/post-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 551, in run_hook_file
remote:     session, repo, user=username, namespace=namespace
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/query.py", line 2874, in _get_project
remote:     return query.one()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3275, in one
remote:     ret = self.one_or_none()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3244, in one_or_none
remote:     ret = list(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3317, in __iter__
remote:     return self._execute_and_instances(context)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3339, in _execute_and_instances
remote:     querycontext, self._connection_from_session, close_with_result=True
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3354, in _get_bind_args
remote:     mapper=self._bind_mapper(), clause=querycontext.statement, **kw
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3332, in _connection_from_session
remote:     conn = self.session.connection(**kw)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1123, in connection
remote:     execution_options=execution_options,
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1129, in _connection_for_bind
remote:     engine, execution_options
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 430, in _connection_for_bind
remote:     conn = bind._contextual_connect()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
remote:     self._wrap_pool_connect(self.pool.connect, None),
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
remote:     e, dialect, self
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
remote:     util.raise_from_cause(sqlalchemy_exception, exc_info)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
remote:     reraise(type(exception), exception, tb=exc_tb, cause=cause)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
remote:     raise value.with_traceback(tb)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
remote:     return fn()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
remote:     return _ConnectionFairy._checkout(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
remote:     fairy = _ConnectionRecord.checkout(pool)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
remote:     rec = pool._do_get()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 138, in _do_get
remote:     self._dec_overflow()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
remote:     compat.reraise(exc_type, exc_value, exc_tb)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 129, in reraise
remote:     raise value
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 135, in _do_get
remote:     return self._create_connection()
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
remote:     return _ConnectionRecord(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in __init__
remote:     self.__connect(first_connect_check=True)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
remote:     connection = pool._invoke_creator(self)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
remote:     return dialect.connect(*cargs, **cparams)
remote:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
remote:     return self.dbapi.connect(*cargs, **cparams)
remote:   File "/usr/lib64/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
remote:     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
remote: sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection timed out
remote:     Is the server running on host "db01.rdu3.fedoraproject.org" (10.16.163.41) and accepting
remote:     TCP/IP connections on port 5432?
remote: 
remote: (Background on this error at: http://sqlalche.me/e/e3q8)
To ssh://pkgs.fedoraproject.org/forks/rathann/rpms/haruna.git
 + 6b50d9c...b49619b ffmpeg8 -> ffmpeg8 (forced update)
branch 'ffmpeg8' set up to track 'rathann/ffmpeg8'.
$

When do you need this to be done by? (YYYY/MM/DD)


As soon as possible.


Metadata Update from @zlopez:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: Needs investigation, high-gain, ops

a month ago

I tried to reach the machine from src.fedoraproject.org machine and it worked fine

$ nc -v db01.rdu3.fedoraproject.org 5432
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Connected to 10.16.163.41:5432.

I'm not sure what happened, but I can't reproduce the issue.

Could have been a temporary high load issue. Maybe related: https://pagure.io/fedora-infrastructure/issue/12814 .

Already reported at https://pagure.io/fedora-infrastructure/issue/12829

And yes, this is likely the same timeout issue. ;(

Metadata Update from @kevin:
- Issue close_status updated to: Duplicate
- Issue status updated to: Closed (was: Open)

a month ago

Log in to comment on this ticket.

Metadata