#11957 Logging code errors when doing pushes on src.fp.o
Closed: Fixed with Explanation a year ago by kevin. Opened a year ago by ngompa.

I'm seeing errors like this when pushing changes up to src.fedoraproject.org:

ngompa@fedora ~/f/libliftoff (rawhide)> fedpkg push
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 16 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 632 bytes | 632.00 KiB/s, done.
Total 5 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks
remote:     authbackend = get_git_auth_helper()
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper
remote:     _log.info("Looking for backend: %s", backend)
remote: Message: 'Looking for backend: %s'
remote: Arguments: ('distgit',)
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 197, in check_acl
remote:     self.debug("Protected namespaces: %s" % self.protected_namespaces)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "Protected namespaces: ['rpms', 'modules', 'container']"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 198, in check_acl
remote:     self.debug("Blocking unspecified refs: %s" % self.block_unspecified)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Blocking unspecified refs: False'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 199, in check_acl
remote:     self.debug("Blacklists: %s" % self.blacklists)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "Blacklists: [re.compile('refs/heads/c[0-9]+.*'), re.compile('refs/heads/master')]"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 200, in check_acl
remote:     self.debug("User: %s" % user)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'User: User: 1089 - name ngompa'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 201, in check_acl
remote:     self.debug("User groups: %s" % usergroups)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "User groups: {'packager', 'xr-sig', 'lxqt-sig', 'kde-sig', 'multimedia-sig', 'asahi-sig', 'python-sig', 'go-sig', 'cloud-sig', 'epel-packagers-sig', 'provenpackager', 'pantheon-sig', 'rust-sig', 'python-packagers-sig'}"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 202, in check_acl
remote:     self.debug("Committer: %s" % may_commit)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Committer: True'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 203, in check_acl
remote:     self.debug("SIG memberships: %s" % user_sigs)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'SIG memberships: set()'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 204, in check_acl
remote:     self.debug("RCM: %s" % is_rcm)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'RCM: False'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 205, in check_acl
remote:     self.debug("By-pass PR-only: %s" % bool(bypass_pr_only))
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'By-pass PR-only: True'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 253, in check_acl
remote:     self.debug("Branch %s is supported" % refname)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Branch refs/heads/rawhide is supported'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 274, in check_acl
remote:     self.debug("Unspecified branch push")
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Unspecified branch push'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/pre-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 291, in check_acl
remote:     self.debug("Committer push")
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Committer push'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks
remote:     authbackend = get_git_auth_helper()
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper
remote:     _log.info("Looking for backend: %s", backend)
remote: Message: 'Looking for backend: %s'
remote: Arguments: ('distgit',)
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 197, in check_acl
remote:     self.debug("Protected namespaces: %s" % self.protected_namespaces)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "Protected namespaces: ['rpms', 'modules', 'container']"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 198, in check_acl
remote:     self.debug("Blocking unspecified refs: %s" % self.block_unspecified)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Blocking unspecified refs: False'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 199, in check_acl
remote:     self.debug("Blacklists: %s" % self.blacklists)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "Blacklists: [re.compile('refs/heads/c[0-9]+.*'), re.compile('refs/heads/master')]"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 200, in check_acl
remote:     self.debug("User: %s" % user)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'User: User: 1089 - name ngompa'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 201, in check_acl
remote:     self.debug("User groups: %s" % usergroups)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: "User groups: {'go-sig', 'python-packagers-sig', 'rust-sig', 'packager', 'multimedia-sig', 'kde-sig', 'xr-sig', 'asahi-sig', 'epel-packagers-sig', 'python-sig', 'pantheon-sig', 'lxqt-sig', 'cloud-sig', 'provenpackager'}"
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 202, in check_acl
remote:     self.debug("Committer: %s" % may_commit)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Committer: True'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 203, in check_acl
remote:     self.debug("SIG memberships: %s" % user_sigs)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'SIG memberships: set()'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 204, in check_acl
remote:     self.debug("RCM: %s" % is_rcm)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'RCM: False'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 205, in check_acl
remote:     self.debug("By-pass PR-only: %s" % bool(bypass_pr_only))
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'By-pass PR-only: True'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 253, in check_acl
remote:     self.debug("Branch %s is supported" % refname)
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Branch refs/heads/rawhide is supported'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 274, in check_acl
remote:     self.debug("Unspecified branch push")
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Unspecified branch push'
remote: Arguments: ()
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "hooks/update", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks
remote:     repodir=repodir,
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 291, in check_acl
remote:     self.debug("Committer push")
remote:   File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug
remote:     _log.debug(msg)
remote: Message: 'Committer push'
remote: Arguments: ()
remote: Emitting a message to the fedora-messaging message bus.
remote: * Publishing information for 1 commits
remote: Sending to redis to log activity and send commit notification emails
remote: * Publishing information for 1 commits
remote:   - to fedora-message
remote: 2024-05-31 11:38:17,832 [WARNING] pagure.lib.notify: pagure is about to send a message that has no schemas: pagure.git.receive
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks
remote:     authbackend = get_git_auth_helper()
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper
remote:     _log.info("Looking for backend: %s", backend)
remote: Message: 'Looking for backend: %s'
remote: Arguments: ('distgit',)
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 412, in run_project_hooks
remote:     pull_request=pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 124, in runhook
remote:     pull_request=pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 449, in post_receive
remote:     pull_request.id if pull_request else None,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 220, in send_notifications
remote:     pagure.lib.notify.blinker_publish(topic, msg)
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/notify.py", line 153, in blinker_publish
remote:     _log.info("Sending blinker signal to: pagure - topic: %s", topic)
remote: Message: 'Sending blinker signal to: pagure - topic: %s'
remote: Arguments: ('git.receive',)
remote: --- Logging error ---
remote: Traceback (most recent call last):
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit
remote:     self.doRollover()
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover
remote:     self.rotate(self.baseFilename, dfn)
remote:   File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate
remote:     os.rename(source, dest)
remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
remote: Call stack:
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module>
remote:     run_hook_file(hooktype)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file
remote:     pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 412, in run_project_hooks
remote:     pull_request=pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 124, in runhook
remote:     pull_request=pull_request,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 449, in post_receive
remote:     pull_request.id if pull_request else None,
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 223, in send_notifications
remote:     send_fedmsg_notifications(project, topic, msg)
remote:   File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 69, in send_fedmsg_notifications
remote:     pagure.lib.notify.fedora_messaging_publish(topic, msg)
remote:   File "/usr/lib/python3.6/site-packages/pagure/lib/notify.py", line 93, in fedora_messaging_publish
remote:     topic,
remote: Message: 'pagure is about to send a message that has no schemas: pagure.%s'
remote: Arguments: ('git.receive',)
To ssh://pkgs.fedoraproject.org/rpms/libliftoff
   d39b917..840ecab  rawhide -> rawhide

Metadata Update from @zlopez:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: Needs investigation, high-gain

a year ago

Boils down to PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' for every step that's logged. I wonder if permissions in the spec file are wrong and reverted some local changes?

I looked into that and the logrotate is running as part of pagure logging. All the files in pagure_auth has correct SELinux context (I can't find anything related to pagure_auth in /var/log/audit/audit.log as well) and the user owning them is the same as is running the pagure_worker.service.

So I'm not sure what could be the issue here.

Found it! The error is from pagure_mirror.service and this is running under UID 1001 (paguremirroring user) instead of UID 1000 (git user).

It seems that the paguremirroring user was either in git group or the folder had some additional ACLs to perform that. Let me do a quick fix by adding the paguremirroring user to git group.

So the paguremirroring user is already in git group, but the files have only 640 permissions.

I did chmod 660 for now and we need to set this to be true for new files somewhere. The error should be now gone. I will check the playbook to see what I can do to make it permanent.

Metadata Update from @zlopez:
- Issue assigned to zlopez

a year ago

Metadata Update from @zlopez:
- Issue untagged with: Needs investigation
- Issue tagged with: medium-trouble

a year ago

I found out that I only fixed the issue for https://pagure.io and totally forgot about htpps://src.fedoraproject.org. I'm sorry about that and a big thank you to @kevin for fixing that.

The issue was similar, only the group was packager in case of https://src.fedoraproject.org instead of git.

That pr has been pushed. Thanks!

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

a year ago

Just randomly found tab with #11763. I hope we have final solution now 😉

There were few remaining issues but they should be solved now and it should be permanent solution.

Log in to comment on this ticket.

Metadata