I'm seeing errors like this when pushing changes up to src.fedoraproject.org:
ngompa@fedora ~/f/libliftoff (rawhide)> fedpkg push Enumerating objects: 9, done. Counting objects: 100% (9/9), done. Delta compression using up to 16 threads Compressing objects: 100% (5/5), done. Writing objects: 100% (5/5), 632 bytes | 632.00 KiB/s, done. Total 5 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0) remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks remote: authbackend = get_git_auth_helper() remote: File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper remote: _log.info("Looking for backend: %s", backend) remote: Message: 'Looking for backend: %s' remote: Arguments: ('distgit',) remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 197, in check_acl remote: self.debug("Protected namespaces: %s" % self.protected_namespaces) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "Protected namespaces: ['rpms', 'modules', 'container']" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 198, in check_acl remote: self.debug("Blocking unspecified refs: %s" % self.block_unspecified) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Blocking unspecified refs: False' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 199, in check_acl remote: self.debug("Blacklists: %s" % self.blacklists) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "Blacklists: [re.compile('refs/heads/c[0-9]+.*'), re.compile('refs/heads/master')]" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 200, in check_acl remote: self.debug("User: %s" % user) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'User: User: 1089 - name ngompa' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 201, in check_acl remote: self.debug("User groups: %s" % usergroups) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "User groups: {'packager', 'xr-sig', 'lxqt-sig', 'kde-sig', 'multimedia-sig', 'asahi-sig', 'python-sig', 'go-sig', 'cloud-sig', 'epel-packagers-sig', 'provenpackager', 'pantheon-sig', 'rust-sig', 'python-packagers-sig'}" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 202, in check_acl remote: self.debug("Committer: %s" % may_commit) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Committer: True' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 203, in check_acl remote: self.debug("SIG memberships: %s" % user_sigs) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'SIG memberships: set()' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 204, in check_acl remote: self.debug("RCM: %s" % is_rcm) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'RCM: False' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 205, in check_acl remote: self.debug("By-pass PR-only: %s" % bool(bypass_pr_only)) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'By-pass PR-only: True' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 253, in check_acl remote: self.debug("Branch %s is supported" % refname) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Branch refs/heads/rawhide is supported' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 274, in check_acl remote: self.debug("Unspecified branch push") remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Unspecified branch push' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/pre-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 291, in check_acl remote: self.debug("Committer push") remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Committer push' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks remote: authbackend = get_git_auth_helper() remote: File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper remote: _log.info("Looking for backend: %s", backend) remote: Message: 'Looking for backend: %s' remote: Arguments: ('distgit',) remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 197, in check_acl remote: self.debug("Protected namespaces: %s" % self.protected_namespaces) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "Protected namespaces: ['rpms', 'modules', 'container']" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 198, in check_acl remote: self.debug("Blocking unspecified refs: %s" % self.block_unspecified) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Blocking unspecified refs: False' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 199, in check_acl remote: self.debug("Blacklists: %s" % self.blacklists) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "Blacklists: [re.compile('refs/heads/c[0-9]+.*'), re.compile('refs/heads/master')]" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 200, in check_acl remote: self.debug("User: %s" % user) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'User: User: 1089 - name ngompa' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 201, in check_acl remote: self.debug("User groups: %s" % usergroups) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: "User groups: {'go-sig', 'python-packagers-sig', 'rust-sig', 'packager', 'multimedia-sig', 'kde-sig', 'xr-sig', 'asahi-sig', 'epel-packagers-sig', 'python-sig', 'pantheon-sig', 'lxqt-sig', 'cloud-sig', 'provenpackager'}" remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 202, in check_acl remote: self.debug("Committer: %s" % may_commit) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Committer: True' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 203, in check_acl remote: self.debug("SIG memberships: %s" % user_sigs) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'SIG memberships: set()' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 204, in check_acl remote: self.debug("RCM: %s" % is_rcm) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'RCM: False' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 205, in check_acl remote: self.debug("By-pass PR-only: %s" % bool(bypass_pr_only)) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'By-pass PR-only: True' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 253, in check_acl remote: self.debug("Branch %s is supported" % refname) remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Branch refs/heads/rawhide is supported' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 274, in check_acl remote: self.debug("Unspecified branch push") remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Unspecified branch push' remote: Arguments: () remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "hooks/update", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 372, in run_project_hooks remote: repodir=repodir, remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 291, in check_acl remote: self.debug("Committer push") remote: File "/usr/lib/python3.6/site-packages/dist_git_auth.py", line 128, in debug remote: _log.debug(msg) remote: Message: 'Committer push' remote: Arguments: () remote: Emitting a message to the fedora-messaging message bus. remote: * Publishing information for 1 commits remote: Sending to redis to log activity and send commit notification emails remote: * Publishing information for 1 commits remote: - to fedora-message remote: 2024-05-31 11:38:17,832 [WARNING] pagure.lib.notify: pagure is about to send a message that has no schemas: pagure.git.receive remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 342, in run_project_hooks remote: authbackend = get_git_auth_helper() remote: File "/usr/lib/python3.6/site-packages/pagure/lib/git_auth.py", line 66, in get_git_auth_helper remote: _log.info("Looking for backend: %s", backend) remote: Message: 'Looking for backend: %s' remote: Arguments: ('distgit',) remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 412, in run_project_hooks remote: pull_request=pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 124, in runhook remote: pull_request=pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 449, in post_receive remote: pull_request.id if pull_request else None, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 220, in send_notifications remote: pagure.lib.notify.blinker_publish(topic, msg) remote: File "/usr/lib/python3.6/site-packages/pagure/lib/notify.py", line 153, in blinker_publish remote: _log.info("Sending blinker signal to: pagure - topic: %s", topic) remote: Message: 'Sending blinker signal to: pagure - topic: %s' remote: Arguments: ('git.receive',) remote: --- Logging error --- remote: Traceback (most recent call last): remote: File "/usr/lib64/python3.6/logging/handlers.py", line 72, in emit remote: self.doRollover() remote: File "/usr/lib64/python3.6/logging/handlers.py", line 402, in doRollover remote: self.rotate(self.baseFilename, dfn) remote: File "/usr/lib64/python3.6/logging/handlers.py", line 113, in rotate remote: os.rename(source, dest) remote: PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' remote: Call stack: remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/files/post-receive", line 48, in <module> remote: run_hook_file(hooktype) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 570, in run_hook_file remote: pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 412, in run_project_hooks remote: pull_request=pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/__init__.py", line 124, in runhook remote: pull_request=pull_request, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 449, in post_receive remote: pull_request.id if pull_request else None, remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 223, in send_notifications remote: send_fedmsg_notifications(project, topic, msg) remote: File "/usr/lib/python3.6/site-packages/pagure/hooks/default.py", line 69, in send_fedmsg_notifications remote: pagure.lib.notify.fedora_messaging_publish(topic, msg) remote: File "/usr/lib/python3.6/site-packages/pagure/lib/notify.py", line 93, in fedora_messaging_publish remote: topic, remote: Message: 'pagure is about to send a message that has no schemas: pagure.%s' remote: Arguments: ('git.receive',) To ssh://pkgs.fedoraproject.org/rpms/libliftoff d39b917..840ecab rawhide -> rawhide
FYI @wombelix
Metadata Update from @zlopez: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: Needs investigation, high-gain
Boils down to PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30' for every step that's logged. I wonder if permissions in the spec file are wrong and reverted some local changes?
PermissionError: [Errno 13] Permission denied: '/var/log/pagure/pagure_auth.log' -> '/var/log/pagure/pagure_auth.log.2024-05-30'
I looked into that and the logrotate is running as part of pagure logging. All the files in pagure_auth has correct SELinux context (I can't find anything related to pagure_auth in /var/log/audit/audit.log as well) and the user owning them is the same as is running the pagure_worker.service.
pagure_auth
/var/log/audit/audit.log
pagure_worker.service
So I'm not sure what could be the issue here.
Found it! The error is from pagure_mirror.service and this is running under UID 1001 (paguremirroring user) instead of UID 1000 (git user).
pagure_mirror.service
UID 1001
paguremirroring
UID 1000
git
It seems that the paguremirroring user was either in git group or the folder had some additional ACLs to perform that. Let me do a quick fix by adding the paguremirroring user to git group.
So the paguremirroring user is already in git group, but the files have only 640 permissions.
640
I did chmod 660 for now and we need to set this to be true for new files somewhere. The error should be now gone. I will check the playbook to see what I can do to make it permanent.
chmod 660
And here is a PR for permanent change https://pagure.io/fedora-infra/ansible/pull-request/2047
Metadata Update from @zlopez: - Issue assigned to zlopez
Metadata Update from @zlopez: - Issue untagged with: Needs investigation - Issue tagged with: medium-trouble
I found out that I only fixed the issue for https://pagure.io and totally forgot about htpps://src.fedoraproject.org. I'm sorry about that and a big thank you to @kevin for fixing that.
The issue was similar, only the group was packager in case of https://src.fedoraproject.org instead of git.
packager
That pr has been pushed. Thanks!
Metadata Update from @kevin: - Issue close_status updated to: Fixed with Explanation - Issue status updated to: Closed (was: Open)
Just randomly found tab with #11763. I hope we have final solution now 😉
There were few remaining issues but they should be solved now and it should be permanent solution.
Log in to comment on this ticket.