Learn more about these different git repos.
Other Git URLs
Both prod and stg versions of ftbfs_weekly_reminder.py weekly cron script are failing.
prod fails with:
xmlrpc.client.Fault: <Fault 306: 'The API key you specified has been revoked by the user that created it.'>
stg fails with:
xmlrpc.client.Fault: <Fault 306: 'The API key you specified is invalid. Please check that you typed it correctly.'>
Metadata Update from @phsmoura: - Issue tagged with: medium-gain, medium-trouble, ops
I have created a new token for prod, not sure where we store it for staging.
Script is working fine now, but outputs:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Oct 8 04:22:02 UTC 2023
Would be nice to supress that and only send email on failures.
Metadata Update from @humaton: - Issue assigned to humaton
Staging:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Jul 21 04:22:02 UTC 2024 Cloning into 'releng'... Gathering bugz, this can take a while... Traceback (most recent call last): File "/tmp/ftbfs_reminder.hsYbjk/releng/scripts/./ftbfs_weekly_reminder.py", line 131, in <module> bugz = new_ftbfs_bugz() ^^^^^^^^^^^^^^^^ File "/tmp/ftbfs_reminder.hsYbjk/releng/scripts/./ftbfs_weekly_reminder.py", line 79, in new_ftbfs_bugz return bzapi.query(query) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/base.py", line 1319, in query r = self._backend.bug_search(query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 185, in bug_search return self._xmlrpc_proxy.Bug.search(paramdict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 1122, in __call__ return self.__send(self.__name, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 125, in _ServerProxy__request ret = ServerProxy._ServerProxy__request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 1461, in __request response = self.__transport.request( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 100, in request return self.__request_helper(url, request_body) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 47, in __request_helper return self.parse_response(response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 84, in parse_response return unmarshaller.close() ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 668, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault: <Fault 306: 'The API key you specified is invalid. Please check that you typed it correctly.'> ftbfs_weekly_reminder.py had an error condition Look in /tmp/ftbfs_reminder.hsYbjk for more info
prod:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Jul 21 04:22:01 UTC 2024 Cloning into 'releng'... Gathering bugz, this can take a while... There are 20 NEW bugz, will send a reminder Updated 20 bugz Moving /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED to /home/ftbfs/.cache/FTBFS_weekly_reminder/~ALREADY_FILED, all bugz filed
So, we still need to fix the stg api key and we need to in general fix the script to not output anything unless there are errors. :)
Metadata Update from @jnsamyak: - Issue assigned to jnsamyak (was: humaton)
@kevin, do you think this will work: https://pagure.io/releng/pull-request/12365? I see there is no external print info for the login tho!
I merged the PR today, let's see if this worked or not! :thought_balloon:
Hey @kevin
How do I get these logs to check for the condition above?
I tried looking at it from compose-x86-01 machine and then grepping for logs
I got this
sudo grep ftbfs /var/log/cron Sep 29 04:22:00 compose-x86-01 run-parts[1457928]: (/etc/cron.weekly) starting ftbfs.cron Sep 29 04:22:39 compose-x86-01 run-parts[1457928]: (/etc/cron.weekly) finished ftbfs.cron
So, the output of this is going to root@fedoraproject.org (ie, sysadmin-main). It's a bit confusing because this cron job is set in cron.weekly, so the Subject of the email is:
Subject: Cron root@compose-x86-01 [ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.weekly
For the prod one:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Sep 22 04:22:00 UTC 2024 Cloning into 'releng'... Gathering bugz, this can take a while... There are 20 NEW bugz, will send a reminder Updated 20 bugz Moving /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED to /home/ftbfs/.cache/FTBFS_weekly_reminder/~ALREADY_FILED, all bugz filed
for the staging one:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Sep 22 04:22:02 UTC 2024 Cloning into 'releng'... Gathering bugz, this can take a while... Traceback (most recent call last): File "/tmp/ftbfs_reminder.Dm1wPY/releng/scripts/./ftbfs_weekly_reminder.py", line 131, in <module> bugz = new_ftbfs_bugz() ^^^^^^^^^^^^^^^^ File "/tmp/ftbfs_reminder.Dm1wPY/releng/scripts/./ftbfs_weekly_reminder.py", line 79, in new_ftbfs_bugz return bzapi.query(query) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/base.py", line 1319, in query r = self._backend.bug_search(query) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 185, in bug_search return self._xmlrpc_proxy.Bug.search(paramdict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 1122, in __call__ return self.__send(self.__name, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 125, in _ServerProxy__request ret = ServerProxy._ServerProxy__request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 1461, in __request response = self.__transport.request( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 100, in request return self.__request_helper(url, request_body) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 47, in __request_helper return self.parse_response(response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/site-packages/bugzilla/_backendxmlrpc.py", line 84, in parse_response return unmarshaller.close() ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/xmlrpc/client.py", line 668, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault: <Fault 306: 'The API key you specified is invalid. Please check that you typed it correctly.'> ftbfs_weekly_reminder.py had an error condition Look in /tmp/ftbfs_reminder.Dm1wPY for more info
idea: Move this job to a regular cron.d entry and have it set MAILTO=releng-cron@lists.fedoraproject.org ? (Unless there's anything sensitive there, but I don't think there is).
+1, I dont think it has anything sensitive info that getting logged here! And, since the PR runs weekly, the last login was on 22, 29 Sept, and then October 6, which is today, so we might be able to check right for the fix, right>?
The run on the 6th:
/etc/cron.weekly/ftbfs.cron: Last login: Sun Sep 29 04:22:02 UTC 2024
stg:
So, they are both working, but still generating the 'Last login' output, causing an email to be sent.
:/ I'll try to look more on where this is coming from?
Log in to comment on this ticket.