Learn more about these different git repos.
Other Git URLs
The https://fedoraproject.org/wiki/Fails_to_build_from_source policy says:
If a FTBFS bug remains in NEW state, a weekly reminder will be sent.
This never happened.
(Opened also as https://pagure.io/fesco/issue/1973 - feel free to close the less appropriate one.)
Metadata Update from @mohanboddu: - Issue tagged with: meeting
Metadata Update from @mohanboddu: - Issue assigned to mohanboddu
We dont have the tooling ready for it, I will work on it asap.
@zbyszek is also working on this and currently uses the Bugzilla NEEDINFO reminders for this.
The package will be orphaned after the FTBFS bug is in NEW state for 8 weeks.
This should happen this week. No remainders have been sent so far.
Is the orphaning still happening or is it blocked by the reminders?
I suppose the orphaning is blocked and it also seems that the F29 FTBFS bugs are not even set to needinfo: https://bugzilla.redhat.com/showdependencytree.cgi?id=1602938&hide_resolved=1
@zbyszek can you maybe post an update?
I am sorry that I poke at this again and again, but is there anything I can do to move this forward?
I can send the reminders myself, if this would make the policy happen. Let me know.
There are 1042 FTBFS bugzillas in NEW state. Those packages should have been orphaned already by this policy, yet nothing happened yet.
Getting unmaintained packages orphaned (and later retired) is important to get rid of old cruft, for me personally this is relevant for the "get rid of python2" effort, where plenty of packages that use python2 (yet nobody updates them) would go away. I also believe this is needed for the general health of the project.
This appears to be the consensus of FESCo anyway https://pagure.io/fesco/issue/1890
@churchyard Can you create a script to do this and send it as PR. As I dont have time for this, I would appreciate your help.
Thanks
I'll have a look in couple days.
Just to make sure, the script will:
Here's a script: https://github.com/hroncok/ftbfs_bugzilla/blob/master/ftbfs_weekly_reminder.py - it is public domain.
It updates bugz one by one and maintains a list of already updated bugz (at filesystem level) = it can be interrupted and run again.
I guess updating in bulk would be faster, but:
So this is slow but quite reliable.
There is also bugzilla.log that gives you all the debug information needed.
The script has a TESTING boolean, set to True if you really want to run this, keep at False to use partner-bugzilla and fake bugz. Uses click and python-bugzilla (available from PyPI or RPMs). Tested on Python 3.7 only, should work with 3.6 as well.
Let me know if this is not what you wanted.
Is the script OK for what you needed? How can I help? Shall I run it myself?
https://pagure.io/releng/pull-request/7881
A week before the mass branching any packages which still have open FTBFS bugs from the previous release will be retired.
That shall be 2019-02-12.
We have 1043 open F29 FTBFS bugs. Are we going to retire 1043 packages in February?
@humaton will talk to @mohanboddu about this ticket.
We have a new FTBFS tracker for a new release. Can we get this started please? How can I help?
So, we need to set this up in infrastructure.
It looks like it uses an interactive login? Can we have it take a user/token for running in cron?
I guess we also need to determine what user we want to use here... releng? bodhi? something new for just this use?
Do you have an example of such script so I don't need to reinvent the wheel?
I guess we also need to determine what user we want to use here...
Releng files those bugs, so I guess we can reuse that user as well.
There is this one:
https://github.com/fedora-infra/fas/blob/develop/tools/export-bugzilla.py
Yeah, releng user makes the most sense I guess...
from turbogears import config, yummy :(
from turbogears import config
Do you have an example config I can see to rewrite this without turbogears? I guess I can use plain ConfigParser.
I'm going to assume this for now:
[bugzilla] username = releng password = hunter2
Adjusted https://pagure.io/releng/pull-request/7881 to use a config file. Adjust the config file paths as desired.
Can we get the cron set up please? I'll try to talk to @kevin on IRC on Monday, so we can work on it together.
Kevin is out on vacation until the 17th. I will try and look at what needs to be done on Monday.
Mohan and I deployed this this morning. It should start bugging people on the weekly cron.weekly run
Metadata Update from @smooge: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Thanks.
While I have seen some of the comments, last time they have happened at 2019-06-17. This is apparently not happening weekly.
Take for example this randomly selected F30FTBFS bug:
The comments happened:
If more comments appear today, we probably just run it bi-weekly instead of weekly.
Metadata Update from @churchyard: - Issue status updated to: Open (was: Closed)
Cron job script was hosed. I have fixed the script and I am now running the script by hand.
OK running the script, it gave the following output:
[root@compose-x86-01 ~][PROD]# su ftbfs /etc/cron.weekly/ftbfs.cron Cloning into 'releng'... remote: Counting objects: 7869, done. remote: Compressing objects: 100% (3487/3487), done. remote: Total 7869 (delta 4422), reused 7712 (delta 4286) Receiving objects: 100% (7869/7869), 1.36 MiB | 1.89 MiB/s, done. Resolving deltas: 100% (4422/4422), done. Loading bug IDs from /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED. Will not fill those. Remove /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED to stop this from happening. Gathering bugz, this can take a while... There are 755 NEW bugz, will send a reminder Will ignore 758 bugz from /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED Will update 25 bugz [####################################] 100% Finished! Updated 0 bugz Failed to update 25 bugz https://bugzilla.redhat.com/show_bug.cgi?id=1674669 https://bugzilla.redhat.com/show_bug.cgi?id=1674680 https://bugzilla.redhat.com/show_bug.cgi?id=1674753 https://bugzilla.redhat.com/show_bug.cgi?id=1674783 https://bugzilla.redhat.com/show_bug.cgi?id=1674817 https://bugzilla.redhat.com/show_bug.cgi?id=1675022 https://bugzilla.redhat.com/show_bug.cgi?id=1675036 https://bugzilla.redhat.com/show_bug.cgi?id=1675097 https://bugzilla.redhat.com/show_bug.cgi?id=1675108 https://bugzilla.redhat.com/show_bug.cgi?id=1675364 https://bugzilla.redhat.com/show_bug.cgi?id=1675407 https://bugzilla.redhat.com/show_bug.cgi?id=1675415 https://bugzilla.redhat.com/show_bug.cgi?id=1675454 https://bugzilla.redhat.com/show_bug.cgi?id=1675586 https://bugzilla.redhat.com/show_bug.cgi?id=1675593 https://bugzilla.redhat.com/show_bug.cgi?id=1675684 https://bugzilla.redhat.com/show_bug.cgi?id=1675784 https://bugzilla.redhat.com/show_bug.cgi?id=1675892 https://bugzilla.redhat.com/show_bug.cgi?id=1675899 https://bugzilla.redhat.com/show_bug.cgi?id=1675918 https://bugzilla.redhat.com/show_bug.cgi?id=1676017 https://bugzilla.redhat.com/show_bug.cgi?id=1676164 https://bugzilla.redhat.com/show_bug.cgi?id=1676181 https://bugzilla.redhat.com/show_bug.cgi?id=1676205 https://bugzilla.redhat.com/show_bug.cgi?id=1679955
I am not sure what is causing bzapi.update_bugs([bug.id], update) to give an exception.
rerunning the script on failure should do.
It seemed to fail identically on rerun
[root@compose-x86-01 ~][PROD]# su ftbfs /etc/cron.weekly/ftbfs.cron Cloning into 'releng'... remote: Counting objects: 7869, done. remote: Compressing objects: 100% (3487/3487), done. remote: Total 7869 (delta 4422), reused 7713 (delta 4286) Receiving objects: 100% (7869/7869), 1.36 MiB | 1.63 MiB/s, done. Resolving deltas: 100% (4422/4422), done. Loading bug IDs from /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED. Will not fill those. Remove /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED to stop this from happening. Gathering bugz, this can take a while... There are 755 NEW bugz, will send a reminder Will ignore 758 bugz from /home/ftbfs/.cache/FTBFS_weekly_reminder/ALREADY_FILED Will update 25 bugz [####################################] 100% Finished! Updated 0 bugz Failed to update 25 bugz https://bugzilla.redhat.com/show_bug.cgi?id=1674669 https://bugzilla.redhat.com/show_bug.cgi?id=1674680 https://bugzilla.redhat.com/show_bug.cgi?id=1674753 https://bugzilla.redhat.com/show_bug.cgi?id=1674783 https://bugzilla.redhat.com/show_bug.cgi?id=1674817 https://bugzilla.redhat.com/show_bug.cgi?id=1675022 https://bugzilla.redhat.com/show_bug.cgi?id=1675036 https://bugzilla.redhat.com/show_bug.cgi?id=1675097 https://bugzilla.redhat.com/show_bug.cgi?id=1675108 https://bugzilla.redhat.com/show_bug.cgi?id=1675364 https://bugzilla.redhat.com/show_bug.cgi?id=1675407 https://bugzilla.redhat.com/show_bug.cgi?id=1675415 https://bugzilla.redhat.com/show_bug.cgi?id=1675454 https://bugzilla.redhat.com/show_bug.cgi?id=1675586 https://bugzilla.redhat.com/show_bug.cgi?id=1675593 https://bugzilla.redhat.com/show_bug.cgi?id=1675684 https://bugzilla.redhat.com/show_bug.cgi?id=1675784 https://bugzilla.redhat.com/show_bug.cgi?id=1675892 https://bugzilla.redhat.com/show_bug.cgi?id=1675899 https://bugzilla.redhat.com/show_bug.cgi?id=1675918 https://bugzilla.redhat.com/show_bug.cgi?id=1676017 https://bugzilla.redhat.com/show_bug.cgi?id=1676164 https://bugzilla.redhat.com/show_bug.cgi?id=1676181 https://bugzilla.redhat.com/show_bug.cgi?id=1676205 https://bugzilla.redhat.com/show_bug.cgi?id=1679955 ftbfs_weekly_reminder.py had an error condition Look in /tmp/ftbfs_reminder.8AGrhM for more info
Here is the script
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
#!/bin/bash # A weekly cron job which actually does the items needed. TMPDIR=`mktemp -d /tmp/ftbfs_reminder.XXXXXX` GITREPO=https://pagure.io/releng.git SCRIPT=ftbfs_weekly_reminder.py if [ $? -eq 0 ]; then cd ${TMPDIR} git clone ${GITREPO} if [ $? -eq 0 ]; then cd releng/scripts ./${SCRIPT} if [ $? -ne 0 ]; then echo "${SCRIPT} had an error condition" echo "Look in ${TMPDIR} for more info" # Do not clean up trash exit 1 fi else echo "Unable to clone ${GITREPO}" echo "Look in ${TMPDIR} for more info" # Do not clean up trash exit 1 fi cd /tmp/ rm -rf $TMPDIR else echo "Unable to create ${TMPDIR}" exit 1 fi
You can't ask Marek Mahut mmahut@redhat.com because that account is disabled.
It seems that the script needs to distinguish temporary BZ nonsense failures against permanent ones :(
do you happen to have /tmp/ftbfs_reminder.8AGrhM ? the script loggs the exceptions.
correction, the exception is logged in bugzilla.log
Traceback (most recent call last): File "ftbfs_weekly_reminder.py", line 90, in send_reminder bzapi.update_bugs([bug.id], update) File "/usr/lib/python3.7/site-packages/bugzilla/base.py", line 1321, in update_bugs return self._proxy.Bug.update(tmp) File "/usr/lib64/python3.7/xmlrpc/client.py", line 1112, in __call__ return self.__send(self.__name, args) File "/usr/lib/python3.7/site-packages/bugzilla/transport.py", line 104, in _ServerProxy__request self)._ServerProxy__request(methodname, params) File "/usr/lib64/python3.7/xmlrpc/client.py", line 1452, in __request verbose=self.__verbose File "/usr/lib/python3.7/site-packages/bugzilla/transport.py", line 201, in request return self._request_helper(url, request_body) File "/usr/lib/python3.7/site-packages/bugzilla/transport.py", line 179, in _request_helper return self.parse_response(response) File "/usr/lib/python3.7/site-packages/bugzilla/transport.py", line 154, in parse_response return unmarshaller.close() File "/usr/lib64/python3.7/xmlrpc/client.py", line 656, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault: <Fault 32000: "You can't ask Martin Preisler <mpreisle@redhat.com> because that account is disabled.">
See https://pagure.io/releng/pull-request/8480 - it now attempts not to add needinfo if it breaks. The script output is also useful in case we want to see what redhat bugzilla accounts are gone.
Thanks @smooge @mohanboddu
Metadata Update from @churchyard: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
https://pagure.io/releng/issue/8522 is the request to retire FTBFS packages according to the policy
I have a feeling that this stopped happening again.
To clarify, the comments happened on 2019-07-02 and than stopped. No comments a week later, no comments 2 weeks later either.
I made a spalling mistake in the cron job so it was not able to run. I have corrected the cron job and run the script by hand.
Metadata Update from @smooge: - Issue status updated to: Open (was: Closed)
Script failed to run:
[root@compose-x86-01 ~][PROD]# su -s /bin/sh ftbfs sh-5.0$ sh -x /usr/local/bin/ftbfs.sh ++ mktemp -d /tmp/ftbfs_reminder.XXXXXX + TMPDIR=/tmp/ftbfs_reminder.jutVAp + GITREPO=https://pagure.io/releng.git + SCRIPT=ftbfs_weekly_reminder.py + '[' 0 -eq 0 ']' + cd /tmp/ftbfs_reminder.jutVAp + git clone https://pagure.io/releng.git Cloning into 'releng'... remote: Counting objects: 7902, done. remote: Compressing objects: 100% (3401/3401), done. remote: Total 7902 (delta 4449), reused 7835 (delta 4405) Receiving objects: 100% (7902/7902), 1.36 MiB | 741.00 KiB/s, done. Resolving deltas: 100% (4449/4449), done. + '[' 0 -eq 0 ']' + cd releng/scripts + ./ftbfs_weekly_reminder.py Gathering bugz, this can take a while... There are 681 NEW bugz, will send a reminder Traceback (most recent call last): File "./ftbfs_weekly_reminder.py", line 126, in <module> if ignore: NameError: name 'ignore' is not defined + '[' 1 -ne 0 ']' + echo 'ftbfs_weekly_reminder.py had an error condition' ftbfs_weekly_reminder.py had an error condition + echo 'Look in /tmp/ftbfs_reminder.jutVAp for more info' Look in /tmp/ftbfs_reminder.jutVAp for more info + exit 1
https://pagure.io/releng/pull-request/8560
Metadata Update from @kevin: - Issue tagged with: backlog
Login to comment on this ticket.