As per taskotron task creation guideline , I see that taskotron task should define taskotron_supported_arches variable only to x86_64, i386, armhfp. It seems, task-abicheck is also following same now. For example, latest elfutils update ran abicheck task only for sub-packages from x86_64 and armv7hl arches https://taskotron.fedoraproject.org/artifacts/all/384bf87c-26ec-11e8-8c58-525400fc9f92/tests.yml/elfutils-0.170-10.fc27.log
Underlying tool Task-abicheck uses is abipkgdiff from libabigail. abipkgdiff can run (and expected behavior) on single host(from any arch) to give right ABI comparison result for sub-packages for all arches for a package update.
What I suggest is, we should continue running task-abicheck for all arches for which package are built in primary koji (which includes: x86_64, i686, aarch64, armv7hl, ppc64le, ppc64, s390x). This will help us to keep a new update arriving in Fedora ABI stable across all arches.
@dodji @kparal @tflink Thoughts?
Fwiw, I agree that ABI verification should happen for all supported architectures. I am not sure why it is now restricted to just a subset of all arches. What is the reason for the restriction?
As per taskotron task creation guideline , I see that taskotron task should define taskotron_supported_arches variable only to x86_64, i386, armhfp.
That's just an example, not "should".
It seems, task-abicheck is also following same now. For example, latest elfutils update ran abicheck task only for sub-packages from x86_64 and armv7hl arches https://taskotron.fedoraproject.org/artifacts/all/384bf87c-26ec-11e8-8c58-525400fc9f92/tests.yml/elfutils-0.170-10.fc27.log
Yes, you can see the actual variables values in a different file in the same results directory: https://taskotron.fedoraproject.org/artifacts/all/384bf87c-26ec-11e8-8c58-525400fc9f92/tests.yml/taskotron/ansible_vars.json.gz
What I suggest is, we should continue running task-abicheck for all arches for which package are built in primary koji (which includes: x86_64, i686, aarch64, armv7hl, ppc64le, ppc64, s390x). Fwiw, I agree that ABI verification should happen for all supported architectures. I am not sure why it is now restricted to just a subset of all arches. What is the reason for the restriction?
What I suggest is, we should continue running task-abicheck for all arches for which package are built in primary koji (which includes: x86_64, i686, aarch64, armv7hl, ppc64le, ppc64, s390x).
The default taskotron_supported_arches value can of course be overriden. Before we converted the task to STI (based on Ansible), the arch selection was overriden to all supported Koji arches, are stated above. However, with previous task format, we were able to detect when the task was active (writing to stdout/logs). That is no longer the case with STI tasks. The outcome was that abicheck tasks were killed very frequently, because they took longer than the default task timeout (currently 20 minutes). We implemented timeout configuration and added 20 extra minutes to the execution timeout (taskotron_keepalive_minutes in tests.yml). But even with 40 minutes timeout, many of the executions timed out. Because abicheck was shortly used as a blocking task (part of Fedora gating decision process), I decided to evaluate only the bare minimum we need to do (i.e. the Fedora primary arches). This radically improved the completion rate of task-abicheck. It still times out for some packages, but very rarely now. task-abicheck is now excluded from gating, until #8 is fixed, but when it is, it will be likely re-enabled again. So it doesn't probably make sense to increase the number of evaluated arches for that short period of time.
taskotron_supported_arches
taskotron_keepalive_minutes
tests.yml
I'd like to implement some features in the future that would allow us to track task progress as we could in the past, but nothing is written yet and it might not even work. Also, I'd like to execute the task for each arch separately in the future, which would decrease the likelihood of a task timing out for important arches. However, that's not implemented either.
Those are the reasons why I limited the number of evaluated arches to just primary arches. Of course, this is completely open to discussion.
Log in to comment on this ticket.