As we are prepared to add more and more different tests to python, it would be nice to get multiple CI results and parallel execution.
E.g. if we have 5 short tests (15 minutes) and one 3 hours long test, I'd like to see the results of the shorts tests immediately as they finish instead of waiting for the long one to finish to see the overall results.
The tests could share a common Koji sctratch build but then execute on different builders.
If not possible, I'd satisfy with gradual delivery of individual tests results to the Pagure PR .
Thanks
Is this something that should eventually be done, or is it nonsense?
Our plan was to have all these tests as separate playbooks and the CI system will just run them in parallel, ansible is a help here. Would that work for you?
Sure! Thanks.
(Note the form a UX perspective, the need to copy-paste the entire YAML file for this is not best, yet we can aim to improve that once we have what you've said.)
Regarding the test metadata: In the bright future of ci.fmf we could indicate parallel execution of tests in the execute section similarly as we have the recent example for test isolation?
ci.fmf
execute
Perhaps something like this:
execute: how: restraint parallel: true
This probably would not apply for restraint but could be used for other "executors" which would support it. Does that make sense? @mvadkert, @happz?
According to https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/BEBBRSXIETUYC4WHTWDUT6SRQS2SKLO3/ - even with separate playbooks, currently it is not possible to gate on individual playbooks.
Login to comment on this ticket.