Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

• os-autoinst-tox : FAILURE in 3m 14s
• os-autoinst-perl : SUCCESS in 5m 19s

I assume you don't want this considered for merging till you remove the [WIP], right? Thanks!

Well, it works on my system, but it's not properly wired into fedora-openqa=
and it probably doesn't meet your requirements, that's why I marked it as =
WIP (I'd be more than happy to get this merged sooner rather than later)=2E=
So by all means please consider it for merging=2E

Perhaps this second time is not needed any more?

I also wonder, what tool are you using to create the needles? This is the first time I have seen click_point defined in them. When OpenQA calculates the click point, it basically halves the width and height values, so basically it always arrives at that exact point you are providing.

So, I tried to run the suite in my OpenQA, but on my computer, there are some problems that cause failures. My observations are based on the i3-tests branch of defolos's fork:

• in templates.fif.json the product is missing the DEPLOY_UPLOAD_TEST which causes the tests run ad hoc without waiting for the install_default_upload to finish and upload the assets.
• Tests desktop_browser, desktop_login, desktop_printing, desktop_postinstall_notifications and i3_tilling are failing on non-matching needles. The needles are there, but they only partially match (40-70%) - I have noticed that you tend to compare multiple areas - in my experience, this leads to failing needles because the chance of hitting a difference on a needle multiplies. We do not use many such needles in our current setup.

You say that the suite passes on your system, which ISO have you used to create the needles?

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

• os-autoinst-tox : FAILURE in 3m 14s
• os-autoinst-perl : SUCCESS in 4m 59s

Perhaps this second time is not needed any more?

Whoops, that was not intentional. I've removed it now.

I also wonder, what tool are you using to create the needles? This is the first time I have seen click_point defined in them. When OpenQA calculates the click point, it basically halves the width and height values, so basically it always arrives at that exact point you are providing.

I have mostly used the webui, but I've also modified a few needles using the openQA needle editor.

So, I tried to run the suite in my OpenQA, but on my computer, there are some problems that cause failures. My observations are based on the i3-tests branch of defolos's fork:

• in templates.fif.json the product is missing the DEPLOY_UPLOAD_TEST which causes the tests run ad hoc without waiting for the install_default_upload to finish and upload the assets.

Ah, I see. I forgot to add this variable to the fedora-i3-live-x86_64 product. That should be fixed now. I worked around this by setting the variable DEPLOY_UPLOAD_TEST in the opeqa-cli invocation.

• Tests desktop_browser, desktop_login, desktop_printing, desktop_postinstall_notifications and i3_tilling are failing on non-matching needles. The needles are there, but they only partially match (40-70%) - I have noticed that you tend to compare multiple areas - in my experience, this leads to failing needles because the chance of hitting a difference on a needle multiplies. We do not use many such needles in our current setup.

You say that the suite passes on your system, which ISO have you used to create the needles?

I have used the latest Fedora Rawhide Compose (when submitting, that was build 20210721.n.0). I'll make a run with 20210812.n.1 overnight and will check if it still passes locally or fix any errors that pop up.

Build succeeded.

• os-autoinst-tox : SUCCESS in 3m 09s
• os-autoinst-perl : SUCCESS in 5m 29s

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.

Build succeeded.

• os-autoinst-tox : SUCCESS in 3m 23s
• os-autoinst-perl : SUCCESS in 4m 41s

@lruzicka I've been running the i3 openqa tests for the last few composes successfully on my local box.

@lruzicka I've been running the i3 openqa tests for the last few composes successfully on my local box.

Sorry, did not see this. I am going to try it now.

@lruzicka @defolos we should really get back to this, but the pull request will need a lot of rebasing at this point, I guess.