I am not sure I understand this issue in full:

1. We want that a DESKTOP upgrade test will upload its image similarly to what install_default_upload does?
2. Do we have such postinstall tests that we can utilize or do we need to write some new ones?
3. If new tests are needed, what do we want to test specifically?

1. yes.
2. by "post-install tests" I just meant the tests we have already that run after installation, the same tests we run on workstation-live-iso.
3. no new tests.

1. yes.
2. by "post-install tests" I just meant the tests we have already that run after installation, the same tests we run on workstation-live-iso.
3. no new tests.

Ok, I will look into it. It seems that a couple of new variables could do the majority of the task then.

So, I have tried to run an upgrade test that would leave a bootable image for further testing and I have found out the following points:

• The upgrade test can be given a STORE_HDD variable to upload the upgraded image.
• The image can be then utilized when it is assigned to the product's DEPLOY_UPLOAD_TEST variable.
• Then, when postinstall tests are scheduled in that product, they will run on this image.

However there are pitfalls:

• Currently, the tests ask for disk_%FLAVOR%_%MACHINE%.qcow2 so either there is a bunch of new records in the templates with a different image name, or the image created in an upgrade name needs the same name, which I believe would cause clashes between the uploaded images in the scope of one product.
• Also all upgrade tests are now part of the Universal product where no DESKTOP variable is defined by default, so the postinstall tests will not us _graphical_wait_login but rather _console_wait_login and they will fail immediately on graphical images.
• I am not sure if I can upload an image in one product and use that file from a different product. I have not tried that yet. On one hand, I would think that it is possible, because the uploaded images end up in the same directory on the OpenQA machine, but on the other hand, I am not sure if I can make the products run one after another in the strict order to avoid race conditions.

As I see that, I think that there should be one extra product for any upgrade image we would like to test and there we can run any postinstall tests we choose to by assigning them to that product. Otherwise, I am not sure how this should be handled.

What do you think @adamwill ?

Currently, the tests ask for disk_%FLAVOR%_%MACHINE%.qcow2 so either there is a bunch of new records in the templates with a different image name, or the image created in an upgrade name needs the same name, which I believe would cause clashes between the uploaded images in the scope of one product.

There's actually a clever mechanism in openQA which should avoid clashes here. When you use STORE_HDD_X, there's a transparent mechanism which adds the job ID as a prefix to the filename as stored on disk, and when a child of that job looks for a disk asset, will look for the filename with the parent job's prefix. You can see this on our instances, if you look - for instance right now, in /var/lib/openqa/share/factory/hdd on the lab instance, you'll see 01286917-disk_Server-dvd-iso_64bit.qcow2, 01286919-disk_Server-dvd-iso_64bit.qcow2, 01287451-disk_Server-dvd-iso_64bit.qcow2 and so on. So I think this would not actually be a practical issue, though it could be confusing.

Also all upgrade tests are now part of the Universal product where no DESKTOP variable is defined by default, so the postinstall tests will not us _graphical_wait_login but rather _console_wait_login and they will fail immediately on graphical images.

Yeah, this is a problem indeed. It would be possible to hack around it somehow, but I can't offhand think of any nice ways.

I am not sure if I can upload an image in one product and use that file from a different product. I have not tried that yet. On one hand, I would think that it is possible, because the uploaded images end up in the same directory on the OpenQA machine, but on the other hand, I am not sure if I can make the products run one after another in the strict order to avoid race conditions.

You can do it, but yes, ordering is a problem, and also we would have your first problem in this case. If you use PUBLISH_HDD_X instead of STORE_HDD_X, the image is uploaded without the job ID prefix. So the filename is known and you can use it from a different flavor. But then you do have the issue of collisions, and you also have the scheduling problem you mentioned (how do you make sure the job that uses the disk starts after the one that creates it?) So this is probably not a good approach to attempt.

On the whole I think my initial idea would be to just move the upgrade tests into their own flavors. That should, I think, solve most of the problems. If that's the same thing you meant in your final paragraph, then yeah, I agree with that idea.

On the whole I think my initial idea would be to just move the upgrade tests into their own flavors. That should, I think, solve most of the problems. If that's the same thing you meant in your final paragraph, then yeah, I agree with that idea.

Coming back to this one.
Yeah, I think I meant it that way. I just want to make sure, what exactly is a flavour. When we speak about Workstation-live-iso, is it a flavour?

Yes. You can see it's literally set as FLAVOR in the test settings :) This is the openQA term for, you know, "what kind of an image is this".

Now, there is PR #253 that has been merged to solve part of this issue for Workstation.
Do we also plan to do the similar for Server and possibly other products?

Ideally, I think it would make sense. Practically, I wonder how badly it'll affect load on the worker hosts. Probably best to leave it for January at this point.