#417 Use of image type 'boot' for ostree_installer images is ambiguous
Closed: Fixed 7 years ago Opened 7 years ago by adamwill.

Impossible to distinguish between Workstation netinst and Workstation Atomic installer images with current metadata

In the Fedora 25 Beta-1.1 compose, the Workstation Atomic installer image showed up for the first time...and this gives us a major metadata headache. Check out the image dicts for the Workstation Atomic installer and the Workstation netinst, from https://kojipkgs.fedoraproject.org/compose/25/Fedora-25-20161005.0/compose/metadata/images.json :

                {
                    "arch": "x86_64", 
                    "bootable": true, 
                    "checksums": {
                        "sha256": "ca2aaa84009b55798eb62ab68f5d109a139f0471443e93fb600b1768555435b3"
                    }, 
                    "disc_count": 1, 
                    "disc_number": 1, 
                    "format": "iso", 
                    "implant_md5": "942e5efbada935b7376a63aeece53123", 
                    "mtime": 1475647664, 
                    "path": "Workstation/x86_64/iso/Fedora-Workstation-dvd-x86_64-25_Beta-1.1.iso", 
                    "size": 2345664512, 
                    "subvariant": "Workstation", 
                    "type": "boot", 
                    "volume_id": "Fedora-25-x86_64"
                }, 
                {
                    "arch": "x86_64", 
                    "bootable": true, 
                    "checksums": {
                        "sha256": "24d4f33576ea53570b8da9dd0196c5cdeab5b267b37109fe5137b89c46d25db8"
                    }, 
                    "disc_count": 1, 
                    "disc_number": 1, 
                    "format": "iso", 
                    "implant_md5": "fad8d7a6ca6310e5c8ffeccc83f3a079", 
                    "mtime": 1475638810, 
                    "path": "Workstation/x86_64/iso/Fedora-Workstation-netinst-x86_64-25_Beta-1.1.iso", 
                    "size": 426770432, 
                    "subvariant": "Workstation", 
                    "type": "boot", 
                    "volume_id": "Fedora-WS-dvd-x86_64-25"
                }

there is, AFAICT, literally no way you could isolate a combination of metadata fields which would let you identify 'the Workstation netinst' or 'the Workstation Atomic installer' in any given compose (without getting into parsing the file name, which is what productmd is supposed to save us from having to do). Up until now the identifier I have been using is subvariant-type-format, but for both images, that is Workstation-boot-iso. I can't see any other field that could be added to the formula that would work to distinguish them.

The easiest way to fix this, probably, is to change the 'type' for Atomic installer images. The question is, what to?

Without changing productmd, we only have a limited set of choices, because productmd defines the valid 'type' values:

https://github.com/release-engineering/productmd/blob/master/productmd/images.py#L42-L44

#: supported image types
SUPPORTED_IMAGE_TYPES = ['boot', 'cd', 'docker', 'dvd', 'ec2', 'kvm', 'live',
'netinst', 'p2v', 'qcow2', 'raw-xz', 'rescue', 'vagrant-libvirt', 'vagrant-virtualbox']

The only one that could vaguely match is 'dvd'. It'd achieve the immediate purpose, but it's not really a good choice, because we kinda already know what DVD means: it means a 'traditional' installer image with a bunch of RPMs on it. It shouldn't really also cover Atomic installer images, I don't think. So I guess I'd say we should send a PR to add a new type to productmd. But what to call it?

Before Fedora was using productmd, when the fedfind library was still used to identify images by parsing their filenames, I had a value for fedfind's equivalent field called canned, which was used for Atomic installer images (the idea being they were installer-type images with a pre-determined payload, a 'canned' payload). This is an option, but I'm not sure it's the best choice. I can foresee a problem with it: imagine down the line we invented some other image type which could also be described as 'an installer image which deploys pre-determined content', but which didn't deploy an ostree-based system. Wouldn't we want to distinguish between ostree-canned installer images and other-canned installer images? We might even build both for one variant and run into this problem again. So I think I'm cooling on canned.

Another choice would be just atomic, but I don't think it doesn't really match the concept that the type field expresses. What we really want to express, I think, is 'installer image that deploys a payload containing an ostree-based distribution'. I mean, Atomic is also a variant; having it as both a variant and a type seems odd. It's also possible to imagine that we might build an image that also deployed a pre-determined, ostree-based payload that wasn't considered an 'Atomic' image, so again, that's a reason not to just call the type 'atomic'.

We do have a precedent for doing a sort of 'general type, subtype' thing in the type value: there's vagrant-libvirt and vagrant-virtualbox. I actually hate that those use '-' as the separator, because we also use _ as a separator in various other metadata values (e.g. arch x86_64, subvariant Cloud_Base), which makes constructing a string out of multiple metadata values a nightmare because there are already values containing both the obvious separators. But it seems like I'm on a losing track trying to get that fixed, so we could follow the precedent and have canned-ostree, or even dvd-ostree or boot-ostree.


I forgot to mention, there's an alternative, a bit more radical solution here, which is to add an entirely new metadata field called something like deployment, with the possible values (for now) rpm or ostree...

As 'identical' images causes quite a lot of real-world problems for our whole Fedora testing process, I can't wait for a fix here, I've had to go ahead and do a 'hotfix' in the meantime. I've done this by having fedfind effectively override the type for these images, and making sure all the stuff I look after - openQA, wikitcms, fedora_nightlies, etc - goes through the fedfind override. I chose to use the dvd-ostree value for this override, as that's the one Dennis liked the most, so heads up: you'll see the stuff that runs through my tools (openQA, wiki validation pages, etc. etc.) using dvd-ostree pretty soon.

I see the problem and fully agree it is a major issue. As a shorter term solution adding dvd-ostree as a valid type seems like the best option. It is already fixed in productmd, and a new release should follow shortly. Now we need to change the type assigned by Pungi.

Pungi patch is in #421. It shouldn't be merged until new productmd is out.

Thanks a lot for the swift attention.

@lsedlar changed the status to Closed

7 years ago

Login to comment on this ticket.

Metadata