PR#10: Support publishing container images to registries (#4) - cloud-image-uploader

cloud-image-uploader

#10 Support publishing container images to registries (#4)

Merged a month ago by jcline. Opened a month ago by adamwill.

adamwill/cloud-image-uploader handle-containers into main

Support publishing container images to registries (#4)

Adam Williamson • a month ago

bd6b7ab

Containerfile

file modified

+3 -1

		`@@ -5,7 +5,9 @@`
		`RUN dnf install -y \`
		`patch \`
		`python3-pip \`
		`- python3-hatchling`
		`+ python3-hatchling \`
		`+ skopeo \`
		`+ buildah`

		`RUN mkdir -p /srv/cloud-uploader/`
		`COPY . /srv/cloud-uploader/src`

README.md

file modified

+3 -1

		`@@ -1,9 +1,11 @@`
		`# cloud-image-uploader`

		`- An AMQP consumer that automatically uploads Cloud images to their respective homes.`
		`+ An AMQP consumer that automatically uploads Cloud and container images to their respective homes.`

		`## Configuring`

		The `fedora-messaging` service is the container entrypoint and is also used to provide most of the configuration. An example configuration file is included and the full list of options are in [fedora-messaging's configuration documentation](https://fedora-messaging.readthedocs.io/en/stable/user-guide/configuration.html). The `FEDORA_MESSAGING_CONF` environment variable should be set to the configuration file's location.

		In addition to `fedora-messaging`, the [Azure Ansible collection](https://docs.ansible.com/ansible/latest/collections/azure/azcollection/) needs Azure credentials provided. It supports using environment variables, a credentials file at `~/.azure/credentials` or the `azure-cli` credentials. For example, to authenticate via environment variables using an app registration in the Microsoft Entra ID service with a client secret, setting the `AZURE_TENANT`, `AZURE_CLIENT_ID`, and `AZURE_SECRET` along with the `AZURE_SUBSCRIPTION_ID` will be picked up by the Ansible playbook. The app registration needs to be configured in the given subscription's access control settings.
		`+`
		+ Container image upload to registries uses the `skopeo` and `buildah` commands, so for this to work, the system must be in a state such that these commands have the necessary credentials to upload to the registries configured, non-interactively. If no container registries are configured, container images will not be handled.

fedora-messaging.toml.example

file modified

		`@@ -53,6 +53,9 @@`
		`regional_replica_count = 2`
		`storage_account_type = "Standard_LRS"`

		`+ [consumer_config.container]`
		`+ registries = ["registry.fedoraproject.org", "quay.io/fedora"]`
		`+`
		`[qos]`
		`prefetch_size = 0`
		`prefetch_count = 25`

fedora_cloud_image_uploader/handler.py

file modified

+115 -28

		`@@ -3,8 +3,10 @@`
		`import logging`
		`import lzma`
		`import os`
		`+ import subprocess`
		`import tempfile`
		`import time`
		`+ from collections.abc import Iterable`
		`from typing import NamedTuple`

		`import ansible_runner`
		`@@ -25,15 +27,22 @@`
		`_log = logging.getLogger(__name__)`


		`- class CantHandle(Exception):`
		`- """An exception meaning this handler does not handle this image."""`
		`-`
		`-`
		`- class NotHandled(Exception):`
		`- """`
		`- An exception meaning this is the right handler, but we do not handle this`
		`- particular image.`
		`- """`
		`+ def _run(args: Iterable[str]):`
		`+ """Run a command and handle errors."""`
		`+ _log.debug("image_uploader running command %s", " ".join(args))`
		`+ try:`
		`+ ret = subprocess.run(args, encoding="utf-8", capture_output=True, timeout=7200)`
		`+ except subprocess.TimeoutExpired:`
		`+ _log.error("Command: %s timed out after two hours", " ".join(args))`
		`+ raise fm_exceptions.Nack()`
		`+ except OSError as err:`
		`+ _log.error("Command: %s caused error %s", " ".join(args), err)`
		`+ raise fm_exceptions.Nack()`
		`+ if ret.returncode:`
		`+ _log.error("Command: %s returned %d", " ".join(args), ret.returncode)`
		`+ _log.error("stdout: %s", ret.stdout)`
		`+ _log.error("stderr: %s", ret.stderr)`
		`+ raise fm_exceptions.Nack()`


		`class ReleaseInfo(NamedTuple):`
		`@@ -54,7 +63,10 @@`
		`self.requests = Session()`
		`retry_config = Retry(total=5, backoff_factor=1)`
		`self.requests.mount("https://", adapters.HTTPAdapter(max_retries=retry_config))`
		`- self.cloud_handlers = (self.handle_azure,)`
		`+ self.handlers = (self.handle_azure, self.handle_container)`
		`+ # tracks the container repos we got images for, for manifest`
		`+ # creation purposes`
		`+ self.container_repos = dict()`

		`def __call__(self, message: fm_message.Message):`
		`"""`
		`@@ -76,22 +88,13 @@`
		`except ff_exceptions.UnsupportedComposeError:`
		`_log.info("Skipping compose %s as it contains no images", compose_id)`
		`return`
		`+ # reset for each message`
		`+ self.container_repos = dict()`
		`relinfo = self.get_relinfo(compose)`
		`- cloud_images = [img for img in compose.all_images if img["subvariant"] == "Cloud_Base"]`
		`-`
		`try:`
		`- for image in cloud_images:`
		`- for handler in self.cloud_handlers:`
		`- try:`
		`- handler(image, relinfo)`
		`- break`
		`- except CantHandle:`
		`- # try the next handler`
		`- pass`
		`- except NotHandled:`
		`- # this means we do not handle the image, go to next image`
		`- break`
		`- _log.debug("Missing handler for '%s'", image["type"])`
		`+ for image in compose.all_images:`
		`+ for handler in self.handlers:`
		`+ handler(image, relinfo)`
		`except fm_exceptions.Nack:`
		`# If we failed to process an image, it's not likely the failure will resolve`
		`# itself in the time it takes to re-queue the message and then consume it again.`
		`@@ -99,6 +102,41 @@`
		`time.sleep(60)`
		`raise`

		`+ if self.container_repos:`
		`+ # manifest stuff`
		`+ for repo in self.container_repos:`
		`+ for registry in self.conf["container"]["registries"]:`
		`+ # something like "registry.fedoraproject.org/fedora:40"`
		`+ regname = f"{registry}/{repo}:{str(relinfo.relnum)}"`
		`+ targets = [regname]`
		`+ # we also create aliased manifests for rawhide and`
		`+ # latest stable`
		`+ if compose.release.lower() == "rawhide":`
		`+ targets.append(f"{registry}/{repo}:rawhide")`
		`+ elif relinfo.relnum == ff_helpers.get_current_release(branched=False):`
		`+ targets.append(f"{registry}/{repo}:latest")`
		`+ for target in targets:`
		`+ # wipe the manifest if it exists already`
		`+ _run(("buildah", "rmi", target))`
		`+ # create the manifest with all arches`
		`+ createargs = ["buildah", "manifest", "create", target]`
		`+ createargs.extend(`
		`+ # it's intentional that this is regname not target`
		`+ f"{regname}-{arch}"`
		`+ for arch in self.container_repos[repo]`
		`+ )`
		`+ _run(createargs)`
		`+ # push it`
		`+ pushargs = (`
		`+ "buildah",`
		`+ "manifest",`
		`+ "push",`
		`+ target,`
		`+ f"docker://{target}",`
		`+ "--all",`
		`+ )`
		`+ _run(pushargs)`
		`+`
		`def get_relinfo(self, ffrel: ff_release.Release):`
		`"""`
		`Return a namedtuple that acts as a container for some useful`
		`@@ -212,12 +250,14 @@`
		`"""`
		`Handle Azure images.`
		`"""`
		`- if image["type"] != "vhd-compressed":`
		`- raise CantHandle()`
		`+ if image.get("subvariant") != "Cloud_Base" or image.get("type") != "vhd-compressed":`
		`+ return`
		`if image["arch"] not in ("x86_64", "aarch64"):`
		`- raise NotHandled("Unsupported arch")`
		`+ # unsupported arch`
		`+ return`
		`if relinfo.relnum < 40:`
		`- raise NotHandled("Images prior to F40 aren't supported")`
		`+ # images prior to F40 aren't supported`
		`+ return`

		`with tempfile.TemporaryDirectory() as workdir:`
		`image_path = self.download_image(image, workdir, decompress=True)`
		`@@ -343,3 +383,50 @@`
		`_log.info(`
		`f"Deleted image definition {image_definition.name} since it has no versions"`
		`)`
		`+`
		`+ def handle_container(self, image: dict, relinfo: ReleaseInfo):`
		`+ """Handle container images."""`
		`+ registries = self.conf.get("container", {}).get("registries")`
		`+ if not registries:`
		`+ # we can't do anything if no registries are configured`
		`+ return`
		`+ if image["type"] not in ("docker", "ociarchive"):`
		`+ # not a known container image type`
		`+ return`
		`+ repos = {`
		`+ "Container_Toolbox": "fedora-toolbox",`
		`+ "Container_Minimal_Base": "fedora-minimal",`
		`+ "Container_Base": "fedora",`
		`+ "Silverblue": "fedora-silverblue",`
		`+ "Kinoite": "fedora-kinoite",`
		`+ "Onyx": "fedora-onyx",`
		`+ "Sericea": "fedora-sericea",`
		`+ }`
		`+ repo = repos.get(image["subvariant"])`
		`+ if not repo:`
		`+ _log.debug("Unknown subvariant %s", image["subvariant"])`
		`+ return`
		`+ if image["type"] == "docker" and relinfo.relnum < 40:`
		`+ # these are actual docker archive images`
		`+ imgformat = "docker-archive"`
		`+ else:`
		`+ # all others are OCI archives; F40+ .oci.tar.xz images`
		`+ # with type "docker" are xz-compressed OCI archives,`
		`+ # .ociarchive images with type "ociarchive" are non-`
		`+ # compressed OCI archives`
		`+ imgformat = "oci-archive"`
		`+ arch = image["arch"]`
		`+ with tempfile.TemporaryDirectory() as workdir:`
		`+ image_path = self.download_image(image, workdir, decompress=True)`
		`+ for registry in registries:`
		`+ args = [`
		`+ "skopeo",`
		`+ "copy",`
		`+ f"{imgformat}:{image_path}",`
		`+ f"docker://{registry}/{repo}:{str(relinfo.relnum)}-{arch}",`
		`+ ]`
		`+ _run(args)`
		`+ if repo in self.container_repos:`
		`+ self.container_repos[repo].append(arch)`
		`+ else:`
		`+ self.container_repos[repo] = [arch]`

tests/fixtures/cassettes/test_containers[compose0].yaml

file added

+683

		`@@ -0,0 +1,683 @@`
		`+ interactions:`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=3038`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:47 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHT2mti5Q4-osOpGWzRAAAA8I`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2217`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:47 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHTzuWhzpWHAjyjksnlAAABI4`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2228`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:47 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHT_EdkCTc_XA9jUzfswAAAhE`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2616`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:48 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUNzYCNJwtm0JAhGCugAABFc`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2778`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:48 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUPYPQ6beqzZJoe4I2AAACsE`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2545`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:48 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUGDKJnmES5WV3NZxdgAAA9I`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2540`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:49 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUc2dRVrGrvfHXIirbwAABsU`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2497`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:49 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUfrbPBQAVrZAyqhchAAABwo`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2203`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:49 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUWmti5Q4-osOpGWztwAAA9E`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Content-Type:`
		`+ - application/json`
		`+ Host:`
		`+ - pdc.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://pdc.fedoraproject.org/rest_api/v1/compose-images/Fedora-Rawhide-20240501.n.0/?page=1`
		`+ response:`
		`+ body:`
		+ string: '{"header":{"version":"1.2","type":"productmd.images"},"payload":{"images":{"Workstation":{"aarch64":[{"subvariant":"Workstation","format":"raw.xz","volume_id":null,"mtime":1714551577,"checksums":{"sha256":"169a31fd5cf10faafaba87b2342ad6475bc1d20ce3e71946d0fa2694bd4484fe"},"arch":"aarch64","size":2806539876,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/aarch64/images/Fedora-Workstation-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714550825,"checksums":{"sha256":"2e6757ccad552f5929f1a69777f3a8166985953b0331ab6386ab6af8ca0e8322"},"arch":"aarch64","size":2609154048,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/aarch64/iso/Fedora-Workstation-Live-osb-Rawhide-20240501.n.0.aarch64.iso","type":"live-osbuild"}],"x86_64":[{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714551289,"checksums":{"sha256":"81584c47d50304bf1a659c17af7b761891e8f70545eb97e1ae0cc7ff511f79ee"},"arch":"x86_64","size":2654552064,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/x86_64/iso/Fedora-Workstation-Live-osb-Rawhide-20240501.n.0.x86_64.iso","type":"live-osbuild"},{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714549615,"checksums":{"sha256":"264d04c31714ba0734940819b8bdc7863701f9cd7a16e553b5b6a5db121effa7"},"arch":"x86_64","size":2314934272,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}],"ppc64le":[{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714549972,"checksums":{"sha256":"1f19f95713627cfbb487cb32ccaf0dcaeb49717e23649c6244ace0e71f6932fe"},"arch":"ppc64le","size":2265411584,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Workstation/ppc64le/iso/Fedora-Workstation-Live-ppc64le-Rawhide-20240501.n.0.iso","type":"live"}]},"Container":{"aarch64":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548151,"checksums":{"sha256":"570b6e8f5e642df8541add9734ce6263396ac8b31410d334affd4f241161bb0e"},"arch":"aarch64","size":45765840,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548144,"checksums":{"sha256":"4410600bf5c55c2ed2d892f448d0f940f1d04bd3758df45c1214e666f909376a"},"arch":"aarch64","size":80061492,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Base-Generic.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548390,"checksums":{"sha256":"14662b170bb2a792ef59471c4f3832aec24a156a63723ae7f3189ae39055198c"},"arch":"aarch64","size":309801336,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Toolbox.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"x86_64":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548002,"checksums":{"sha256":"99762e812b170a2b5ae21ffdfcc26d6f821064c3347c3456bcfb0946b51d3e39"},"arch":"x86_64","size":47325456,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548128,"checksums":{"sha256":"22ec94af77d239c4be0d6441b57b1a36e7f5ab78da4ebeb2fa0ebc5e2d84ab99"},"arch":"x86_64","size":81582552,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Base-Generic.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548391,"checksums":{"sha256":"4338e4bf47b0f98cde51fcd90f4c8dd0ec6e44e19f066ac71a3a8f0b156bd613"},"arch":"x86_64","size":333172684,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Toolbox.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"s390x":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714547999,"checksums":{"sha256":"5126ea913a0cce891f146c98d64f7041f88ec97fdf833050b66bcb1761963e7e"},"arch":"s390x","size":47592832,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548085,"checksums":{"sha256":"b872fec000a3c09d0b0875d14ab11df3ae8fac9841c9ce2d75479d87b99b77bb"},"arch":"s390x","size":82633556,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Base-Generic.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548383,"checksums":{"sha256":"254e6199117fd50a0a402a8e0bcf0d1a497457712525e05e67bde46c6b43a6ee"},"arch":"s390x","size":295170680,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Toolbox.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"ppc64le":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548195,"checksums":{"sha256":"a1609b38c5ca8b5725a5b861e6d223ebd7efb540a5a8dcb0d124c8143edacc15"},"arch":"ppc64le","size":53859988,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548207,"checksums":{"sha256":"16232ae6ac7d85480a12307b418ea86c62097889369f241607a6da3fdc810294"},"arch":"ppc64le","size":89383320,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Base-Generic.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548405,"checksums":{"sha256":"f24b0e9d19a19e509bef289c02ce0ce017b8abaa3d94dd3e160756cfbfe9a1e8"},"arch":"ppc64le","size":317277716,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Toolbox.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}]},"Everything":{"aarch64":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-aarch64-rawh","mtime":1714543271,"checksums":{"sha256":"c8761f0c0c969b2208bc1eec38608a3d421c74168e11bf6842ce0649c0b6e2c1"},"arch":"aarch64","size":881444864,"disc_count":1,"bootable":true,"implant_md5":"eb260a2607dea1ea7b4c70a3bc3b3309","disc_number":1,"path":"Everything/aarch64/iso/Fedora-Everything-netinst-aarch64-Rawhide-20240501.n.0.iso","type":"boot"}],"x86_64":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-x86_64-rawh","mtime":1714543598,"checksums":{"sha256":"6a4c569813b8fa3269122d4de538302d212be395f2465f192c3b42c3bd29c4d6"},"arch":"x86_64","size":862216192,"disc_count":1,"bootable":true,"implant_md5":"4fada428441a95574831d3cb8b254e44","disc_number":1,"path":"Everything/x86_64/iso/Fedora-Everything-netinst-x86_64-Rawhide-20240501.n.0.iso","type":"boot"}],"s390x":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-s390x-rawh","mtime":1714543372,"checksums":{"sha256":"1a1d0489e884cee0f5611adf10dcdc2cc8cecd8a43ca72e9133835cd0c993726"},"arch":"s390x","size":538773504,"disc_count":1,"bootable":true,"implant_md5":"d75c8272c5b65a38083becafe0114e2c","disc_number":1,"path":"Everything/s390x/iso/Fedora-Everything-netinst-s390x-Rawhide-20240501.n.0.iso","type":"boot"}],"ppc64le":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-ppc64le-rawh","mtime":1714544248,"checksums":{"sha256":"06e517e99fc1ad551afc5796ba574f96940c93321ec8e1af0597c44fceef1829"},"arch":"ppc64le","size":868366336,"disc_count":1,"bootable":true,"implant_md5":"447733a53e635d41f0221cd038799cea","disc_number":1,"path":"Everything/ppc64le/iso/Fedora-Everything-netinst-ppc64le-Rawhide-20240501.n.0.iso","type":"boot"}]},"Onyx":{"x86_64":[{"subvariant":"Onyx","format":"iso","volume_id":"Fedora-Onyx-ostree-x86_64-rawh","mtime":1714546339,"checksums":{"sha256":"c7fc13f5fbd63ede8dcee60880ec9353e192d27e1298d70553987667472d468e"},"arch":"x86_64","size":2758076416,"disc_count":1,"bootable":true,"implant_md5":"eea8885d7c0c04c811dbb7fadb88c18b","disc_number":1,"path":"Onyx/x86_64/iso/Fedora-Onyx-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Server":{"aarch64":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548481,"checksums":{"sha256":"7df0a82f10cf9ff246a4367c9d2738dcfa38adeab43f6259fd59c248334e754d"},"arch":"aarch64","size":679477248,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/aarch64/images/Fedora-Server-KVM-Rawhide-20240501.n.0.aarch64.qcow2","type":"qcow2"},{"subvariant":"Server","format":"raw.xz","volume_id":null,"mtime":1714549351,"checksums":{"sha256":"606840743d5f6949f6a24a087a83ee30ba75061efccae97dc10b0a9911eb647f"},"arch":"aarch64","size":1156440656,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/aarch64/images/Fedora-Server-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-aarch64-rawh","mtime":1714547827,"checksums":{"sha256":"9254a157dd98c83ec69bd6bfa32c332132a77a244196d986e61a8e07dcef482b"},"arch":"aarch64","size":2571698176,"disc_count":1,"bootable":true,"implant_md5":"3b4bf7e119d816a9b7d7ff43bb1e7397","disc_number":1,"path":"Server/aarch64/iso/Fedora-Server-dvd-aarch64-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-aarch64-rawh","mtime":1714543282,"checksums":{"sha256":"f97a38209469f4aabebf80734d97f672d0e7a76599178e627f551be0efca705d"},"arch":"aarch64","size":891131904,"disc_count":1,"bootable":true,"implant_md5":"56a7d1aa0db7f8885ce2bb582431c20a","disc_number":1,"path":"Server/aarch64/iso/Fedora-Server-netinst-aarch64-Rawhide-20240501.n.0.iso","type":"boot"}],"x86_64":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548432,"checksums":{"sha256":"874dca83ba136eda1395d5b4195252b80c9c46586b5d0959cab8a2228fa87981"},"arch":"x86_64","size":663355392,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/x86_64/images/Fedora-Server-KVM-Rawhide-20240501.n.0.x86_64.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-x86_64-rawh","mtime":1714547835,"checksums":{"sha256":"28f2da7d8092d8d9fdf9b2e55a6c01cb8df4d91040698b4e5eeaefb59bc0562e"},"arch":"x86_64","size":2659516416,"disc_count":1,"bootable":true,"implant_md5":"09ab21ecd902b709225a183bdb4d221f","disc_number":1,"path":"Server/x86_64/iso/Fedora-Server-dvd-x86_64-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-x86_64-rawh","mtime":1714544390,"checksums":{"sha256":"284d59258c0097df13b6e534e7286cc0aef3ff97355867c958b50ad1fbcefbd2"},"arch":"x86_64","size":872001536,"disc_count":1,"bootable":true,"implant_md5":"415e2b42be4a7ab863b531a9f103609b","disc_number":1,"path":"Server/x86_64/iso/Fedora-Server-netinst-x86_64-Rawhide-20240501.n.0.iso","type":"boot"}],"s390x":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548425,"checksums":{"sha256":"fb3c65a91db222dd53642ddfc8bc79f93d6b368daba2de08fa29995c56b51f25"},"arch":"s390x","size":642777088,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/s390x/images/Fedora-Server-KVM-Rawhide-20240501.n.0.s390x.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-s390x-rawh","mtime":1714548033,"checksums":{"sha256":"1b01c52808aca1e6612318816d9f23d28731433213b9dca0f5001ac176e571f6"},"arch":"s390x","size":2002780160,"disc_count":1,"bootable":true,"implant_md5":"4c9027c3bdf2355b1bd44d2e1510e20b","disc_number":1,"path":"Server/s390x/iso/Fedora-Server-dvd-s390x-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-s390x-rawh","mtime":1714543378,"checksums":{"sha256":"bd5408c0fef7d15cb9ecefd6890473cfea0c8eb2c3ac87eaeb03469d7b8bc05a"},"arch":"s390x","size":549619712,"disc_count":1,"bootable":true,"implant_md5":"239d338c78263b5528a0ef1a2da78540","disc_number":1,"path":"Server/s390x/iso/Fedora-Server-netinst-s390x-Rawhide-20240501.n.0.iso","type":"boot"}],"ppc64le":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714555324,"checksums":{"sha256":"8697bb87795aeb0cfdbf67683c72672e5390c0c26d8c456147b3c237a35c6467"},"arch":"ppc64le","size":684392448,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/ppc64le/images/Fedora-Server-KVM-Rawhide-20240501.n.0.ppc64le.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-ppc64le-rawh","mtime":1714548000,"checksums":{"sha256":"1eb107a7627f035ab3fe6f21db00b2b5d6766af991453287db444edd5532b625"},"arch":"ppc64le","size":2409299968,"disc_count":1,"bootable":true,"implant_md5":"52aa0d9a2a7e40ed64c6cc301020308e","disc_number":1,"path":"Server/ppc64le/iso/Fedora-Server-dvd-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-ppc64le-rawh","mtime":1714543594,"checksums":{"sha256":"f74d20418c881571be50a2184f240d13b8cfdf7bbad72bfc2571bb819674390a"},"arch":"ppc64le","size":879071232,"disc_count":1,"bootable":true,"implant_md5":"5e55736ea6f1dc86ce1c22b93e8fa0b3","disc_number":1,"path":"Server/ppc64le/iso/Fedora-Server-netinst-ppc64le-Rawhide-20240501.n.0.iso","type":"boot"}]},"Labs":{"aarch64":[{"subvariant":"Python_Classroom","format":"raw.xz","volume_id":null,"mtime":1714549628,"checksums":{"sha256":"53517e834f444d1bbfdb95506d3c97d6563736c63d23fcb7cb0485c8e8555345"},"arch":"aarch64","size":2717602640,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/aarch64/images/Fedora-Python-Classroom-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"}],"x86_64":[{"subvariant":"Python_Classroom","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548988,"checksums":{"sha256":"85eb263a920688d56d5e74958161ced4044578d7093b0018d09b78245712c63a"},"arch":"x86_64","size":1567637359,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Python-Classroom-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-libvirt.box","type":"vagrant-libvirt"},{"subvariant":"Python_Classroom","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714549060,"checksums":{"sha256":"81685cb0637678d5111e8b50dc61e94df119a2c2bb3b2ec216d04d35c5356527"},"arch":"x86_64","size":1589186560,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Python-Classroom-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Scientific","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714549937,"checksums":{"sha256":"6a9b6f17eb655962a8b3e343f09ed06cb5fca92ad3faef6a01215ae673a62bad"},"arch":"x86_64","size":4906517741,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Scientific-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-libvirt.box","type":"vagrant-libvirt"},{"subvariant":"Scientific","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714550139,"checksums":{"sha256":"29547a3dc363baf76c26c53350a066d76b4287e644019d5f3e43c16e8aad196c"},"arch":"x86_64","size":4963287040,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Scientific-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Astronomy_KDE","format":"iso","volume_id":null,"mtime":1714549753,"checksums":{"sha256":"eedb523885c20bfb5b246563def3e4a593d7698d7847923cf5292a2f726ab772"},"arch":"x86_64","size":4663750656,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Astronomy_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Comp_Neuro","format":"iso","volume_id":null,"mtime":1714549174,"checksums":{"sha256":"78a2911eb3c6fe4f86bbdcc93930eb8b2172f8b4971e5f83324bc496c1d9ad3f"},"arch":"x86_64","size":3089092608,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Comp_Neuro-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Games","format":"iso","volume_id":null,"mtime":1714549710,"checksums":{"sha256":"bd14181af753ff6d6273d0cc6575b2b13ee601564f526ab43c8060e9a44a8833"},"arch":"x86_64","size":6911944704,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Games-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Jam_KDE","format":"iso","volume_id":null,"mtime":1714549112,"checksums":{"sha256":"d612fc08962b47f04a6cc7549f45d7deb8740c0cf7838bd48423d4147aa2803f"},"arch":"x86_64","size":3543447552,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Jam_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Python_Classroom","format":"iso","volume_id":null,"mtime":1714549398,"checksums":{"sha256":"5f0fd5c2f81e6838409adfd70f71f532a73435505fd939f6f1c78c9ba57795bd"},"arch":"x86_64","size":2366535680,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Python-Classroom-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Robotics","format":"iso","volume_id":null,"mtime":1714549620,"checksums":{"sha256":"a91c562e1e2878977ec7639e7fe6056acc649822456fd4d50f9184dec9ee6376"},"arch":"x86_64","size":3167330304,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Robotics-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Scientific_KDE","format":"iso","volume_id":null,"mtime":1714549887,"checksums":{"sha256":"cdb127b1b26e6b1b4541be85c890bc6a20f36c58e596d77042d6b99d61d40c55"},"arch":"x86_64","size":5520687104,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Scientific_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Security","format":"iso","volume_id":null,"mtime":1714549047,"checksums":{"sha256":"ba32f7df92892f6185e46349e825c995ba81c5a26b482e46554079f22b4da894"},"arch":"x86_64","size":2481698816,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Security-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}]},"Silverblue":{"aarch64":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-aarch64-rawh","mtime":1714546273,"checksums":{"sha256":"b5a25b696cc0fdea442671eebcbb999287378e87542cfdb4b68b52dd2bd0ef4b"},"arch":"aarch64","size":3620956160,"disc_count":1,"bootable":true,"implant_md5":"461ca41e418493e1475d5bcd5fc1546f","disc_number":1,"path":"Silverblue/aarch64/iso/Fedora-Silverblue-ostree-aarch64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"x86_64":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-x86_64-rawh","mtime":1714546928,"checksums":{"sha256":"c138b73ec6e460d2ef300d04052e5f851e22d97bc00b96663a0b19daf37da973"},"arch":"x86_64","size":3640899584,"disc_count":1,"bootable":true,"implant_md5":"ac049c322f096d2dbdd55b7369048584","disc_number":1,"path":"Silverblue/x86_64/iso/Fedora-Silverblue-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"ppc64le":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-ppc64le-rawh","mtime":1714547457,"checksums":{"sha256":"02951538e73b9a53657e667a4fe745ba31ad70c9a07a445ba299369c487f3047"},"arch":"ppc64le","size":3572103168,"disc_count":1,"bootable":true,"implant_md5":"8a9f0707386f692fc93ae051f97487fb","disc_number":1,"path":"Silverblue/ppc64le/iso/Fedora-Silverblue-ostree-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Cloud":{"aarch64":[{"subvariant":"Cloud_Base","format":"raw.xz","volume_id":null,"mtime":1714548549,"checksums":{"sha256":"761269846a3fb0750fdf3853cc7838189ee1317c376e45de4225a6364ce51907"},"arch":"aarch64","size":372905420,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-AmazonEC2.aarch64-Rawhide-20240501.n.0.raw.xz","type":"raw-xz"},{"subvariant":"Cloud_Base","format":"vhd.xz","volume_id":null,"mtime":1714548597,"checksums":{"sha256":"aa977ff3c52903c0000338914b4c0691f8d857675742ac0bda12ed8af6b6fd71"},"arch":"aarch64","size":438226428,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Azure.aarch64-Rawhide-20240501.n.0.vhdfixed.xz","type":"vhd-compressed"},{"subvariant":"Cloud_Base","format":"tar.gz","volume_id":null,"mtime":1714548410,"checksums":{"sha256":"acdb1bd9065c6a648f7f45ed68ed001a333f9d1fee96768a758cf56885e7191f"},"arch":"aarch64","size":415969669,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-GCE.aarch64-Rawhide-20240501.n.0.tar.gz","type":"docker"},{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548405,"checksums":{"sha256":"ceaee75dd6a3a6c4244a30eb59184a935bbc0d004dce13f29f22a62631819b4e"},"arch":"aarch64","size":415891456,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Generic.aarch64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base_UKI","format":"qcow2","volume_id":null,"mtime":1714548578,"checksums":{"sha256":"2487abdf4e5ae7a9be4439833ae769ea946bb7f14632236f65b91913001ee1d7"},"arch":"aarch64","size":430243840,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-UEFI-UKI.aarch64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548420,"checksums":{"sha256":"18ddad07c623baa1c33552ad6261423bc4e0dc689f8399cda6224e53fe705c67"},"arch":"aarch64","size":571752165,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Vagrant-libvirt.aarch64-Rawhide-20240501.n.0.vagrant.libvirt.box","type":"vagrant-libvirt"}],"x86_64":[{"subvariant":"Cloud_Base","format":"raw.xz","volume_id":null,"mtime":1714548401,"checksums":{"sha256":"c376b61792828219d46f918f5c9cbcc1966c0ec4fd841b3ac8dc1b590eb87ece"},"arch":"x86_64","size":377126452,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-AmazonEC2.x86_64-Rawhide-20240501.n.0.raw.xz","type":"raw-xz"},{"subvariant":"Cloud_Base","format":"vhd.xz","volume_id":null,"mtime":1714548407,"checksums":{"sha256":"b95bef74af4a21cbedb1c590eada488bcf23bd1ca84b111bd6ba803c8783eff8"},"arch":"x86_64","size":452488716,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Azure.x86_64-Rawhide-20240501.n.0.vhdfixed.xz","type":"vhd-compressed"},{"subvariant":"Cloud_Base","format":"tar.gz","volume_id":null,"mtime":1714548403,"checksums":{"sha256":"6b2b7114f924cd610d54d1166a5ca74250c49fbbe3e83ebf132150a8185f7bf5"},"arch":"x86_64","size":411774493,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-GCE.x86_64-Rawhide-20240501.n.0.tar.gz","type":"docker"},{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548401,"checksums":{"sha256":"b872fe26d3e5a8093f4de7600411dd935b1ea3614542a6dc5a22f3cea4015936"},"arch":"x86_64","size":408944640,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Generic.x86_64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base_UKI","format":"qcow2","volume_id":null,"mtime":1714548404,"checksums":{"sha256":"f9b82e1b9e226df36117143c55dd534a006aaf90c3ca158037c211ef4535db81"},"arch":"x86_64","size":439222272,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-UEFI-UKI.x86_64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714548418,"checksums":{"sha256":"89a2cbab39f0750125b51be6e04da067aa0484598a86e558cbeb6cab074cd311"},"arch":"x86_64","size":571837191,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Vagrant-VirtualBox.x86_64-Rawhide-20240501.n.0.vagrant.virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Cloud_Base","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548418,"checksums":{"sha256":"f96fee2a6ac8e0d63400eb7dfe1a1c3100041a6a61c7be378b4ae0dcc223343b"},"arch":"x86_64","size":581492129,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Vagrant-libvirt.x86_64-Rawhide-20240501.n.0.vagrant.libvirt.box","type":"vagrant-libvirt"}],"s390x":[{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548400,"checksums":{"sha256":"69a9e590a7fafdf5bcc6ab7b00943e453930f34136571aff0b77a028346f36fa"},"arch":"s390x","size":374772736,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/s390x/images/Fedora-Cloud-Base-Generic.s390x-Rawhide-20240501.n.0.qcow2","type":"qcow2"}],"ppc64le":[{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548579,"checksums":{"sha256":"60b69830dde0dd01d250277e61f2f779a78636274157f76ed4922f91e0c66b04"},"arch":"ppc64le","size":405536768,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/ppc64le/images/Fedora-Cloud-Base-Generic.ppc64le-Rawhide-20240501.n.0.qcow2","type":"qcow2"}]},"Kinoite":{"x86_64":[{"subvariant":"Kinoite","format":"iso","volume_id":"Fedora-Knt-ostree-x86_64-rawh","mtime":1714547170,"checksums":{"sha256":"b9f83ad46bd54203e71d7694ed2a93c926ef90a6eadfb4e54fdcc878bd3b5c55"},"arch":"x86_64","size":4265914368,"disc_count":1,"bootable":true,"implant_md5":"0cdc1ad51e553cd561b707fc7649880b","disc_number":1,"path":"Kinoite/x86_64/iso/Fedora-Kinoite-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"ppc64le":[{"subvariant":"Kinoite","format":"iso","volume_id":"Fedora-Knt-ostree-ppc64le-rawh","mtime":1714547314,"checksums":{"sha256":"eb53b4a4803f3f07b70440e6e418a3d99fad77554a8d24e637f593d7a4111c8b"},"arch":"ppc64le","size":3977838592,"disc_count":1,"bootable":true,"implant_md5":"a4a70257ed61704a79ba87fbd4e858bf","disc_number":1,"path":"Kinoite/ppc64le/iso/Fedora-Kinoite-ostree-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Spins":{"aarch64":[{"subvariant":"KDE","format":"raw.xz","volume_id":null,"mtime":1714551236,"checksums":{"sha256":"1f59bcccda3ce19825729af0f5d2e1728312757e85d8eac0790b828723b3edbb"},"arch":"aarch64","size":3323626052,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-KDE-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"LXQt","format":"raw.xz","volume_id":null,"mtime":1714550923,"checksums":{"sha256":"b85c09dfce672d5844edeaac41f45d7595bf971be0ff5ff2733fc816594a731e"},"arch":"aarch64","size":2160802488,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-LXQt-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Minimal","format":"raw.xz","volume_id":null,"mtime":1714548583,"checksums":{"sha256":"722e1717d73bf43e2eb6e0cb4fb8ae3cb19b4a2de8cf1c49da4d6020597d3b66"},"arch":"aarch64","size":933003100,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-Minimal-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"SoaS","format":"raw.xz","volume_id":null,"mtime":1714549638,"checksums":{"sha256":"242229d68cf1af9ce239192ad87965f216c118c75d9fd74e72b08ab0f00e24ed"},"arch":"aarch64","size":1885278248,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-SoaS-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Xfce","format":"raw.xz","volume_id":null,"mtime":1714549789,"checksums":{"sha256":"c707ac0edeffe9f1fc3fef644bb49c421f94f01f2f385b46a07533c18932895d"},"arch":"aarch64","size":2286809604,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-Xfce-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"KDE","format":"iso","volume_id":null,"mtime":1714549170,"checksums":{"sha256":"64eb2f3cd6e54b724ccd3528867d49a4057789ed8a00e5f01d2ba1f37a24bc2c"},"arch":"aarch64","size":2649317376,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/iso/Fedora-KDE-Live-aarch64-Rawhide-20240501.n.0.iso","type":"live"}],"x86_64":[{"subvariant":"Budgie","format":"iso","volume_id":null,"mtime":1714549119,"checksums":{"sha256":"7170cec0da8874d774b611afa4f398684d050407cd476672c50897f8c23a271b"},"arch":"x86_64","size":2154031104,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Budgie-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Cinnamon","format":"iso","volume_id":null,"mtime":1714549323,"checksums":{"sha256":"6ca934500ad73394e30cb6394eac7f01cf8f9b034a7338f2ea259f3a72287153"},"arch":"x86_64","size":2566842368,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Cinnamon-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"KDE","format":"iso","volume_id":null,"mtime":1714549325,"checksums":{"sha256":"8b10a757116d53ede4725a6e08766af3b3fa5c0c953c24021f6c07616fda8485"},"arch":"x86_64","size":2673795072,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"LXDE","format":"iso","volume_id":null,"mtime":1714549033,"checksums":{"sha256":"1a082a163d6a4a083f7a14752bf05444810759e09d7c032449c1ec39db03d670"},"arch":"x86_64","size":1755189248,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-LXDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"LXQt","format":"iso","volume_id":null,"mtime":1714549069,"checksums":{"sha256":"b206c9622050720e8dab00ff071e8ab3c4a5aeba04a810da933969c02a7f0785"},"arch":"x86_64","size":1881649152,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-LXQt-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Mate","format":"iso","volume_id":null,"mtime":1714549314,"checksums":{"sha256":"218b1e1e8efb78b34a9706b0d995f0f6d2450086635cf07477cd4330f8c8c24e"},"arch":"x86_64","size":2452094976,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-MATE_Compiz-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"SoaS","format":"iso","volume_id":null,"mtime":1714548992,"checksums":{"sha256":"ddb3d9ad6c2169f79c1ffa6cad759199d79c2d51e3012eb7ea18599ab0ec3864"},"arch":"x86_64","size":1459724288,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-SoaS-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Sway","format":"iso","volume_id":null,"mtime":1714549007,"checksums":{"sha256":"2fb50be1ed0b5d12a648d1b113b9c2bdb2debece729eee65d219b94b2d2f21d3"},"arch":"x86_64","size":1651877888,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Sway-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Xfce","format":"iso","volume_id":null,"mtime":1714549105,"checksums":{"sha256":"0d845b914b0f5e83ca2ce845652d25da556537db31e090b16167e34aca314094"},"arch":"x86_64","size":1903425536,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Xfce-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"i3","format":"iso","volume_id":null,"mtime":1714549001,"checksums":{"sha256":"cae0b4113cc260962b573315cbf0ce01df7d14f4d6d7794a0e700a0e30d8f0ac"},"arch":"x86_64","size":1637480448,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-i3-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}]},"Sericea":{"x86_64":[{"subvariant":"Sericea","format":"iso","volume_id":"Fedora-Src-ostree-x86_64-rawh","mtime":1714546202,"checksums":{"sha256":"4e33ca3626e68a99d87e2da6552536bb1da53f4a885f265f3e41b2026ff13a9c"},"arch":"x86_64","size":2600185856,"disc_count":1,"bootable":true,"implant_md5":"7edb7a3c6255c9485d706bfaaf10e410","disc_number":1,"path":"Sericea/x86_64/iso/Fedora-Sericea-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]}},"compose":{"date":"20240501","respin":0,"type":"nightly","id":"Fedora-Rawhide-20240501.n.0"}}}'
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:50 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ Transfer-Encoding:`
		`+ - chunked`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy04.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHUrAMQ1iYCtlRmxpLEAAAA9M`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ allow:`
		`+ - GET, HEAD, OPTIONS`
		`+ apptime:`
		`+ - D=1025542`
		`+ cache-control:`
		`+ - private, max-age=0, must-revalidate`
		`+ content-type:`
		`+ - application/json`
		`+ set-cookie:`
		`+ - SERVERID=pdc-web01; path=/`
		`+ vary:`
		`+ - Accept,Cookie,Accept-Encoding`
		`+ x-fedora-appserver:`
		`+ - pdc-web01.iad2.fedoraproject.org`
		`+ x-frame-options:`
		`+ - SAMEORIGIN`
		`+ - SAMEORIGIN`
		`+ status:`
		`+ code: 200`
		`+ message: OK`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Accept:`
		`+ - '/'`
		`+ Accept-Encoding:`
		`+ - gzip, deflate, br`
		`+ Connection:`
		`+ - keep-alive`
		`+ User-Agent:`
		`+ - python-requests/2.31.0`
		`+ method: GET`
		`+ uri: https://bodhi.fedoraproject.org/releases/F41`
		`+ response:`
		`+ body:`
		`+ string: '{"name": "F41", "long_name": "Fedora 41", "version": "41", "id_prefix":`
		`+ "FEDORA", "branch": "rawhide", "dist_tag": "f41", "stable_tag": "f41", "testing_tag":`
		`+ "f41-updates-testing", "candidate_tag": "f41-updates-candidate", "pending_signing_tag":`
		`+ "f41-signing-pending", "pending_testing_tag": "f41-updates-testing-pending",`
		`+ "pending_stable_tag": "f41-updates-pending", "override_tag": "f41-override",`
		`+ "mail_template": "fedora_errata_template", "state": "pending", "composed_by_bodhi":`
		`+ false, "create_automatic_updates": true, "package_manager": "unspecified",`
		`+ "testing_repository": null, "released_on": null, "eol": null, "setting_status":`
		`+ "pre_beta"}'`
		`+ headers:`
		`+ AppTime:`
		`+ - D=44636`
		`+ Connection:`
		`+ - Keep-Alive`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:24:51 GMT`
		`+ Keep-Alive:`
		`+ - timeout=15, max=500`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - gunicorn`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy14.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHU8_z7IlprDyBbLxCtQAABoU`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '650'`
		`+ content-type:`
		`+ - application/json`
		`+ set-cookie:`
		`+ - 1caa5c4232b1a1f24f8c4f6e0f496284=54379c9e3297d922e85a653bc349fdb4; path=/;`
		`+ HttpOnly; Secure; SameSite=None`
		`+ x-content-type-options:`
		`+ - nosniff`
		`+ - nosniff`
		`+ status:`
		`+ code: 200`
		`+ message: OK`
		`+ version: 1`

tests/fixtures/cassettes/test_containers[compose1].yaml

file added

+1450

The added file is too large to be shown here, see it at: tests/fixtures/cassettes/test_containers[compose1].yaml

tests/fixtures/cassettes/test_containers_registries_not_configured.yaml

file added

+683

		`@@ -0,0 +1,683 @@`
		`+ interactions:`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2376`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:01 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXU0FjxMk9tie55ZJIwAACwM`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2643`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:01 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXWmti5Q4-osOpGW0QgAAA8s`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=3161`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:02 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXo16DgLUPyhgv1bd5QAAAQ0`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=3629`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:02 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXgube8DRnI9SdhUm2QAABgU`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2573`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:02 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXgube8DRnI9SdhUm7QAABhg`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=5483`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:02 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXvYPQ6beqzZJoe4JqAAACsc`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/COMPOSE_ID`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=6606`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:03 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy01.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHXzuWhzpWHAjyjkso3gAABJE`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=3372`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:03 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHX2czgTxQbJmZVvtIWQAABAA`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Host:`
		`+ - kojipkgs.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20240501.n.0/compose`
		`+ response:`
		`+ body:`
		`+ string: '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">`
		`+`
		`+ <html><head>`
		`+`
		`+ <title>404 Not Found</title>`
		`+`
		`+ </head><body>`
		`+`
		`+ <h1>Not Found</h1>`
		`+`
		`+ <p>The requested URL was not found on this server.</p>`
		`+`
		`+ </body></html>`
		`+`
		`+ '`
		`+ headers:`
		`+ AppTime:`
		`+ - D=2843`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:03 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy10.iad2.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHX3NjG4RPecDt0yhy9AAAB8M`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '196'`
		`+ content-type:`
		`+ - text/html; charset=iso-8859-1`
		`+ strict-transport-security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ status:`
		`+ code: 404`
		`+ message: Not Found`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Content-Type:`
		`+ - application/json`
		`+ Host:`
		`+ - pdc.fedoraproject.org`
		`+ User-Agent:`
		`+ - Python-urllib/3.12`
		`+ method: GET`
		`+ uri: https://pdc.fedoraproject.org/rest_api/v1/compose-images/Fedora-Rawhide-20240501.n.0/?page=1`
		`+ response:`
		`+ body:`
		+ string: '{"header":{"version":"1.2","type":"productmd.images"},"payload":{"images":{"Workstation":{"aarch64":[{"subvariant":"Workstation","format":"raw.xz","volume_id":null,"mtime":1714551577,"checksums":{"sha256":"169a31fd5cf10faafaba87b2342ad6475bc1d20ce3e71946d0fa2694bd4484fe"},"arch":"aarch64","size":2806539876,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/aarch64/images/Fedora-Workstation-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714550825,"checksums":{"sha256":"2e6757ccad552f5929f1a69777f3a8166985953b0331ab6386ab6af8ca0e8322"},"arch":"aarch64","size":2609154048,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/aarch64/iso/Fedora-Workstation-Live-osb-Rawhide-20240501.n.0.aarch64.iso","type":"live-osbuild"}],"x86_64":[{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714551289,"checksums":{"sha256":"81584c47d50304bf1a659c17af7b761891e8f70545eb97e1ae0cc7ff511f79ee"},"arch":"x86_64","size":2654552064,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Workstation/x86_64/iso/Fedora-Workstation-Live-osb-Rawhide-20240501.n.0.x86_64.iso","type":"live-osbuild"},{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714549615,"checksums":{"sha256":"264d04c31714ba0734940819b8bdc7863701f9cd7a16e553b5b6a5db121effa7"},"arch":"x86_64","size":2314934272,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}],"ppc64le":[{"subvariant":"Workstation","format":"iso","volume_id":null,"mtime":1714549972,"checksums":{"sha256":"1f19f95713627cfbb487cb32ccaf0dcaeb49717e23649c6244ace0e71f6932fe"},"arch":"ppc64le","size":2265411584,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Workstation/ppc64le/iso/Fedora-Workstation-Live-ppc64le-Rawhide-20240501.n.0.iso","type":"live"}]},"Container":{"aarch64":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548151,"checksums":{"sha256":"570b6e8f5e642df8541add9734ce6263396ac8b31410d334affd4f241161bb0e"},"arch":"aarch64","size":45765840,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548144,"checksums":{"sha256":"4410600bf5c55c2ed2d892f448d0f940f1d04bd3758df45c1214e666f909376a"},"arch":"aarch64","size":80061492,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Base-Generic.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548390,"checksums":{"sha256":"14662b170bb2a792ef59471c4f3832aec24a156a63723ae7f3189ae39055198c"},"arch":"aarch64","size":309801336,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/aarch64/images/Fedora-Container-Toolbox.aarch64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"x86_64":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548002,"checksums":{"sha256":"99762e812b170a2b5ae21ffdfcc26d6f821064c3347c3456bcfb0946b51d3e39"},"arch":"x86_64","size":47325456,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548128,"checksums":{"sha256":"22ec94af77d239c4be0d6441b57b1a36e7f5ab78da4ebeb2fa0ebc5e2d84ab99"},"arch":"x86_64","size":81582552,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Base-Generic.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548391,"checksums":{"sha256":"4338e4bf47b0f98cde51fcd90f4c8dd0ec6e44e19f066ac71a3a8f0b156bd613"},"arch":"x86_64","size":333172684,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/x86_64/images/Fedora-Container-Toolbox.x86_64-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"s390x":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714547999,"checksums":{"sha256":"5126ea913a0cce891f146c98d64f7041f88ec97fdf833050b66bcb1761963e7e"},"arch":"s390x","size":47592832,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548085,"checksums":{"sha256":"b872fec000a3c09d0b0875d14ab11df3ae8fac9841c9ce2d75479d87b99b77bb"},"arch":"s390x","size":82633556,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Base-Generic.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548383,"checksums":{"sha256":"254e6199117fd50a0a402a8e0bcf0d1a497457712525e05e67bde46c6b43a6ee"},"arch":"s390x","size":295170680,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/s390x/images/Fedora-Container-Toolbox.s390x-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}],"ppc64le":[{"subvariant":"Container_Minimal_Base","format":"tar.xz","volume_id":null,"mtime":1714548195,"checksums":{"sha256":"a1609b38c5ca8b5725a5b861e6d223ebd7efb540a5a8dcb0d124c8143edacc15"},"arch":"ppc64le","size":53859988,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Base","format":"tar.xz","volume_id":null,"mtime":1714548207,"checksums":{"sha256":"16232ae6ac7d85480a12307b418ea86c62097889369f241607a6da3fdc810294"},"arch":"ppc64le","size":89383320,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Base-Generic.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"},{"subvariant":"Container_Toolbox","format":"tar.xz","volume_id":null,"mtime":1714548405,"checksums":{"sha256":"f24b0e9d19a19e509bef289c02ce0ce017b8abaa3d94dd3e160756cfbfe9a1e8"},"arch":"ppc64le","size":317277716,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Container/ppc64le/images/Fedora-Container-Toolbox.ppc64le-Rawhide-20240501.n.0.oci.tar.xz","type":"docker"}]},"Everything":{"aarch64":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-aarch64-rawh","mtime":1714543271,"checksums":{"sha256":"c8761f0c0c969b2208bc1eec38608a3d421c74168e11bf6842ce0649c0b6e2c1"},"arch":"aarch64","size":881444864,"disc_count":1,"bootable":true,"implant_md5":"eb260a2607dea1ea7b4c70a3bc3b3309","disc_number":1,"path":"Everything/aarch64/iso/Fedora-Everything-netinst-aarch64-Rawhide-20240501.n.0.iso","type":"boot"}],"x86_64":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-x86_64-rawh","mtime":1714543598,"checksums":{"sha256":"6a4c569813b8fa3269122d4de538302d212be395f2465f192c3b42c3bd29c4d6"},"arch":"x86_64","size":862216192,"disc_count":1,"bootable":true,"implant_md5":"4fada428441a95574831d3cb8b254e44","disc_number":1,"path":"Everything/x86_64/iso/Fedora-Everything-netinst-x86_64-Rawhide-20240501.n.0.iso","type":"boot"}],"s390x":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-s390x-rawh","mtime":1714543372,"checksums":{"sha256":"1a1d0489e884cee0f5611adf10dcdc2cc8cecd8a43ca72e9133835cd0c993726"},"arch":"s390x","size":538773504,"disc_count":1,"bootable":true,"implant_md5":"d75c8272c5b65a38083becafe0114e2c","disc_number":1,"path":"Everything/s390x/iso/Fedora-Everything-netinst-s390x-Rawhide-20240501.n.0.iso","type":"boot"}],"ppc64le":[{"subvariant":"Everything","format":"iso","volume_id":"Fedora-E-dvd-ppc64le-rawh","mtime":1714544248,"checksums":{"sha256":"06e517e99fc1ad551afc5796ba574f96940c93321ec8e1af0597c44fceef1829"},"arch":"ppc64le","size":868366336,"disc_count":1,"bootable":true,"implant_md5":"447733a53e635d41f0221cd038799cea","disc_number":1,"path":"Everything/ppc64le/iso/Fedora-Everything-netinst-ppc64le-Rawhide-20240501.n.0.iso","type":"boot"}]},"Onyx":{"x86_64":[{"subvariant":"Onyx","format":"iso","volume_id":"Fedora-Onyx-ostree-x86_64-rawh","mtime":1714546339,"checksums":{"sha256":"c7fc13f5fbd63ede8dcee60880ec9353e192d27e1298d70553987667472d468e"},"arch":"x86_64","size":2758076416,"disc_count":1,"bootable":true,"implant_md5":"eea8885d7c0c04c811dbb7fadb88c18b","disc_number":1,"path":"Onyx/x86_64/iso/Fedora-Onyx-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Server":{"aarch64":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548481,"checksums":{"sha256":"7df0a82f10cf9ff246a4367c9d2738dcfa38adeab43f6259fd59c248334e754d"},"arch":"aarch64","size":679477248,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/aarch64/images/Fedora-Server-KVM-Rawhide-20240501.n.0.aarch64.qcow2","type":"qcow2"},{"subvariant":"Server","format":"raw.xz","volume_id":null,"mtime":1714549351,"checksums":{"sha256":"606840743d5f6949f6a24a087a83ee30ba75061efccae97dc10b0a9911eb647f"},"arch":"aarch64","size":1156440656,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/aarch64/images/Fedora-Server-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-aarch64-rawh","mtime":1714547827,"checksums":{"sha256":"9254a157dd98c83ec69bd6bfa32c332132a77a244196d986e61a8e07dcef482b"},"arch":"aarch64","size":2571698176,"disc_count":1,"bootable":true,"implant_md5":"3b4bf7e119d816a9b7d7ff43bb1e7397","disc_number":1,"path":"Server/aarch64/iso/Fedora-Server-dvd-aarch64-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-aarch64-rawh","mtime":1714543282,"checksums":{"sha256":"f97a38209469f4aabebf80734d97f672d0e7a76599178e627f551be0efca705d"},"arch":"aarch64","size":891131904,"disc_count":1,"bootable":true,"implant_md5":"56a7d1aa0db7f8885ce2bb582431c20a","disc_number":1,"path":"Server/aarch64/iso/Fedora-Server-netinst-aarch64-Rawhide-20240501.n.0.iso","type":"boot"}],"x86_64":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548432,"checksums":{"sha256":"874dca83ba136eda1395d5b4195252b80c9c46586b5d0959cab8a2228fa87981"},"arch":"x86_64","size":663355392,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/x86_64/images/Fedora-Server-KVM-Rawhide-20240501.n.0.x86_64.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-x86_64-rawh","mtime":1714547835,"checksums":{"sha256":"28f2da7d8092d8d9fdf9b2e55a6c01cb8df4d91040698b4e5eeaefb59bc0562e"},"arch":"x86_64","size":2659516416,"disc_count":1,"bootable":true,"implant_md5":"09ab21ecd902b709225a183bdb4d221f","disc_number":1,"path":"Server/x86_64/iso/Fedora-Server-dvd-x86_64-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-x86_64-rawh","mtime":1714544390,"checksums":{"sha256":"284d59258c0097df13b6e534e7286cc0aef3ff97355867c958b50ad1fbcefbd2"},"arch":"x86_64","size":872001536,"disc_count":1,"bootable":true,"implant_md5":"415e2b42be4a7ab863b531a9f103609b","disc_number":1,"path":"Server/x86_64/iso/Fedora-Server-netinst-x86_64-Rawhide-20240501.n.0.iso","type":"boot"}],"s390x":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714548425,"checksums":{"sha256":"fb3c65a91db222dd53642ddfc8bc79f93d6b368daba2de08fa29995c56b51f25"},"arch":"s390x","size":642777088,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/s390x/images/Fedora-Server-KVM-Rawhide-20240501.n.0.s390x.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-s390x-rawh","mtime":1714548033,"checksums":{"sha256":"1b01c52808aca1e6612318816d9f23d28731433213b9dca0f5001ac176e571f6"},"arch":"s390x","size":2002780160,"disc_count":1,"bootable":true,"implant_md5":"4c9027c3bdf2355b1bd44d2e1510e20b","disc_number":1,"path":"Server/s390x/iso/Fedora-Server-dvd-s390x-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-s390x-rawh","mtime":1714543378,"checksums":{"sha256":"bd5408c0fef7d15cb9ecefd6890473cfea0c8eb2c3ac87eaeb03469d7b8bc05a"},"arch":"s390x","size":549619712,"disc_count":1,"bootable":true,"implant_md5":"239d338c78263b5528a0ef1a2da78540","disc_number":1,"path":"Server/s390x/iso/Fedora-Server-netinst-s390x-Rawhide-20240501.n.0.iso","type":"boot"}],"ppc64le":[{"subvariant":"Server_KVM","format":"qcow2","volume_id":null,"mtime":1714555324,"checksums":{"sha256":"8697bb87795aeb0cfdbf67683c72672e5390c0c26d8c456147b3c237a35c6467"},"arch":"ppc64le","size":684392448,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Server/ppc64le/images/Fedora-Server-KVM-Rawhide-20240501.n.0.ppc64le.qcow2","type":"qcow2"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-ppc64le-rawh","mtime":1714548000,"checksums":{"sha256":"1eb107a7627f035ab3fe6f21db00b2b5d6766af991453287db444edd5532b625"},"arch":"ppc64le","size":2409299968,"disc_count":1,"bootable":true,"implant_md5":"52aa0d9a2a7e40ed64c6cc301020308e","disc_number":1,"path":"Server/ppc64le/iso/Fedora-Server-dvd-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd"},{"subvariant":"Server","format":"iso","volume_id":"Fedora-S-dvd-ppc64le-rawh","mtime":1714543594,"checksums":{"sha256":"f74d20418c881571be50a2184f240d13b8cfdf7bbad72bfc2571bb819674390a"},"arch":"ppc64le","size":879071232,"disc_count":1,"bootable":true,"implant_md5":"5e55736ea6f1dc86ce1c22b93e8fa0b3","disc_number":1,"path":"Server/ppc64le/iso/Fedora-Server-netinst-ppc64le-Rawhide-20240501.n.0.iso","type":"boot"}]},"Labs":{"aarch64":[{"subvariant":"Python_Classroom","format":"raw.xz","volume_id":null,"mtime":1714549628,"checksums":{"sha256":"53517e834f444d1bbfdb95506d3c97d6563736c63d23fcb7cb0485c8e8555345"},"arch":"aarch64","size":2717602640,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/aarch64/images/Fedora-Python-Classroom-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"}],"x86_64":[{"subvariant":"Python_Classroom","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548988,"checksums":{"sha256":"85eb263a920688d56d5e74958161ced4044578d7093b0018d09b78245712c63a"},"arch":"x86_64","size":1567637359,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Python-Classroom-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-libvirt.box","type":"vagrant-libvirt"},{"subvariant":"Python_Classroom","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714549060,"checksums":{"sha256":"81685cb0637678d5111e8b50dc61e94df119a2c2bb3b2ec216d04d35c5356527"},"arch":"x86_64","size":1589186560,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Python-Classroom-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Scientific","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714549937,"checksums":{"sha256":"6a9b6f17eb655962a8b3e343f09ed06cb5fca92ad3faef6a01215ae673a62bad"},"arch":"x86_64","size":4906517741,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Scientific-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-libvirt.box","type":"vagrant-libvirt"},{"subvariant":"Scientific","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714550139,"checksums":{"sha256":"29547a3dc363baf76c26c53350a066d76b4287e644019d5f3e43c16e8aad196c"},"arch":"x86_64","size":4963287040,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/images/Fedora-Scientific-Vagrant-Rawhide-20240501.n.0.x86_64.vagrant-virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Astronomy_KDE","format":"iso","volume_id":null,"mtime":1714549753,"checksums":{"sha256":"eedb523885c20bfb5b246563def3e4a593d7698d7847923cf5292a2f726ab772"},"arch":"x86_64","size":4663750656,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Astronomy_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Comp_Neuro","format":"iso","volume_id":null,"mtime":1714549174,"checksums":{"sha256":"78a2911eb3c6fe4f86bbdcc93930eb8b2172f8b4971e5f83324bc496c1d9ad3f"},"arch":"x86_64","size":3089092608,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Comp_Neuro-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Games","format":"iso","volume_id":null,"mtime":1714549710,"checksums":{"sha256":"bd14181af753ff6d6273d0cc6575b2b13ee601564f526ab43c8060e9a44a8833"},"arch":"x86_64","size":6911944704,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Games-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Jam_KDE","format":"iso","volume_id":null,"mtime":1714549112,"checksums":{"sha256":"d612fc08962b47f04a6cc7549f45d7deb8740c0cf7838bd48423d4147aa2803f"},"arch":"x86_64","size":3543447552,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Jam_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Python_Classroom","format":"iso","volume_id":null,"mtime":1714549398,"checksums":{"sha256":"5f0fd5c2f81e6838409adfd70f71f532a73435505fd939f6f1c78c9ba57795bd"},"arch":"x86_64","size":2366535680,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Python-Classroom-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Robotics","format":"iso","volume_id":null,"mtime":1714549620,"checksums":{"sha256":"a91c562e1e2878977ec7639e7fe6056acc649822456fd4d50f9184dec9ee6376"},"arch":"x86_64","size":3167330304,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Robotics-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Scientific_KDE","format":"iso","volume_id":null,"mtime":1714549887,"checksums":{"sha256":"cdb127b1b26e6b1b4541be85c890bc6a20f36c58e596d77042d6b99d61d40c55"},"arch":"x86_64","size":5520687104,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Scientific_KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Security","format":"iso","volume_id":null,"mtime":1714549047,"checksums":{"sha256":"ba32f7df92892f6185e46349e825c995ba81c5a26b482e46554079f22b4da894"},"arch":"x86_64","size":2481698816,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Labs/x86_64/iso/Fedora-Security-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}]},"Silverblue":{"aarch64":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-aarch64-rawh","mtime":1714546273,"checksums":{"sha256":"b5a25b696cc0fdea442671eebcbb999287378e87542cfdb4b68b52dd2bd0ef4b"},"arch":"aarch64","size":3620956160,"disc_count":1,"bootable":true,"implant_md5":"461ca41e418493e1475d5bcd5fc1546f","disc_number":1,"path":"Silverblue/aarch64/iso/Fedora-Silverblue-ostree-aarch64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"x86_64":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-x86_64-rawh","mtime":1714546928,"checksums":{"sha256":"c138b73ec6e460d2ef300d04052e5f851e22d97bc00b96663a0b19daf37da973"},"arch":"x86_64","size":3640899584,"disc_count":1,"bootable":true,"implant_md5":"ac049c322f096d2dbdd55b7369048584","disc_number":1,"path":"Silverblue/x86_64/iso/Fedora-Silverblue-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"ppc64le":[{"subvariant":"Silverblue","format":"iso","volume_id":"Fedora-SB-ostree-ppc64le-rawh","mtime":1714547457,"checksums":{"sha256":"02951538e73b9a53657e667a4fe745ba31ad70c9a07a445ba299369c487f3047"},"arch":"ppc64le","size":3572103168,"disc_count":1,"bootable":true,"implant_md5":"8a9f0707386f692fc93ae051f97487fb","disc_number":1,"path":"Silverblue/ppc64le/iso/Fedora-Silverblue-ostree-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Cloud":{"aarch64":[{"subvariant":"Cloud_Base","format":"raw.xz","volume_id":null,"mtime":1714548549,"checksums":{"sha256":"761269846a3fb0750fdf3853cc7838189ee1317c376e45de4225a6364ce51907"},"arch":"aarch64","size":372905420,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-AmazonEC2.aarch64-Rawhide-20240501.n.0.raw.xz","type":"raw-xz"},{"subvariant":"Cloud_Base","format":"vhd.xz","volume_id":null,"mtime":1714548597,"checksums":{"sha256":"aa977ff3c52903c0000338914b4c0691f8d857675742ac0bda12ed8af6b6fd71"},"arch":"aarch64","size":438226428,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Azure.aarch64-Rawhide-20240501.n.0.vhdfixed.xz","type":"vhd-compressed"},{"subvariant":"Cloud_Base","format":"tar.gz","volume_id":null,"mtime":1714548410,"checksums":{"sha256":"acdb1bd9065c6a648f7f45ed68ed001a333f9d1fee96768a758cf56885e7191f"},"arch":"aarch64","size":415969669,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-GCE.aarch64-Rawhide-20240501.n.0.tar.gz","type":"docker"},{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548405,"checksums":{"sha256":"ceaee75dd6a3a6c4244a30eb59184a935bbc0d004dce13f29f22a62631819b4e"},"arch":"aarch64","size":415891456,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Generic.aarch64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base_UKI","format":"qcow2","volume_id":null,"mtime":1714548578,"checksums":{"sha256":"2487abdf4e5ae7a9be4439833ae769ea946bb7f14632236f65b91913001ee1d7"},"arch":"aarch64","size":430243840,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-UEFI-UKI.aarch64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548420,"checksums":{"sha256":"18ddad07c623baa1c33552ad6261423bc4e0dc689f8399cda6224e53fe705c67"},"arch":"aarch64","size":571752165,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/aarch64/images/Fedora-Cloud-Base-Vagrant-libvirt.aarch64-Rawhide-20240501.n.0.vagrant.libvirt.box","type":"vagrant-libvirt"}],"x86_64":[{"subvariant":"Cloud_Base","format":"raw.xz","volume_id":null,"mtime":1714548401,"checksums":{"sha256":"c376b61792828219d46f918f5c9cbcc1966c0ec4fd841b3ac8dc1b590eb87ece"},"arch":"x86_64","size":377126452,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-AmazonEC2.x86_64-Rawhide-20240501.n.0.raw.xz","type":"raw-xz"},{"subvariant":"Cloud_Base","format":"vhd.xz","volume_id":null,"mtime":1714548407,"checksums":{"sha256":"b95bef74af4a21cbedb1c590eada488bcf23bd1ca84b111bd6ba803c8783eff8"},"arch":"x86_64","size":452488716,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Azure.x86_64-Rawhide-20240501.n.0.vhdfixed.xz","type":"vhd-compressed"},{"subvariant":"Cloud_Base","format":"tar.gz","volume_id":null,"mtime":1714548403,"checksums":{"sha256":"6b2b7114f924cd610d54d1166a5ca74250c49fbbe3e83ebf132150a8185f7bf5"},"arch":"x86_64","size":411774493,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-GCE.x86_64-Rawhide-20240501.n.0.tar.gz","type":"docker"},{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548401,"checksums":{"sha256":"b872fe26d3e5a8093f4de7600411dd935b1ea3614542a6dc5a22f3cea4015936"},"arch":"x86_64","size":408944640,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Generic.x86_64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base_UKI","format":"qcow2","volume_id":null,"mtime":1714548404,"checksums":{"sha256":"f9b82e1b9e226df36117143c55dd534a006aaf90c3ca158037c211ef4535db81"},"arch":"x86_64","size":439222272,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-UEFI-UKI.x86_64-Rawhide-20240501.n.0.qcow2","type":"qcow2"},{"subvariant":"Cloud_Base","format":"vagrant-virtualbox.box","volume_id":null,"mtime":1714548418,"checksums":{"sha256":"89a2cbab39f0750125b51be6e04da067aa0484598a86e558cbeb6cab074cd311"},"arch":"x86_64","size":571837191,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Vagrant-VirtualBox.x86_64-Rawhide-20240501.n.0.vagrant.virtualbox.box","type":"vagrant-virtualbox"},{"subvariant":"Cloud_Base","format":"vagrant-libvirt.box","volume_id":null,"mtime":1714548418,"checksums":{"sha256":"f96fee2a6ac8e0d63400eb7dfe1a1c3100041a6a61c7be378b4ae0dcc223343b"},"arch":"x86_64","size":581492129,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/x86_64/images/Fedora-Cloud-Base-Vagrant-libvirt.x86_64-Rawhide-20240501.n.0.vagrant.libvirt.box","type":"vagrant-libvirt"}],"s390x":[{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548400,"checksums":{"sha256":"69a9e590a7fafdf5bcc6ab7b00943e453930f34136571aff0b77a028346f36fa"},"arch":"s390x","size":374772736,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/s390x/images/Fedora-Cloud-Base-Generic.s390x-Rawhide-20240501.n.0.qcow2","type":"qcow2"}],"ppc64le":[{"subvariant":"Cloud_Base","format":"qcow2","volume_id":null,"mtime":1714548579,"checksums":{"sha256":"60b69830dde0dd01d250277e61f2f779a78636274157f76ed4922f91e0c66b04"},"arch":"ppc64le","size":405536768,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Cloud/ppc64le/images/Fedora-Cloud-Base-Generic.ppc64le-Rawhide-20240501.n.0.qcow2","type":"qcow2"}]},"Kinoite":{"x86_64":[{"subvariant":"Kinoite","format":"iso","volume_id":"Fedora-Knt-ostree-x86_64-rawh","mtime":1714547170,"checksums":{"sha256":"b9f83ad46bd54203e71d7694ed2a93c926ef90a6eadfb4e54fdcc878bd3b5c55"},"arch":"x86_64","size":4265914368,"disc_count":1,"bootable":true,"implant_md5":"0cdc1ad51e553cd561b707fc7649880b","disc_number":1,"path":"Kinoite/x86_64/iso/Fedora-Kinoite-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}],"ppc64le":[{"subvariant":"Kinoite","format":"iso","volume_id":"Fedora-Knt-ostree-ppc64le-rawh","mtime":1714547314,"checksums":{"sha256":"eb53b4a4803f3f07b70440e6e418a3d99fad77554a8d24e637f593d7a4111c8b"},"arch":"ppc64le","size":3977838592,"disc_count":1,"bootable":true,"implant_md5":"a4a70257ed61704a79ba87fbd4e858bf","disc_number":1,"path":"Kinoite/ppc64le/iso/Fedora-Kinoite-ostree-ppc64le-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]},"Spins":{"aarch64":[{"subvariant":"KDE","format":"raw.xz","volume_id":null,"mtime":1714551236,"checksums":{"sha256":"1f59bcccda3ce19825729af0f5d2e1728312757e85d8eac0790b828723b3edbb"},"arch":"aarch64","size":3323626052,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-KDE-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"LXQt","format":"raw.xz","volume_id":null,"mtime":1714550923,"checksums":{"sha256":"b85c09dfce672d5844edeaac41f45d7595bf971be0ff5ff2733fc816594a731e"},"arch":"aarch64","size":2160802488,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-LXQt-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Minimal","format":"raw.xz","volume_id":null,"mtime":1714548583,"checksums":{"sha256":"722e1717d73bf43e2eb6e0cb4fb8ae3cb19b4a2de8cf1c49da4d6020597d3b66"},"arch":"aarch64","size":933003100,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-Minimal-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"SoaS","format":"raw.xz","volume_id":null,"mtime":1714549638,"checksums":{"sha256":"242229d68cf1af9ce239192ad87965f216c118c75d9fd74e72b08ab0f00e24ed"},"arch":"aarch64","size":1885278248,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-SoaS-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"Xfce","format":"raw.xz","volume_id":null,"mtime":1714549789,"checksums":{"sha256":"c707ac0edeffe9f1fc3fef644bb49c421f94f01f2f385b46a07533c18932895d"},"arch":"aarch64","size":2286809604,"disc_count":1,"bootable":false,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/images/Fedora-Xfce-Rawhide-20240501.n.0.aarch64.raw.xz","type":"raw-xz"},{"subvariant":"KDE","format":"iso","volume_id":null,"mtime":1714549170,"checksums":{"sha256":"64eb2f3cd6e54b724ccd3528867d49a4057789ed8a00e5f01d2ba1f37a24bc2c"},"arch":"aarch64","size":2649317376,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/aarch64/iso/Fedora-KDE-Live-aarch64-Rawhide-20240501.n.0.iso","type":"live"}],"x86_64":[{"subvariant":"Budgie","format":"iso","volume_id":null,"mtime":1714549119,"checksums":{"sha256":"7170cec0da8874d774b611afa4f398684d050407cd476672c50897f8c23a271b"},"arch":"x86_64","size":2154031104,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Budgie-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Cinnamon","format":"iso","volume_id":null,"mtime":1714549323,"checksums":{"sha256":"6ca934500ad73394e30cb6394eac7f01cf8f9b034a7338f2ea259f3a72287153"},"arch":"x86_64","size":2566842368,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Cinnamon-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"KDE","format":"iso","volume_id":null,"mtime":1714549325,"checksums":{"sha256":"8b10a757116d53ede4725a6e08766af3b3fa5c0c953c24021f6c07616fda8485"},"arch":"x86_64","size":2673795072,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-KDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"LXDE","format":"iso","volume_id":null,"mtime":1714549033,"checksums":{"sha256":"1a082a163d6a4a083f7a14752bf05444810759e09d7c032449c1ec39db03d670"},"arch":"x86_64","size":1755189248,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-LXDE-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"LXQt","format":"iso","volume_id":null,"mtime":1714549069,"checksums":{"sha256":"b206c9622050720e8dab00ff071e8ab3c4a5aeba04a810da933969c02a7f0785"},"arch":"x86_64","size":1881649152,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-LXQt-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Mate","format":"iso","volume_id":null,"mtime":1714549314,"checksums":{"sha256":"218b1e1e8efb78b34a9706b0d995f0f6d2450086635cf07477cd4330f8c8c24e"},"arch":"x86_64","size":2452094976,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-MATE_Compiz-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"SoaS","format":"iso","volume_id":null,"mtime":1714548992,"checksums":{"sha256":"ddb3d9ad6c2169f79c1ffa6cad759199d79c2d51e3012eb7ea18599ab0ec3864"},"arch":"x86_64","size":1459724288,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-SoaS-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Sway","format":"iso","volume_id":null,"mtime":1714549007,"checksums":{"sha256":"2fb50be1ed0b5d12a648d1b113b9c2bdb2debece729eee65d219b94b2d2f21d3"},"arch":"x86_64","size":1651877888,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Sway-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"Xfce","format":"iso","volume_id":null,"mtime":1714549105,"checksums":{"sha256":"0d845b914b0f5e83ca2ce845652d25da556537db31e090b16167e34aca314094"},"arch":"x86_64","size":1903425536,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-Xfce-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"},{"subvariant":"i3","format":"iso","volume_id":null,"mtime":1714549001,"checksums":{"sha256":"cae0b4113cc260962b573315cbf0ce01df7d14f4d6d7794a0e700a0e30d8f0ac"},"arch":"x86_64","size":1637480448,"disc_count":1,"bootable":true,"implant_md5":null,"disc_number":1,"path":"Spins/x86_64/iso/Fedora-i3-Live-x86_64-Rawhide-20240501.n.0.iso","type":"live"}]},"Sericea":{"x86_64":[{"subvariant":"Sericea","format":"iso","volume_id":"Fedora-Src-ostree-x86_64-rawh","mtime":1714546202,"checksums":{"sha256":"4e33ca3626e68a99d87e2da6552536bb1da53f4a885f265f3e41b2026ff13a9c"},"arch":"x86_64","size":2600185856,"disc_count":1,"bootable":true,"implant_md5":"7edb7a3c6255c9485d706bfaaf10e410","disc_number":1,"path":"Sericea/x86_64/iso/Fedora-Sericea-ostree-x86_64-Rawhide-20240501.n.0.iso","type":"dvd-ostree"}]}},"compose":{"date":"20240501","respin":0,"type":"nightly","id":"Fedora-Rawhide-20240501.n.0"}}}'
		`+ headers:`
		`+ Connection:`
		`+ - close`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:04 GMT`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - Apache`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ Transfer-Encoding:`
		`+ - chunked`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy03.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHYN2bvcvk-rfKwhoX9AAAAM8`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ allow:`
		`+ - GET, HEAD, OPTIONS`
		`+ apptime:`
		`+ - D=591670`
		`+ cache-control:`
		`+ - private, max-age=0, must-revalidate`
		`+ content-type:`
		`+ - application/json`
		`+ set-cookie:`
		`+ - SERVERID=pdc-web01; path=/`
		`+ vary:`
		`+ - Accept,Cookie,Accept-Encoding`
		`+ x-fedora-appserver:`
		`+ - pdc-web01.iad2.fedoraproject.org`
		`+ x-frame-options:`
		`+ - SAMEORIGIN`
		`+ - SAMEORIGIN`
		`+ status:`
		`+ code: 200`
		`+ message: OK`
		`+ - request:`
		`+ body: null`
		`+ headers:`
		`+ Accept:`
		`+ - '/'`
		`+ Accept-Encoding:`
		`+ - gzip, deflate, br`
		`+ Connection:`
		`+ - keep-alive`
		`+ User-Agent:`
		`+ - python-requests/2.31.0`
		`+ method: GET`
		`+ uri: https://bodhi.fedoraproject.org/releases/F41`
		`+ response:`
		`+ body:`
		`+ string: '{"name": "F41", "long_name": "Fedora 41", "version": "41", "id_prefix":`
		`+ "FEDORA", "branch": "rawhide", "dist_tag": "f41", "stable_tag": "f41", "testing_tag":`
		`+ "f41-updates-testing", "candidate_tag": "f41-updates-candidate", "pending_signing_tag":`
		`+ "f41-signing-pending", "pending_testing_tag": "f41-updates-testing-pending",`
		`+ "pending_stable_tag": "f41-updates-pending", "override_tag": "f41-override",`
		`+ "mail_template": "fedora_errata_template", "state": "pending", "composed_by_bodhi":`
		`+ false, "create_automatic_updates": true, "package_manager": "unspecified",`
		`+ "testing_repository": null, "released_on": null, "eol": null, "setting_status":`
		`+ "pre_beta"}'`
		`+ headers:`
		`+ AppTime:`
		`+ - D=4678477`
		`+ Connection:`
		`+ - Keep-Alive`
		`+ Date:`
		`+ - Thu, 30 May 2024 00:25:10 GMT`
		`+ Keep-Alive:`
		`+ - timeout=15, max=500`
		`+ Referrer-Policy:`
		`+ - same-origin`
		`+ Server:`
		`+ - gunicorn`
		`+ Strict-Transport-Security:`
		`+ - max-age=31536000; includeSubDomains; preload`
		`+ X-Content-Type-Options:`
		`+ - nosniff`
		`+ - nosniff`
		`+ X-Fedora-ProxyServer:`
		`+ - proxy11.fedoraproject.org`
		`+ X-Fedora-RequestID:`
		`+ - ZlfHYWxxRpEfid_rZWXlmgAAAJg`
		`+ X-Frame-Options:`
		`+ - SAMEORIGIN`
		`+ X-Xss-Protection:`
		`+ - 1; mode=block`
		`+ content-length:`
		`+ - '650'`
		`+ content-type:`
		`+ - application/json`
		`+ set-cookie:`
		`+ - 1caa5c4232b1a1f24f8c4f6e0f496284=cbca343a697424f4648bad1112e1f4a5; path=/;`
		`+ HttpOnly; Secure; SameSite=None`
		`+ x-content-type-options:`
		`+ - nosniff`
		`+ - nosniff`
		`+ status:`
		`+ code: 200`
		`+ message: OK`
		`+ version: 1`

tests/test_handler.py

file modified

+191 -30

		`@@ -19,11 +19,30 @@`
		`from requests.exceptions import RequestException`

		`from fedora_cloud_image_uploader import PLAYBOOKS, Uploader`
		`- from fedora_cloud_image_uploader.handler import CantHandle, NotHandled`
		`+ from fedora_cloud_image_uploader.handler import _run`
		`+`
		`+ # disable fedfind caching, as it can cause things to be left out of`
		`+ # pyvcr cassettes`
		`+ os.environ["FEDFIND_NO_CACHE"] = "1"`
		`+`
		`+`
		`+ def _mock_download_image(self, image: dict, dest_dir: str, decompress=False) -> str:`
		`+ """`
		`+ A mocked-out download_image that behaves somewhat like the real`
		`+ one.`
		`+ """`
		`+ fn = os.path.basename(image["path"])`
		`+ if decompress and fn.endswith(".xz"):`
		`+ fn = fn.removesuffix(".xz")`
		`+ return os.path.join(dest_dir, fn)`


		`- @mock.patch("fedora_cloud_image_uploader.handler.ansible_runner")`
		`@pytest.mark.vcr`
		`+ @mock.patch(`
		`+ "fedora_cloud_image_uploader.handler.Uploader.download_image",`
		`+ lambda a, b, c, decompress: f"/test/{os.path.basename(b['path'].removesuffix('.xz'))}",`
		`+ )`
		`+ @mock.patch("fedora_cloud_image_uploader.handler.ansible_runner")`
		`@pytest.mark.parametrize(`
		`"compose",`
		`[`
		`@@ -51,7 +70,8 @@`
		`}`

		`consumer = Uploader()`
		`- consumer.download_image = mock.Mock()`
		`+ # disable handlers we don't want to hit in this test`
		`+ consumer.handlers = [consumer.handle_azure]`
		`consumer(msg)`

		`assert mock_runner.interface.run.call_count == 2`
		`@@ -63,6 +83,133 @@`


		`@pytest.mark.vcr`
		`+ @mock.patch(`
		`+ "fedora_cloud_image_uploader.handler.Uploader.download_image",`
		`+ lambda a, b, c, decompress: f"/test/{os.path.basename(b['path'].removesuffix('.xz'))}",`
		`+ )`
		`+ @mock.patch("subprocess.run")`
		`+ @pytest.mark.parametrize(`
		`+ "compose",`
		`+ [`
		`+ (`
		`+ "messages/rawhide_compose.json",`
		`+ "Rawhide-20240501.n.0",`
		`+ "41",`
		`+ "rawhide",`
		`+ {`
		`+ "fedora-minimal": ["aarch64", "x86_64", "s390x", "ppc64le"],`
		`+ "fedora": ["aarch64", "x86_64", "s390x", "ppc64le"],`
		`+ "fedora-toolbox": ["aarch64", "x86_64", "s390x", "ppc64le"],`
		`+ },`
		`+ ),`
		`+ (`
		`+ "messages/rc_compose.json",`
		`+ "40-1.14",`
		`+ "40",`
		`+ "latest",`
		`+ {`
		`+ "fedora-minimal": ["aarch64", "ppc64le", "s390x", "x86_64"],`
		`+ "fedora": ["aarch64", "ppc64le", "s390x", "x86_64"],`
		`+ "fedora-toolbox": ["aarch64", "ppc64le", "s390x", "x86_64"],`
		`+ "fedora-kinoite": ["aarch64", "ppc64le", "x86_64"],`
		`+ "fedora-onyx": ["x86_64"],`
		`+ "fedora-sericea": ["aarch64", "x86_64"],`
		`+ "fedora-silverblue": ["aarch64", "ppc64le", "x86_64"],`
		`+ },`
		`+ ),`
		`+ ],`
		`+ )`
		`+ def test_containers(mock_subrun, fixtures_dir, compose):`
		`+ mock_subrun.return_value.returncode = 0`
		`+ message_file, cidorlabel, relnum, alias, expected_images = compose`
		`+ registries = ["registry.fedoraproject.org", "quay.io/fedora"]`
		`+ # mapping of reponames to expected image filename base strings`
		`+ repotoid = {`
		`+ "fedora": "Fedora-Container-Base-Generic",`
		`+ "fedora-minimal": "Fedora-Container-Base-Generic-Minimal",`
		`+ "fedora-toolbox": "Fedora-Container-Toolbox",`
		`+ "fedora-kinoite": "Fedora-Kinoite",`
		`+ "fedora-onyx": "Fedora-Onyx",`
		`+ "fedora-sericea": "Fedora-Sericea",`
		`+ "fedora-silverblue": "Fedora-Silverblue",`
		`+ }`
		`+`
		`+ with open(os.path.join(fixtures_dir, message_file)) as fd:`
		`+ msg = message.load_message(json.load(fd))`
		`+ config.conf["consumer_config"]["container"] = {"registries": registries}`
		`+`
		`+ consumer = Uploader()`
		`+ # disable handlers we don't want to hit in this test`
		`+ consumer.handlers = [consumer.handle_container]`
		`+ consumer(msg)`
		`+`
		`+ # this gives us a list of strings representing the commands run`
		`+ # (space-joined args iterables passed to _run)`
		`+ # we will check that every command we expect is in here, remove`
		`+ # them all, and assert it's empty at the end`
		`+ calls = [" ".join(call.args[0]) for call in mock_subrun.call_args_list]`
		`+`
		`+ for registry in registries:`
		`+ for exprepo, arches in expected_images.items():`
		`+ # find the expected calls to skopeo copy`
		`+ for arch in arches:`
		`+ ident = repotoid[exprepo]`
		`+ if "Container" in ident:`
		`+ expfn = f"oci-archive:/test/{ident}.{arch}-{cidorlabel}.oci.tar"`
		`+ else:`
		`+ # atomic desktop filenames don't have the arch and`
		`+ # use a non-standard compose label representation`
		`+ # https://gitlab.com/fedora/ostree/sig/-/issues/31`
		`+ expfn = f"oci-archive:/test/{ident}-{cidorlabel.replace('-', '.')}.ociarchive"`
		`+ expcall = f"skopeo copy {expfn} docker://{registry}/{exprepo}:{relnum}-{arch}"`
		`+ assert expcall in calls`
		`+ calls.remove(expcall)`
		`+ repopath = f"{registry}/{exprepo}:{relnum}"`
		`+ aliaspath = f"{registry}/{exprepo}:{alias}"`
		`+ # find the expected calls to buildah`
		`+ expcalls = (`
		`+ f"buildah rmi {repopath}",`
		`+ f"buildah rmi {aliaspath}",`
		`+ f"buildah manifest create {repopath} "`
		`+ + " ".join(f"{repopath}-{arch}" for arch in arches),`
		`+ f"buildah manifest create {aliaspath} "`
		`+ + " ".join(f"{repopath}-{arch}" for arch in arches),`
		`+ f"buildah manifest push {repopath} docker://{repopath} --all",`
		`+ f"buildah manifest push {aliaspath} docker://{aliaspath} --all",`
		`+ )`
		`+ for call in expcalls:`
		`+ assert call in calls`
		`+ calls.remove(call)`
		`+`
		`+ assert len(calls) == 0`
		`+`
		`+`
		`+ @pytest.mark.vcr`
		`+ @mock.patch("subprocess.run")`
		`+ @mock.patch("fedora_cloud_image_uploader.handler.Uploader.download_image")`
		`+ def test_containers_registries_not_configured(mock_dl, mock_run, fixtures_dir):`
		`+ """`
		`+ Test we correctly skip container handling if registries are not`
		`+ configured.`
		`+ """`
		`+ with open(os.path.join(fixtures_dir, "messages/rawhide_compose.json")) as fd:`
		`+ msg = message.load_message(json.load(fd))`
		`+ config.conf["consumer_config"]["container"] = {}`
		`+`
		`+ consumer = Uploader()`
		`+ # disable handlers we don't want to hit in this test`
		`+ consumer.handlers = [consumer.handle_container]`
		`+ consumer(msg)`
		`+`
		`+ assert mock_dl.call_count == 0`
		`+ assert mock_run.call_count == 0`
		`+`
		`+`
		`+ @pytest.mark.vcr`
		`+ @mock.patch(`
		`+ "fedora_cloud_image_uploader.handler.Uploader.download_image",`
		`+ lambda a, b, c, decompress: f"/test/{os.path.basename(b['path'].removesuffix('.xz'))}",`
		`+ )`
		`@mock.patch("fedora_cloud_image_uploader.handler.ansible_runner")`
		`def test_old_unsupported_azure_compose(mock_runner, fixtures_dir):`
		`mock_runner.interface.run.return_value.rc = 0`
		`@@ -78,9 +225,11 @@`
		`"storage_container_name": "vhds",`
		`"target_regions": {},`
		`}`
		`+ config.conf["consumer_config"]["container"] = {`
		`+ "registries": ["registry.fedoraproject.org", "quay.io/fedora"]`
		`+ }`

		`consumer = Uploader()`
		`- consumer.download_image = mock.Mock()`
		`consumer(msg)`
		`assert mock_runner.interface.run.call_count == 0`

		`@@ -128,25 +277,26 @@`
		`assert caplog.records[-1].msg == "Playbook failed with return code 1"`


		`- def test_azure_filters():`
		`+ @mock.patch("fedora_cloud_image_uploader.handler.ansible_runner")`
		`+ def test_azure_filters(mock_runner):`
		`"""Test the cases where AzureHandler should decide not to handle."""`
		`config.conf["consumer_config"]["azure"] = {}`
		`relinfo = mock.MagicMock()`
		`relinfo.relnum = 40`
		`- image = {"type": "notonewelike", "arch": "x86_64"}`
		`+ image = {"type": "notonewelike", "arch": "x86_64", "subvariant": "Cloud_Base"}`
		`consumer = Uploader()`
		`- with pytest.raises(CantHandle):`
		`- consumer.handle_azure(image, relinfo)`
		`+ consumer.handle_azure(image, relinfo)`
		`+ assert mock_runner.call_count == 0`
		`+`
		`image["type"] = "vhd-compressed"`
		`image["arch"] = "ppc64le"`
		`- with pytest.raises(NotHandled) as excinfo:`
		`- consumer.handle_azure(image, relinfo)`
		`- assert str(excinfo.value) == "Unsupported arch"`
		`+ consumer.handle_azure(image, relinfo)`
		`+ assert mock_runner.call_count == 0`
		`+`
		`image["arch"] = "x86_64"`
		`relinfo.relnum = 39`
		`- with pytest.raises(NotHandled) as excinfo:`
		`- consumer.handle_azure(image, relinfo)`
		`- assert str(excinfo.value) == "Images prior to F40 aren't supported"`
		`+ consumer.handle_azure(image, relinfo)`
		`+ assert mock_runner.call_count == 0`


		`@mock.patch("fedora_cloud_image_uploader.handler.ansible_runner")`
		`@@ -189,25 +339,13 @@`
		`consumer = Uploader()`

		`ffrel = mock_getrel.return_value`
		`- ffrel.all_images = [{"subvariant": "Cloud_Base", "type": "foo"}]`
		`+ ffrel.all_images = [{"subvariant": "Cloud_Base", "type": "foo", "format": "foo"}]`
		`mock_handler1 = mock.MagicMock()`
		`- mock_handler1.side_effect = CantHandle`
		`+ mock_handler1.side_effect = exceptions.Nack`
		`mock_handler2 = mock.MagicMock()`
		`- with mock.patch.object(consumer, "cloud_handlers", [mock_handler1, mock_handler2]):`
		`- consumer(msg)`
		`- # we should have continued to the second handler for CantHandle`
		`- assert mock_handler2.call_count == 1`
		`- mock_handler2.reset_mock()`
		`- mock_handler1.side_effect = NotHandled`
		`- with mock.patch.object(consumer, "cloud_handlers", [mock_handler1, mock_handler2]):`
		`+ consumer.handlers = [mock_handler1, mock_handler2]`
		`+ with pytest.raises(exceptions.Nack):`
		`consumer(msg)`
		`- # for NotHandled we should bail out and never reach second handler`
		`- assert mock_handler2.call_count == 0`
		`- mock_sleep.reset_mock()`
		`- mock_handler1.side_effect = exceptions.Nack`
		`- with mock.patch.object(consumer, "cloud_handlers", [mock_handler1, mock_handler2]):`
		`- with pytest.raises(exceptions.Nack):`
		`- consumer(msg)`
		`assert mock_handler2.call_count == 0`
		`# we should sleep before re-raising the Nack`
		`assert mock_sleep.call_count == 1`
		`@@ -389,3 +527,26 @@`
		`expected_calls.reverse()`
		`actual_calls = [call for call in client.gallery_image_versions.begin_delete.call_args_list]`
		`assert expected_calls == actual_calls`
		`+`
		`+`
		`+ def test_run_error(caplog):`
		`+ """Test the error handling in _run."""`
		`+ with pytest.raises(exceptions.Nack):`
		`+ _run(("ls", "/some/where/that/doesnt/exist"))`
		`+ assert (`
		`+ caplog.records[-1].getMessage()`
		`+ == "stderr: ls: cannot access '/some/where/that/doesnt/exist': No such file or directory\n"`
		`+ )`
		`+ assert caplog.records[-2].getMessage() == "stdout: "`
		`+ assert caplog.records[-3].getMessage() == "Command: ls /some/where/that/doesnt/exist returned 2"`
		`+ with pytest.raises(exceptions.Nack):`
		`+ _run(("commandthatdoesntexist",))`
		`+ assert caplog.records[-1].getMessage() == (`
		`+ "Command: commandthatdoesntexist caused error [Errno 2] "`
		`+ "No such file or directory: 'commandthatdoesntexist'"`
		`+ )`
		`+ with pytest.raises(exceptions.Nack):`
		`+ # this is a cute freeze_time trick to force a timeout`
		`+ with freeze_time("2024-05-28", auto_tick_seconds=10000):`
		`+ _run(("ls",))`
		`+ assert caplog.records[-1].getMessage() == "Command: ls timed out after two hours"`

adamwill commented a month ago

This adds support for publishing container images to registries.
It is intended to replace the sync-latest-container-base-image.sh
and sync-ostree-base-containers.sh scripts we currently use for
this purpose.

It finds "docker" or "ociarchive" type images in the compose and
matches them against a list of known "repos" (fedora-toolbox,
fedora-silverblue etc.) Images that match are pushed to the
registry using skopeo copy. Then a manifest is produced from
the published images, and published itself; if the compose is a
Rawhide compose, or a compose of the current stable release, a
manifest is also published under an alias ("rawhide" or "latest",
respectively).

This initial implementation is intentionally closely based on the
sync-latest-container-base-image.sh approach. The other script
uses a somewhat different approach where the release numbered
manifest is created from the local image files and then pushed
with --all, which also causes the image files to be pushed,
then the manifest is copied to the 'aliased' name. I think the
eventual outcome is really the same in both cases, though.

The container support can be activated by configuring at least
one registry in the consumer config. If this is not done,
container images will not be handled.

Signed-off-by: Adam Williamson awilliam@redhat.com

rebased onto 5adae52

a month ago

adamwill commented a month ago

For the record, here is the complete list of commands it would run for each compose. The reason why the atomic desktops are missed for the Rawhide compose is a bit interesting. That compose has aged out from the mirrors now, so fedfind - when asked for the images from that compose - falls back on getting the info from PDC (you can see this happening if you look at the pyvcr "cassette" for the test).

However, PDC doesn't actually contain the full data because of this bug - PDC doesn't just dump in the JSON file's contents unmodified, it wants to check or parse it somehow (I don't know why), and it doesn't know about ociarchive images, so it leaves them out. We are trying to get rid of PDC, so that bug won't likely ever get fixed (if PDC ever does go away, fedfind just won't be able to give you information on the images that were in a compose which is no longer present on the mirrors).

If I'd recorded the cassette while the compose was still on the mirrors, fedfind would've got the unmodified metadata from the compose tree and the ociarchive images would be there, but I didn't.

Rawhide commands:

skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.aarch64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.aarch64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.aarch64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.aarch64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.x86_64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.x86_64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.x86_64-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.x86_64-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.s390x-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.s390x-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.s390x-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.s390x-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.ppc64le-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.ppc64le-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora:41-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.ppc64le-Rawhide-20240501.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.ppc64le-Rawhide-20240501.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-ppc64le
buildah rmi registry.fedoraproject.org/fedora-minimal:41
buildah manifest create registry.fedoraproject.org/fedora-minimal:41 registry.fedoraproject.org/fedora-minimal:41-aarch64 registry.fedoraproject.org/fedora-minimal:41-x86_64 registry.fedoraproject.org/fedora-minimal:41-s390x registry.fedoraproject.org/fedora-minimal:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora-minimal:41 docker://registry.fedoraproject.org/fedora-minimal:41 --all
buildah rmi registry.fedoraproject.org/fedora-minimal:rawhide
buildah manifest create registry.fedoraproject.org/fedora-minimal:rawhide registry.fedoraproject.org/fedora-minimal:41-aarch64 registry.fedoraproject.org/fedora-minimal:41-x86_64 registry.fedoraproject.org/fedora-minimal:41-s390x registry.fedoraproject.org/fedora-minimal:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora-minimal:rawhide docker://registry.fedoraproject.org/fedora-minimal:rawhide --all
buildah rmi quay.io/fedora/fedora-minimal:41
buildah manifest create quay.io/fedora/fedora-minimal:41 quay.io/fedora/fedora-minimal:41-aarch64 quay.io/fedora/fedora-minimal:41-x86_64 quay.io/fedora/fedora-minimal:41-s390x quay.io/fedora/fedora-minimal:41-ppc64le
buildah manifest push quay.io/fedora/fedora-minimal:41 docker://quay.io/fedora/fedora-minimal:41 --all
buildah rmi quay.io/fedora/fedora-minimal:rawhide
buildah manifest create quay.io/fedora/fedora-minimal:rawhide quay.io/fedora/fedora-minimal:41-aarch64 quay.io/fedora/fedora-minimal:41-x86_64 quay.io/fedora/fedora-minimal:41-s390x quay.io/fedora/fedora-minimal:41-ppc64le
buildah manifest push quay.io/fedora/fedora-minimal:rawhide docker://quay.io/fedora/fedora-minimal:rawhide --all
buildah rmi registry.fedoraproject.org/fedora:41
buildah manifest create registry.fedoraproject.org/fedora:41 registry.fedoraproject.org/fedora:41-aarch64 registry.fedoraproject.org/fedora:41-x86_64 registry.fedoraproject.org/fedora:41-s390x registry.fedoraproject.org/fedora:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora:41 docker://registry.fedoraproject.org/fedora:41 --all
buildah rmi registry.fedoraproject.org/fedora:rawhide
buildah manifest create registry.fedoraproject.org/fedora:rawhide registry.fedoraproject.org/fedora:41-aarch64 registry.fedoraproject.org/fedora:41-x86_64 registry.fedoraproject.org/fedora:41-s390x registry.fedoraproject.org/fedora:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora:rawhide docker://registry.fedoraproject.org/fedora:rawhide --all
buildah rmi quay.io/fedora/fedora:41
buildah manifest create quay.io/fedora/fedora:41 quay.io/fedora/fedora:41-aarch64 quay.io/fedora/fedora:41-x86_64 quay.io/fedora/fedora:41-s390x quay.io/fedora/fedora:41-ppc64le
buildah manifest push quay.io/fedora/fedora:41 docker://quay.io/fedora/fedora:41 --all
buildah rmi quay.io/fedora/fedora:rawhide
buildah manifest create quay.io/fedora/fedora:rawhide quay.io/fedora/fedora:41-aarch64 quay.io/fedora/fedora:41-x86_64 quay.io/fedora/fedora:41-s390x quay.io/fedora/fedora:41-ppc64le
buildah manifest push quay.io/fedora/fedora:rawhide docker://quay.io/fedora/fedora:rawhide --all
buildah rmi registry.fedoraproject.org/fedora-toolbox:41
buildah manifest create registry.fedoraproject.org/fedora-toolbox:41 registry.fedoraproject.org/fedora-toolbox:41-aarch64 registry.fedoraproject.org/fedora-toolbox:41-x86_64 registry.fedoraproject.org/fedora-toolbox:41-s390x registry.fedoraproject.org/fedora-toolbox:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora-toolbox:41 docker://registry.fedoraproject.org/fedora-toolbox:41 --all
buildah rmi registry.fedoraproject.org/fedora-toolbox:rawhide
buildah manifest create registry.fedoraproject.org/fedora-toolbox:rawhide registry.fedoraproject.org/fedora-toolbox:41-aarch64 registry.fedoraproject.org/fedora-toolbox:41-x86_64 registry.fedoraproject.org/fedora-toolbox:41-s390x registry.fedoraproject.org/fedora-toolbox:41-ppc64le
buildah manifest push registry.fedoraproject.org/fedora-toolbox:rawhide docker://registry.fedoraproject.org/fedora-toolbox:rawhide --all
buildah rmi quay.io/fedora/fedora-toolbox:41
buildah manifest create quay.io/fedora/fedora-toolbox:41 quay.io/fedora/fedora-toolbox:41-aarch64 quay.io/fedora/fedora-toolbox:41-x86_64 quay.io/fedora/fedora-toolbox:41-s390x quay.io/fedora/fedora-toolbox:41-ppc64le
buildah manifest push quay.io/fedora/fedora-toolbox:41 docker://quay.io/fedora/fedora-toolbox:41 --all
buildah rmi quay.io/fedora/fedora-toolbox:rawhide
buildah manifest create quay.io/fedora/fedora-toolbox:rawhide quay.io/fedora/fedora-toolbox:41-aarch64 quay.io/fedora/fedora-toolbox:41-x86_64 quay.io/fedora/fedora-toolbox:41-s390x quay.io/fedora/fedora-toolbox:41-ppc64le
buildah manifest push quay.io/fedora/fedora-toolbox:rawhide docker://quay.io/fedora/fedora-toolbox:rawhide --all

F40 candidate compose commands:

skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.aarch64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-minimal:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.aarch64-40-1.14.oci.tar docker://quay.io/fedora/fedora-minimal:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.aarch64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.aarch64-40-1.14.oci.tar docker://quay.io/fedora/fedora:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.aarch64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.aarch64-40-1.14.oci.tar docker://quay.io/fedora/fedora-toolbox:40-aarch64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.ppc64le-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-minimal:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.ppc64le-40-1.14.oci.tar docker://quay.io/fedora/fedora-minimal:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.ppc64le-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.ppc64le-40-1.14.oci.tar docker://quay.io/fedora/fedora:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.ppc64le-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.ppc64le-40-1.14.oci.tar docker://quay.io/fedora/fedora-toolbox:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.s390x-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-minimal:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.s390x-40-1.14.oci.tar docker://quay.io/fedora/fedora-minimal:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.s390x-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.s390x-40-1.14.oci.tar docker://quay.io/fedora/fedora:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.s390x-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.s390x-40-1.14.oci.tar docker://quay.io/fedora/fedora-toolbox:40-s390x
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.x86_64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-minimal:40-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic-Minimal.x86_64-40-1.14.oci.tar docker://quay.io/fedora/fedora-minimal:40-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.x86_64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora:40-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Base-Generic.x86_64-40-1.14.oci.tar docker://quay.io/fedora/fedora:40-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.x86_64-40-1.14.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:40-x86_64
skopeo copy oci-archive:/test/Fedora-Container-Toolbox.x86_64-40-1.14.oci.tar docker://quay.io/fedora/fedora-toolbox:40-x86_64
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-kinoite:40-aarch64
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://quay.io/fedora/fedora-kinoite:40-aarch64
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-kinoite:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://quay.io/fedora/fedora-kinoite:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-kinoite:40-x86_64
skopeo copy oci-archive:/test/Fedora-Kinoite-40.1.14.ociarchive docker://quay.io/fedora/fedora-kinoite:40-x86_64
skopeo copy oci-archive:/test/Fedora-Onyx-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-onyx:40-x86_64
skopeo copy oci-archive:/test/Fedora-Onyx-40.1.14.ociarchive docker://quay.io/fedora/fedora-onyx:40-x86_64
skopeo copy oci-archive:/test/Fedora-Sericea-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-sericea:40-aarch64
skopeo copy oci-archive:/test/Fedora-Sericea-40.1.14.ociarchive docker://quay.io/fedora/fedora-sericea:40-aarch64
skopeo copy oci-archive:/test/Fedora-Sericea-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-sericea:40-x86_64
skopeo copy oci-archive:/test/Fedora-Sericea-40.1.14.ociarchive docker://quay.io/fedora/fedora-sericea:40-x86_64
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-silverblue:40-aarch64
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://quay.io/fedora/fedora-silverblue:40-aarch64
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-silverblue:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://quay.io/fedora/fedora-silverblue:40-ppc64le
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://registry.fedoraproject.org/fedora-silverblue:40-x86_64
skopeo copy oci-archive:/test/Fedora-Silverblue-40.1.14.ociarchive docker://quay.io/fedora/fedora-silverblue:40-x86_64
buildah rmi registry.fedoraproject.org/fedora-minimal:40
buildah manifest create registry.fedoraproject.org/fedora-minimal:40 registry.fedoraproject.org/fedora-minimal:40-aarch64 registry.fedoraproject.org/fedora-minimal:40-ppc64le registry.fedoraproject.org/fedora-minimal:40-s390x registry.fedoraproject.org/fedora-minimal:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-minimal:40 docker://registry.fedoraproject.org/fedora-minimal:40 --all
buildah rmi registry.fedoraproject.org/fedora-minimal:latest
buildah manifest create registry.fedoraproject.org/fedora-minimal:latest registry.fedoraproject.org/fedora-minimal:40-aarch64 registry.fedoraproject.org/fedora-minimal:40-ppc64le registry.fedoraproject.org/fedora-minimal:40-s390x registry.fedoraproject.org/fedora-minimal:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-minimal:latest docker://registry.fedoraproject.org/fedora-minimal:latest --all
buildah rmi quay.io/fedora/fedora-minimal:40
buildah manifest create quay.io/fedora/fedora-minimal:40 quay.io/fedora/fedora-minimal:40-aarch64 quay.io/fedora/fedora-minimal:40-ppc64le quay.io/fedora/fedora-minimal:40-s390x quay.io/fedora/fedora-minimal:40-x86_64
buildah manifest push quay.io/fedora/fedora-minimal:40 docker://quay.io/fedora/fedora-minimal:40 --all
buildah rmi quay.io/fedora/fedora-minimal:latest
buildah manifest create quay.io/fedora/fedora-minimal:latest quay.io/fedora/fedora-minimal:40-aarch64 quay.io/fedora/fedora-minimal:40-ppc64le quay.io/fedora/fedora-minimal:40-s390x quay.io/fedora/fedora-minimal:40-x86_64
buildah manifest push quay.io/fedora/fedora-minimal:latest docker://quay.io/fedora/fedora-minimal:latest --all
buildah rmi registry.fedoraproject.org/fedora:40
buildah manifest create registry.fedoraproject.org/fedora:40 registry.fedoraproject.org/fedora:40-aarch64 registry.fedoraproject.org/fedora:40-ppc64le registry.fedoraproject.org/fedora:40-s390x registry.fedoraproject.org/fedora:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora:40 docker://registry.fedoraproject.org/fedora:40 --all
buildah rmi registry.fedoraproject.org/fedora:latest
buildah manifest create registry.fedoraproject.org/fedora:latest registry.fedoraproject.org/fedora:40-aarch64 registry.fedoraproject.org/fedora:40-ppc64le registry.fedoraproject.org/fedora:40-s390x registry.fedoraproject.org/fedora:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora:latest docker://registry.fedoraproject.org/fedora:latest --all
buildah rmi quay.io/fedora/fedora:40
buildah manifest create quay.io/fedora/fedora:40 quay.io/fedora/fedora:40-aarch64 quay.io/fedora/fedora:40-ppc64le quay.io/fedora/fedora:40-s390x quay.io/fedora/fedora:40-x86_64
buildah manifest push quay.io/fedora/fedora:40 docker://quay.io/fedora/fedora:40 --all
buildah rmi quay.io/fedora/fedora:latest
buildah manifest create quay.io/fedora/fedora:latest quay.io/fedora/fedora:40-aarch64 quay.io/fedora/fedora:40-ppc64le quay.io/fedora/fedora:40-s390x quay.io/fedora/fedora:40-x86_64
buildah manifest push quay.io/fedora/fedora:latest docker://quay.io/fedora/fedora:latest --all
buildah rmi registry.fedoraproject.org/fedora-toolbox:40
buildah manifest create registry.fedoraproject.org/fedora-toolbox:40 registry.fedoraproject.org/fedora-toolbox:40-aarch64 registry.fedoraproject.org/fedora-toolbox:40-ppc64le registry.fedoraproject.org/fedora-toolbox:40-s390x registry.fedoraproject.org/fedora-toolbox:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-toolbox:40 docker://registry.fedoraproject.org/fedora-toolbox:40 --all
buildah rmi registry.fedoraproject.org/fedora-toolbox:latest
buildah manifest create registry.fedoraproject.org/fedora-toolbox:latest registry.fedoraproject.org/fedora-toolbox:40-aarch64 registry.fedoraproject.org/fedora-toolbox:40-ppc64le registry.fedoraproject.org/fedora-toolbox:40-s390x registry.fedoraproject.org/fedora-toolbox:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-toolbox:latest docker://registry.fedoraproject.org/fedora-toolbox:latest --all
buildah rmi quay.io/fedora/fedora-toolbox:40
buildah manifest create quay.io/fedora/fedora-toolbox:40 quay.io/fedora/fedora-toolbox:40-aarch64 quay.io/fedora/fedora-toolbox:40-ppc64le quay.io/fedora/fedora-toolbox:40-s390x quay.io/fedora/fedora-toolbox:40-x86_64
buildah manifest push quay.io/fedora/fedora-toolbox:40 docker://quay.io/fedora/fedora-toolbox:40 --all
buildah rmi quay.io/fedora/fedora-toolbox:latest
buildah manifest create quay.io/fedora/fedora-toolbox:latest quay.io/fedora/fedora-toolbox:40-aarch64 quay.io/fedora/fedora-toolbox:40-ppc64le quay.io/fedora/fedora-toolbox:40-s390x quay.io/fedora/fedora-toolbox:40-x86_64
buildah manifest push quay.io/fedora/fedora-toolbox:latest docker://quay.io/fedora/fedora-toolbox:latest --all
buildah rmi registry.fedoraproject.org/fedora-kinoite:40
buildah manifest create registry.fedoraproject.org/fedora-kinoite:40 registry.fedoraproject.org/fedora-kinoite:40-aarch64 registry.fedoraproject.org/fedora-kinoite:40-ppc64le registry.fedoraproject.org/fedora-kinoite:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-kinoite:40 docker://registry.fedoraproject.org/fedora-kinoite:40 --all
buildah rmi registry.fedoraproject.org/fedora-kinoite:latest
buildah manifest create registry.fedoraproject.org/fedora-kinoite:latest registry.fedoraproject.org/fedora-kinoite:40-aarch64 registry.fedoraproject.org/fedora-kinoite:40-ppc64le registry.fedoraproject.org/fedora-kinoite:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-kinoite:latest docker://registry.fedoraproject.org/fedora-kinoite:latest --all
buildah rmi quay.io/fedora/fedora-kinoite:40
buildah manifest create quay.io/fedora/fedora-kinoite:40 quay.io/fedora/fedora-kinoite:40-aarch64 quay.io/fedora/fedora-kinoite:40-ppc64le quay.io/fedora/fedora-kinoite:40-x86_64
buildah manifest push quay.io/fedora/fedora-kinoite:40 docker://quay.io/fedora/fedora-kinoite:40 --all
buildah rmi quay.io/fedora/fedora-kinoite:latest
buildah manifest create quay.io/fedora/fedora-kinoite:latest quay.io/fedora/fedora-kinoite:40-aarch64 quay.io/fedora/fedora-kinoite:40-ppc64le quay.io/fedora/fedora-kinoite:40-x86_64
buildah manifest push quay.io/fedora/fedora-kinoite:latest docker://quay.io/fedora/fedora-kinoite:latest --all
buildah rmi registry.fedoraproject.org/fedora-onyx:40
buildah manifest create registry.fedoraproject.org/fedora-onyx:40 registry.fedoraproject.org/fedora-onyx:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-onyx:40 docker://registry.fedoraproject.org/fedora-onyx:40 --all
buildah rmi registry.fedoraproject.org/fedora-onyx:latest
buildah manifest create registry.fedoraproject.org/fedora-onyx:latest registry.fedoraproject.org/fedora-onyx:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-onyx:latest docker://registry.fedoraproject.org/fedora-onyx:latest --all
buildah rmi quay.io/fedora/fedora-onyx:40
buildah manifest create quay.io/fedora/fedora-onyx:40 quay.io/fedora/fedora-onyx:40-x86_64
buildah manifest push quay.io/fedora/fedora-onyx:40 docker://quay.io/fedora/fedora-onyx:40 --all
buildah rmi quay.io/fedora/fedora-onyx:latest
buildah manifest create quay.io/fedora/fedora-onyx:latest quay.io/fedora/fedora-onyx:40-x86_64
buildah manifest push quay.io/fedora/fedora-onyx:latest docker://quay.io/fedora/fedora-onyx:latest --all
buildah rmi registry.fedoraproject.org/fedora-sericea:40
buildah manifest create registry.fedoraproject.org/fedora-sericea:40 registry.fedoraproject.org/fedora-sericea:40-aarch64 registry.fedoraproject.org/fedora-sericea:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-sericea:40 docker://registry.fedoraproject.org/fedora-sericea:40 --all
buildah rmi registry.fedoraproject.org/fedora-sericea:latest
buildah manifest create registry.fedoraproject.org/fedora-sericea:latest registry.fedoraproject.org/fedora-sericea:40-aarch64 registry.fedoraproject.org/fedora-sericea:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-sericea:latest docker://registry.fedoraproject.org/fedora-sericea:latest --all
buildah rmi quay.io/fedora/fedora-sericea:40
buildah manifest create quay.io/fedora/fedora-sericea:40 quay.io/fedora/fedora-sericea:40-aarch64 quay.io/fedora/fedora-sericea:40-x86_64
buildah manifest push quay.io/fedora/fedora-sericea:40 docker://quay.io/fedora/fedora-sericea:40 --all
buildah rmi quay.io/fedora/fedora-sericea:latest
buildah manifest create quay.io/fedora/fedora-sericea:latest quay.io/fedora/fedora-sericea:40-aarch64 quay.io/fedora/fedora-sericea:40-x86_64
buildah manifest push quay.io/fedora/fedora-sericea:latest docker://quay.io/fedora/fedora-sericea:latest --all
buildah rmi registry.fedoraproject.org/fedora-silverblue:40
buildah manifest create registry.fedoraproject.org/fedora-silverblue:40 registry.fedoraproject.org/fedora-silverblue:40-aarch64 registry.fedoraproject.org/fedora-silverblue:40-ppc64le registry.fedoraproject.org/fedora-silverblue:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-silverblue:40 docker://registry.fedoraproject.org/fedora-silverblue:40 --all
buildah rmi registry.fedoraproject.org/fedora-silverblue:latest
buildah manifest create registry.fedoraproject.org/fedora-silverblue:latest registry.fedoraproject.org/fedora-silverblue:40-aarch64 registry.fedoraproject.org/fedora-silverblue:40-ppc64le registry.fedoraproject.org/fedora-silverblue:40-x86_64
buildah manifest push registry.fedoraproject.org/fedora-silverblue:latest docker://registry.fedoraproject.org/fedora-silverblue:latest --all
buildah rmi quay.io/fedora/fedora-silverblue:40
buildah manifest create quay.io/fedora/fedora-silverblue:40 quay.io/fedora/fedora-silverblue:40-aarch64 quay.io/fedora/fedora-silverblue:40-ppc64le quay.io/fedora/fedora-silverblue:40-x86_64
buildah manifest push quay.io/fedora/fedora-silverblue:40 docker://quay.io/fedora/fedora-silverblue:40 --all
buildah rmi quay.io/fedora/fedora-silverblue:latest
buildah manifest create quay.io/fedora/fedora-silverblue:latest quay.io/fedora/fedora-silverblue:40-aarch64 quay.io/fedora/fedora-silverblue:40-ppc64le quay.io/fedora/fedora-silverblue:40-x86_64
buildah manifest push quay.io/fedora/fedora-silverblue:latest docker://quay.io/fedora/fedora-silverblue:latest --all

obviously /test/ is just a test path, in reality it would download the images to a temporary directory and the path used in the skopeo commands would include that directory.

Edited a month ago by adamwill

adamwill commented a month ago

@walters @kevin @siosm if you could check this looks to be doing the right things, that'd be great.

Edited a month ago by adamwill

jcline commented on line 19 of fedora_cloud_image_uploader/handler.py a month ago

I'd recommend adding a timeout to this call with timeout=<seconds>. I think it can be quite generous (hours even), but that way the service won't hang if a subprocess gets stuck.

Also, Python doesn't seem to document that the subprocess.run call can raise OSError-based exceptions beyond hinting it pipes things along to Popen, but it will do so if, for example, the executable doesn't exist:

>>> subprocess.run(["notinpath", "--oh-no"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'notinpath'

I'd recommend handling that exception in addition to the TimeoutError that's raised when a timeout is hit.

jcline commented on line 140 of fedora_cloud_image_uploader/handler.py a month ago

I'd recommend writing this as

registries = self.conf.get("container", {}).get("registries")

if you want it to be optional to have a container section of the config since at it currently is written, it'll raise an AttributeError when it calls get on the default None returned from the first get call.

kevin commented a month ago

Commands all look reasonable to me.

adamwill commented a month ago

I tend to err on the side of just letting exceptions run, for message consumers, because it sure lets you know when something's wrong with the consumer. But as this is your project, how do you want to "handle" exceptions exactly, in this case? Hide them with error messages nobody will ever look at? Funnel them into the "wait 60 seconds, then raise Nack instead" mechanism? Something else?

rebased onto d859c30

a month ago

adamwill commented a month ago

I'd recommend writing this as
registries = self.conf.get("container", {}).get("registries")
if you want it to be optional to have a container section of the config since at it currently is written, it'll raise an AttributeError when it calls get on the default None returned from the first get call.

Oh, yeah, I meant to do it that way but forgot. Thanks. Fixed.

jcline commented a month ago

I tend to err on the side of just letting exceptions run, for message consumers, because it sure lets you know when something's wrong with the consumer. But as this is your project, how do you want to "handle" exceptions exactly, in this case? Hide them with error messages nobody will ever look at? Funnel them into the "wait 60 seconds, then raise Nack instead" mechanism? Something else?

Ordinarily I'd wire something like this up to Sentry (or a similar error-aggregating service), but I guess the only way to get notified of a service problem is to crash the pod? Or can we at least alert off error-level logs?

I think for the timeout error that's likely a transient network issue and could be handled via a little wait and Nack. For the OSError, eh, crashing is okay I guess since the container would need re-building, but I would recommend including a comment on the call explicitly noting you're handling the OSError that way.

adamwill commented a month ago

Ah, right, you're running the consumer as a pod. I tend to run consumers as services in a VM, which probably gives somewhat different behaviour.

This is documented at https://fedora-messaging.readthedocs.io/en/stable/user-guide/consuming.html#exceptions . The difference between "raising Nack" and "raising any non-special exception" seems to be that in the "Nack" case, the consumer is not "cancelled" (whatever that means to the server, exactly) and no error is logged. In the "any other exception" case, the consumer is "cancelled", and the exception is logged as an error.

In a VM, when the consumer raises an exception, in practice what happens is I get an email - because the consumer configuration says to email me error logs - then the consumer service dies, auto-restarts itself and immediately tries to consume the message again (so when something goes wrong overnight I wake up to several hundred emails, which sure lets me know something went wrong, unless I put in a backoff mechanism). I guess if you're running the consumer in a pod, when it dies, the pod dies...so then it depends whether you configure the pod to auto-restart?

So...on the whole, yeah, I agree if we can be sure an error looks like a network connectivity issue, we should wait and raise Nack, at least a few times (but maybe if this happens more than five times in a row we should crash). If we're not sure what the problem is, I don't think we should eat the original exception or raise Nack, I think it's probably best to crash on the initial exception, because otherwise the consumer might get stuck just looping over a message it can't handle every 60 seconds and not telling anyone about it. In which case we might not notice till someone realized the container images were stale, or the backlog of messages got so big it started triggering infra monitoring alerts. Or, I guess alternatively, we could log the exception trace as an error - and ensure the consumer config is set up to email somebody when that happens - and then wait and raise Nack?

Edited a month ago by adamwill

adamwill commented a month ago

oh, yeah, logging is very configurable via consumer config, to elaborate on that a bit. e.g. for the openQA consumers we have this stanza in the consumer config file:

[log_config.handlers.email]
class = "logging.handlers.SMTPHandler"
formatter = "simple"
level = "ERROR"
mailhost = "bastion"
fromaddr = "root@openqa.fedoraproject.org"
toaddrs = ["adamwill@fp.o","lruzicka@fp.o",]
subject = "openQA scheduler error"

which is what implements "email me and lukas if anything goes wrong".

jcline commented a month ago

Ah, I guess I should clarify what I was (ineffectually) inquiring about was if there was a better solution available than configuring the pod to email error logs to me. If that is what we've got, I can make that work.

I do get an email if the pod crashes and it will restart, however I don't love that as an intentional error handling strategy. My preference would be to catch the exception, log it at the error level (or even at the exception level for the traceback) with the assumption that the person running the service has a way to hear about errors, and Nack the message after introducing a little sleep.

Especially if there's no log aggregator, not crashing has the advantage of allowing the investigator to see the logs proceeding the error.

jcline commented on line 176 of fedora_cloud_image_uploader/handler.py a month ago

You can drop this if/else block if you switch self.container_repos to a defaultdict:

>>> from collections import defaultdict
>>> x = defaultdict(list)
>>> x["key"].append(1)
>>> x["key"].append(2)
>>> x
defaultdict(<class 'list'>, {'key': [1, 2]})
>>> x["key"]
[1, 2]
>>>

jcline commented on line 37 of tests/test_handler.py a month ago

This is fine, but does:

consumer.container_handlers = []

not just work?

adamwill commented a month ago

You can drop this if/else block if you switch self.container_repos to a defaultdict:
```

from collections import defaultdict
x = defaultdict(list)
x["key"].append(1)
x["key"].append(2)
x
defaultdict(<class 'list'>, {'key': [1, 2]})
x["key"]
[1, 2]

```

yes, I know, but I tend to prefer the if/else for trivial implementations. IIRC defaultdict has been moved around before, which was a bit of a pain for code that uses it, so I tend to use it only when the alternative is really painful.

jcline commented on line 50 of fedora_cloud_image_uploader/handler.py a month ago

How about pushing the subvariant check down into the cloud handlers?

Since you've got a CantHandle exception which is being raised in some cases to indicate "I'm not the handler for this thing" I think it's clearer to use that as the one and only filtering mechanism. You can rename self.cloud_handlers to just self.handlers and add self.handle_container to the list.

adamwill commented a month ago

This is fine, but does:
consumer.container_handlers = []
not just work?

oh, yeah, it probably would...overthinking...

adamwill commented a month ago

How about pushing the subvariant check down into the cloud handlers?

Since you've got a CantHandle exception which is being raised in some cases to indicate "I'm not the handler for this thing" I think it's clearer to use that as the one and only filtering mechanism. You can rename self.cloud_handlers to just self.handlers and add self.handle_container to the list.

I mean, sure, we could, I just wasn't sure whether you'd want to, and it seemed potentially a slightly different scope from this PR. the existing code at least implies that the approach in this PR is what's "intended" when adding a new "class" of consumers; changing to a "we throw every consumer at every image and see what happens" model seems like kind of a separate change from "adding a new class of consumers", although I guess it's arguable.

if that's the way you want to go, it's fine with me, I might make it a separate, earlier commit, though...

adamwill commented a month ago

note that would mean every cloud consumer would have to be careful not to try and handle non-cloud images, and every container consumer would have to be careful not to try and handle non-container images. in practice at present that's fine, it's just...worth noting.

Edited a month ago by adamwill

rebased onto a40c3dc

a month ago

adamwill commented a month ago

okay, rejigged with more error handling, and a handler vs. image battle royale! in the end I decided it was fine for it all to be in one commit.

jcline commented on line 135 of fedora_cloud_image_uploader/handler.py a month ago

This check needs to move before any other check as, if this runs before a handler that could deal with the image and there's no registries configured, the image will be incorrectly skipped.

Alternatively (or additionally), it might make sense to be rid of the NotHandled exception entirely, log it at info level, and just return early. As an interface it's a little tricky as this bug shows, and if all handlers are supposed to decide if they want to handle the image or not, one handler probably shouldn't be able to skip all remaining handlers.

kevin commented a month ago

To be clear, when you are talking about NACKing things, thats reject/put back on queue? Or is that drop message so it never gets processed?

As far as monitoring, yeah, we have openshift mailing on things like pod crashes, etc.
It should also notify on health checks failing, so we could perhaps add a health check, if we get errors/issues, start reporting unhealthy so it alerts?

We are also slowly ramping up zabbix, and could probibly do some monitoring from there longer term.

jcline commented a month ago

To be clear, when you are talking about NACKing things, thats reject/put back on queue? Or is that drop message so it never gets processed?

Yeah, it'll tell the broker to re-queue the message and try again later. While I don't recall if the specification indicates whether or not it goes at the back of the queue, in practice other queued messages will get delivered first so it's an okay strategy if the message is the problem.

We could implement more clever client-side retrying, buuuut when you've got a hammer and something that looks vaguely like a nail...

As far as monitoring, yeah, we have openshift mailing on things like pod crashes, etc.
It should also notify on health checks failing, so we could perhaps add a health check, if we get errors/issues, start reporting unhealthy so it alerts?

We are also slowly ramping up zabbix, and could probibly do some monitoring from there longer term.

I think email is fine for the short-term (assuming the pods have the ability to connect to "bastion" for email). A health check would be nice, but I think would require a bit more work - my memory says it needs to be HTTP?

rebased onto 8a526d6

a month ago

rebased onto e286e1e

a month ago

adamwill commented a month ago

IIRC container healthcheck is just an arbitrary command. it's very common to have it be just a curl command that tries to hit an endpoint of the webapp running in the container, but it doesn't have to be.

adamwill commented a month ago

This check needs to move before any other check as, if this runs before a handler that could deal with the image and there's no registries configured, the image will be incorrectly skipped.

Good catch, fixed.

Alternatively (or additionally), it might make sense to be rid of the NotHandled exception entirely, log it at info level, and just return early. As an interface it's a little tricky as this bug shows, and if all handlers are supposed to decide if they want to handle the image or not, one handler probably shouldn't be able to skip all remaining handlers.

Yeah, I understand the point, I'm a bit on the fence about it. Honestly the initial reason I invented NotHandled was to avoid incorrectly logging "no handler found" when we fell out of the loop, but I had to sacrifice that log message for the new approach anyway (because we now really iterate over all images in the compose, so we won't find a handler for tons of them).

I guess conceptually it kinda depends whether our model is "there should be exactly one correct handler for each image" (in which case it seems defensible for the handler that has decided it is The Handler for the given image to declare it NotHandled), or "any number of handlers might possibly handle any given image" (in which case that isn't really defensible). I guess the current code kinda implicitly splits the difference (because we don't drop out of the loop once any single handler decides to handle an image, but we do have NotHandled). WDYT?

adamwill commented a month ago

I suppose we can't really enforce One Handler Per Image unless we get fancier about it, which is harder with the "handlers are just callables" model. With the "handlers are classes" design we could have the handlers have a match method and have __call__ call that first, and if it gets more than one "match" for an image it could blow up (this was my initial design, with the class-y approach, before I simplified it to just do both the filtering and actual handling within a single method). But with "handlers are callables" it's difficult.

So...I guess that angles towards "declare the model is 'multiple handlers can potentially handle the same image' and get rid of NotHandled"?

jcline commented a month ago

I don't think there's a particular reason to require one handler per image. In practice it'll likely be that way, but if more than one handler wants to do a thing with an image that's not a problem.

adamwill commented a month ago

...and I suppose if we get rid of NotHandled we don't really need CantHandle either, we just have the handlers return...

rebased onto f639d0e

a month ago

adamwill commented a month ago

okay, rebased with the CantHandle and NotHandled concepts now dropped, handlers just return if they don't want to handle the image. We could have more logging of this, I guess, but it'd get kinda verbose as we'd log on dozens of images per compose. I think it might be better to just add logging on the fly if we actually run into a problem.

rebased onto 2ca7189

a month ago

adamwill commented a month ago

For comparison, I hacked up sync-latest-container-base-image.sh to just print out the commands it runs, and also cut out candidate-registry for a closer comparison, and this is what it would do right now, for Rawhide (so it's finding images from the current Rawhide compose):

skopeo copy oci-archive:Fedora-Container-Base-Generic.aarch64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-aarch64
skopeo copy oci-archive:Fedora-Container-Base-Generic.aarch64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora:41-aarch64
skopeo copy oci-archive:Fedora-Container-Base-Generic.ppc64le-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Base-Generic.ppc64le-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Base-Generic.s390x-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-s390x
skopeo copy oci-archive:Fedora-Container-Base-Generic.s390x-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora:41-s390x
skopeo copy oci-archive:Fedora-Container-Base-Generic.x86_64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora:41-x86_64
skopeo copy oci-archive:Fedora-Container-Base-Generic.x86_64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora:41-x86_64
buildah rmi registry.fedoraproject.org/fedora:rawhide
buildah manifest create registry.fedoraproject.org/fedora:rawhide docker://registry.fedoraproject.org/fedora:41-aarch64 docker://registry.fedoraproject.org/fedora:41-ppc64le docker://registry.fedoraproject.org/fedora:41-s390x docker://registry.fedoraproject.org/fedora:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora:rawhide docker://registry.fedoraproject.org/fedora:rawhide --all
buildah rmi registry.fedoraproject.org/fedora:41
buildah manifest create registry.fedoraproject.org/fedora:41 docker://registry.fedoraproject.org/fedora:41-aarch64 docker://registry.fedoraproject.org/fedora:41-ppc64le docker://registry.fedoraproject.org/fedora:41-s390x docker://registry.fedoraproject.org/fedora:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora:41 docker://registry.fedoraproject.org/fedora:41 --all
buildah rmi quay.io/fedora/fedora:rawhide
buildah manifest create quay.io/fedora/fedora:rawhide docker://quay.io/fedora/fedora:41-aarch64 docker://quay.io/fedora/fedora:41-ppc64le docker://quay.io/fedora/fedora:41-s390x docker://quay.io/fedora/fedora:41-x86_64
buildah manifest push quay.io/fedora/fedora:rawhide docker://quay.io/fedora/fedora:rawhide --all
buildah rmi quay.io/fedora/fedora:41
buildah manifest create quay.io/fedora/fedora:41 docker://quay.io/fedora/fedora:41-aarch64 docker://quay.io/fedora/fedora:41-ppc64le docker://quay.io/fedora/fedora:41-s390x docker://quay.io/fedora/fedora:41-x86_64
buildah manifest push quay.io/fedora/fedora:41 docker://quay.io/fedora/fedora:41 --all
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-aarch64
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.aarch64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-aarch64
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.ppc64le-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-s390x
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.s390x-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-s390x
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-minimal:41-x86_64
skopeo copy oci-archive:Fedora-Container-Base-Generic-Minimal.x86_64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-minimal:41-x86_64
buildah rmi registry.fedoraproject.org/fedora-minimal:rawhide
buildah manifest create registry.fedoraproject.org/fedora-minimal:rawhide docker://registry.fedoraproject.org/fedora-minimal:41-aarch64 docker://registry.fedoraproject.org/fedora-minimal:41-ppc64le docker://registry.fedoraproject.org/fedora-minimal:41-s390x docker://registry.fedoraproject.org/fedora-minimal:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora-minimal:rawhide docker://registry.fedoraproject.org/fedora-minimal:rawhide --all
buildah rmi registry.fedoraproject.org/fedora-minimal:41
buildah manifest create registry.fedoraproject.org/fedora-minimal:41 docker://registry.fedoraproject.org/fedora-minimal:41-aarch64 docker://registry.fedoraproject.org/fedora-minimal:41-ppc64le docker://registry.fedoraproject.org/fedora-minimal:41-s390x docker://registry.fedoraproject.org/fedora-minimal:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora-minimal:41 docker://registry.fedoraproject.org/fedora-minimal:41 --all
buildah rmi quay.io/fedora/fedora-minimal:rawhide
buildah manifest create quay.io/fedora/fedora-minimal:rawhide docker://quay.io/fedora/fedora-minimal:41-aarch64 docker://quay.io/fedora/fedora-minimal:41-ppc64le docker://quay.io/fedora/fedora-minimal:41-s390x docker://quay.io/fedora/fedora-minimal:41-x86_64
buildah manifest push quay.io/fedora/fedora-minimal:rawhide docker://quay.io/fedora/fedora-minimal:rawhide --all
buildah rmi quay.io/fedora/fedora-minimal:41
buildah manifest create quay.io/fedora/fedora-minimal:41 docker://quay.io/fedora/fedora-minimal:41-aarch64 docker://quay.io/fedora/fedora-minimal:41-ppc64le docker://quay.io/fedora/fedora-minimal:41-s390x docker://quay.io/fedora/fedora-minimal:41-x86_64
buildah manifest push quay.io/fedora/fedora-minimal:41 docker://quay.io/fedora/fedora-minimal:41 --all
skopeo copy oci-archive:Fedora-Container-Toolbox.aarch64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-aarch64
skopeo copy oci-archive:Fedora-Container-Toolbox.aarch64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-aarch64
skopeo copy oci-archive:Fedora-Container-Toolbox.ppc64le-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Toolbox.ppc64le-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-ppc64le
skopeo copy oci-archive:Fedora-Container-Toolbox.s390x-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-s390x
skopeo copy oci-archive:Fedora-Container-Toolbox.s390x-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-s390x
skopeo copy oci-archive:Fedora-Container-Toolbox.x86_64-Rawhide-20240528.n.0.oci.tar docker://registry.fedoraproject.org/fedora-toolbox:41-x86_64
skopeo copy oci-archive:Fedora-Container-Toolbox.x86_64-Rawhide-20240528.n.0.oci.tar docker://quay.io/fedora/fedora-toolbox:41-x86_64
buildah rmi registry.fedoraproject.org/fedora-toolbox:rawhide
buildah manifest create registry.fedoraproject.org/fedora-toolbox:rawhide docker://registry.fedoraproject.org/fedora-toolbox:41-aarch64 docker://registry.fedoraproject.org/fedora-toolbox:41-ppc64le docker://registry.fedoraproject.org/fedora-toolbox:41-s390x docker://registry.fedoraproject.org/fedora-toolbox:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora-toolbox:rawhide docker://registry.fedoraproject.org/fedora-toolbox:rawhide --all
buildah rmi registry.fedoraproject.org/fedora-toolbox:41
buildah manifest create registry.fedoraproject.org/fedora-toolbox:41 docker://registry.fedoraproject.org/fedora-toolbox:41-aarch64 docker://registry.fedoraproject.org/fedora-toolbox:41-ppc64le docker://registry.fedoraproject.org/fedora-toolbox:41-s390x docker://registry.fedoraproject.org/fedora-toolbox:41-x86_64
buildah manifest push registry.fedoraproject.org/fedora-toolbox:41 docker://registry.fedoraproject.org/fedora-toolbox:41 --all
buildah rmi quay.io/fedora/fedora-toolbox:rawhide
buildah manifest create quay.io/fedora/fedora-toolbox:rawhide docker://quay.io/fedora/fedora-toolbox:41-aarch64 docker://quay.io/fedora/fedora-toolbox:41-ppc64le docker://quay.io/fedora/fedora-toolbox:41-s390x docker://quay.io/fedora/fedora-toolbox:41-x86_64
buildah manifest push quay.io/fedora/fedora-toolbox:rawhide docker://quay.io/fedora/fedora-toolbox:rawhide --all
buildah rmi quay.io/fedora/fedora-toolbox:41
buildah manifest create quay.io/fedora/fedora-toolbox:41 docker://quay.io/fedora/fedora-toolbox:41-aarch64 docker://quay.io/fedora/fedora-toolbox:41-ppc64le docker://quay.io/fedora/fedora-toolbox:41-s390x docker://quay.io/fedora/fedora-toolbox:41-x86_64
buildah manifest push quay.io/fedora/fedora-toolbox:41 docker://quay.io/fedora/fedora-toolbox:41 --all

siosm commented a month ago

Thanks Adam for the work here. I'll try to take a look at this PR tomorrow.

rebased onto 282de09

a month ago

siosm commented a month ago

i did not review the full code changes but the output listed above looks in https://pagure.io/cloud-image-uploader/pull-request/10#comment-203462 good to me.

I think we'll have to give it a try and resolve the issues "live" in the staging environment.

adamwill commented a month ago

I do want to change the fedfind cache disabling approach today (to use an env var, which should be much simpler and less potentially polluting of your home dir), so I'll do one more rebase today.

adamwill commented a month ago

OK, I just rebased this again with the env var approach, which I've released in fedfind 5.3.0 (updates coming soon). This is much simpler than the magic file approach and doesn't have the risk of leaving the change 'live' if the test process fails in an unfortunate way.

jcline commented a month ago

Things look good to me, tests all pass now!