From 9ddf02d96d30b4b7d43d4e76ea3889e743fba26a Mon Sep 17 00:00:00 2001 From: Davide Cavalca Date: May 31 2024 22:32:08 +0000 Subject: Add how it's made page and rework deviations page --- diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 80acae3..9598468 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,4 +1,5 @@ * xref:installation.adoc[Installation] * xref:deviations.adoc[Deviations] +* xref:how-its-made.adoc[How it's made] * xref:talks.adoc[Conference talks] * xref:faq.adoc[FAQ] diff --git a/modules/ROOT/pages/deviations.adoc b/modules/ROOT/pages/deviations.adoc index 476c501..d966933 100644 --- a/modules/ROOT/pages/deviations.adoc +++ b/modules/ROOT/pages/deviations.adoc @@ -35,9 +35,9 @@ Fedora Asahi Remix installation images are built using https://osinside.github.i === Installation images are built and hosted outsite of Fedora infrastructure -Because the installation images include additional components that are not part of stock Fedora Linux, they cannot currently be built or hosted on Fedora Infrastructure. We are instead using AWS EC2 instances to perform builds and upload them to AWS S3, triggered by an https://gitlab.com/fedora/sigs/asahi/overseer[AWS Lambda] (which runs daily). We use another https://gitlab.com/fedora/sigs/asahi/manifest-generator[Lambda] to generate the manifest for these https://fedora-asahi-remix.org/builds.html[daily builds] to be consumed by the Asahi Installer. +Because the installation images include additional components that are not part of stock Fedora Linux, they cannot currently be built or hosted on Fedora Infrastructure. We are instead leveraging AWS for this, and more details on the infrastructure deployment are available on our https://docs.fedoraproject.org/en-US/fedora-asahi-remix/how-its-made/#_installation_images[how it's made page]. -We host our https://fedora-asahi-remix.org/[website] on AWS S3, fronted by AWS Cloudfront; we use https://gitlab.com/fedora/sigs/asahi/cdn-invalidation[another Lambda] to handle CDN invalidation. The website is automatically deployed from its https://gitlab.com/fedora/sigs/asahi/website[GitLab repository]; this is also where the manually-maintained https://gitlab.com/fedora/sigs/asahi/website/-/blob/main/installer_data_stable.json[installer manifest] for release images (as opposed to dailies) lives. The Lambdas are also automatically deployed via Gitlab Pipelines using AWS Chalice. +Since Fedora Linux 40 it is possible to https://fedoraproject.org/wiki/Changes/KiwiBuiltCloudImages[build Kiwi images in Koji]; this will enable the future efforts to build stock Fedora Linux images with Apple Silicon support. === No support for legacy X11 desktops @@ -51,14 +51,7 @@ These components include `mesa` (https://pagure.io/fedora-asahi/mesa/[source], h The kernel is also maintained downstream in a https://gitlab.com/fedora-asahi/kernel-asahi[fork] of the main https://gitlab.com/cki-project/kernel-ark[kernel-ark] repository. The kernel is in active development -- while platform enablement for Apple Silicon is https://github.com/AsahiLinux/docs/wiki/Feature-Support[in the process] of being upstreamed, currently a downstream kernel is required for the best experience. -Finally, a number of packages specific to the Remix implementation are also maintained downstream: - -- `asahi-platform-metapackage` (https://pagure.io/fedora-asahi/asahi-platform-metapackage[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides a metapackage that declares all other Asahi platform package dependencies -- `asahi-repos` (https://pagure.io/fedora-asahi/asahi-repos[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides Yum repository definitions for our copr-provided packages -- `calamares-firstboot-configs` (https://pagure.io/fedora-asahi/calamares-firstboot-config[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides the Calamares configuration used for the first-boot installer on the KDE edition -- `fedora-asahi-remix-appstream-metadata` (https://pagure.io/fedora-asahi/fedora-asahi-remix-appstream-metadata[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides the Remix-specific AppStream metadata that is required to support updates between major releases via PackageKit -- `fedora-asahi-remix-release` (https://pagure.io/fedora-asahi/fedora-asahi-remix-release[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides the distribution branding for the Remix -- `fedora-asahi-remix-scripts` (https://pagure.io/fedora-asahi/fedora-asahi-remix-scripts[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides various utility scripts and systemd services used in the Remix +Finally, a number of packages specific to the Remix implementation are also maintained downstream; more details on these are available on our http://localhost:8080/fedora-asahi-remix/how-its-made/#_remix_specific_plumbing[how it's made page]. === Fedora Asahi Remix uses 16K pages diff --git a/modules/ROOT/pages/how-its-made.adoc b/modules/ROOT/pages/how-its-made.adoc new file mode 100644 index 0000000..716d97d --- /dev/null +++ b/modules/ROOT/pages/how-its-made.adoc @@ -0,0 +1,96 @@ += How it's made + +This page attempts to document how Fedora Asahi Remix is put together. Useful prerequisites to read before this: + +- our xref:deviations.adoc[Deviations] page, documenting where the Remix deviates from Fedora Linux and why +- https://github.com/AsahiLinux/docs/wiki/Introduction-to-Apple-Silicon[Introduction to Apple Silicon], which covers some of this platform peculiarities +- https://github.com/AsahiLinux/docs/wiki/Open-OS-Ecosystem-on-Apple-Silicon-Macs[Open OS Ecosystem on Apple Silicon Macs], which explains the boot process and how the OS is laid out + +Throughout this document, components hosted outside of Fedora infrastructure will be marked with a ⚠️. + +== Packages + +The https://fedoraproject.org/wiki/SIGs/Asahi[Fedora Asahi SIG] maintains a number of packages that are required for plaform enablement, integration and implementation. As many of these as possible are maintained in Fedora itself under the https://src.fedoraproject.org/group/asahi-sig[`asahi-sig`] FAS group. This isn't an option for https://docs.fedoraproject.org/en-US/fedora-asahi-remix/deviations/#_downstream_packages_required_for_platform_enablement_are_included[some packages] ⚠️, and those are maintained in https://copr.fedorainfracloud.org[copr] instead, under the https://copr.fedorainfracloud.org/groups/g/asahi/coprs/[`@asahi`] group. + +=== Installation + +Fedora Asahi Remix is installed by the https://src.fedoraproject.org/rpms/asahi-installer[`asahi-installer`], which runs from macOS and guides the user through the installation process. The installation involves resizing partitions, installing a stripped-down standalone copy of macOS and laying the Remix image on disk; the system is then rebooted into recoveryOS, where the the installer second stage takes over and guides the user to adjusting the security settings for the standalone macOS and replacing its kernel with the bundled m1n1 (which comes from the `m1n1-stage1` package), which will later act as a first stage bootloader for the installed system. + +The installer is delivered in `asahi-installer-package`; its build process relies on two prebuilt macOS binary artifacts (Python and libffi), which have received a https://pagure.io/fesco/issue/3212[FESCo exception] to the https://docs.fedoraproject.org/en-US/packaging-guidelines/what-can-be-packaged/#prebuilt-binaries-or-libraries[prebuilt policy]. The https://src.fedoraproject.org/rpms/asahi-installer[`asahi-installer`] source package also provides `python3-asahi_firmware`, which is used by https://src.fedoraproject.org/rpms/asahi-scripts[`asahi-scripts`] for firmware management in userspace. + +=== Boot process ⚠️ + +After the installation the system boots into Linux by default. The boot flow starts with https://src.fedoraproject.org/rpms/m1n1[`m1n1`]; its stage 1, which was placed by the installer, is responsible for finding and executing the stage 2 from the EFI partition. The main difference between stage 1 and stage 2 is that the former is rarely updated (as the process requires running the installer again and going through recoveryOS), while the latter is distributed by Fedora in the `m1n1` binary package and updated together with the distro (via `update-m1n1` in https://src.fedoraproject.org/rpms/asahi-scripts[`asahi-scripts`]). + +Once it's done with platform initialization, `m1n1` will find https://pagure.io/fedora-asahi/uboot-tools[`u-boot`] ⚠️ and pass control to it. U-Boot acts as the third stage bootloader, performing some more platform initialization and providing a minimal preboot environment. In the default flow, U-Boot is setup to provide an emulated UEFI environment, which is used to load GRUB. From here on the boot flow is the standard Fedora Linux one. + +=== Firmware + +Apple Silicon machines rely on a large number of firmware blobs to work. Firmware collection is implemented in https://src.fedoraproject.org/rpms/asahi-installer[`asahi-installer`]. Firmware is then loaded as needed in the initramfs via `dracut-asahi`, which is part of https://src.fedoraproject.org/rpms/asahi-scripts[`asahi-scripts`]. An `asahi-fwupdate` package is also provided (also from https://src.fedoraproject.org/rpms/asahi-scripts[`asahi-scripts`]) to apply firmware updates on the Linux side in case new firmware becomes available. + +The Asahi Linux project has https://github.com/AsahiLinux/docs/wiki/Open-OS-Ecosystem-on-Apple-Silicon-Macs#firmware-provisioning[in-depth documentation] of the firmare provisioning process, which is meant to be standardized between distributions. + +=== Kernel and userspace drivers ⚠️ + +The https://gitlab.com/fedora-asahi/kernel-asahi[kernel package] ⚠️ for Fedora Asahi Remix is maintained as a downstream fork of https://gitlab.com/cki-project/kernel-ark[kernel-ark], including patches from the upstream https://github.com/AsahiLinux/linux[Asahi Linux tree]. Asahi Linux also maintains a https://github.com/AsahiLinux/docs/wiki/Feature-Support[detailed tracker] of the upstreaming status for every component. + +The GPU driver also has a userspace counterpart in the https://docs.mesa3d.org/drivers/asahi.html[asahi driver] in https://pagure.io/fedora-asahi/mesa/[mesa] ⚠️. This is tightly coupled with the AGX driver in the kernel. + +=== Audio + +Apple Silicon machines have a complex speaker setup that requires speaker protection to be safe and a dedicated DSP chain to sound good. This is implemented by https://src.fedoraproject.org/rpms/asahi-audio[`asahi-audio`], which leverages https://src.fedoraproject.org/rpms/rust-bankstown-lv2[`rust-bankstown-lv2`] for bass enhancement and https://src.fedoraproject.org/rpms/rust-speakersafetyd[`rust-speakersafetyd`] for speaker protection, plus https://src.fedoraproject.org/rpms/alsa-ucm-asahi[`alsa-ucm-asahi`] and https://src.fedoraproject.org/rpms/rust-alsa[`rust-alsa`]. PipeWire and WirePlumber have also been enhanced to create the correct virtual audio devices and present them to the user in an understandable way. + +=== Touch Bar + +Some Apple Silicon Macbooks have a https://developer.apple.com/design/human-interface-guidelines/touch-bar[Touch Bar] taking the place of the first row of the keyboard. On Linux this is presented as a regular (albeit odd-sized) display, and it can be driven as such. To make it useful https://src.fedoraproject.org/rpms/rust-tiny-dfr[`rust-tiny-dfr`] renders a set of function keys on it, mimicking what would be available on a physical keyboard. + +=== Media playback and codecs + +Fedora Asahi Remix ships with out of the box support for H.264-encoded content. This is implemented by having https://src.fedoraproject.org/rpms/asahi-installer[`asahi-installer`] download the RPMs and putting them onto the EFI partition; a systemd unit in https://pagure.io/fedora-asahi/fedora-asahi-remix-scripts[`fedora-asahi-remix-scripts`] then installs them on the first boot. + +We also provide https://src.fedoraproject.org/rpms/widevine-installer[`widevine-installer`] to automatically enable Widevine playback by downloading and extracting the necessary bits from a ChromeOS image. + +=== NVram and default boot entry + +Apple Silicon systems store some low-level system configuration settings in NVram. We provide a set of packages to interact with this, but they are not installed by default as there is currently no safe way to enforce a single writer (concurrent writes can be racy and lead to corruption). + +The default boot entry can be changed using https://src.fedoraproject.org/rpms/rust-asahi-bless[`rust-asahi-bless`] (a CLI tool) or https://src.fedoraproject.org/rpms/rust-startup-disk[`rust-startup-disk`] (a GUI). Two experimental tools are also provided to sync Bluetooth (https://src.fedoraproject.org/rpms/rust-asahi-btsync[`rust-asahi-btsync`]) and Wi-Fi (https://src.fedoraproject.org/rpms/rust-asahi-wifisync[`rust-wifisync`]) settings between macOS and Linux. All of these tools are implemented on top of https://src.fedoraproject.org/rpms/rust-apple-nvram[`rust-apple-nvram`] and https://src.fedoraproject.org/rpms/rust-asahi-nvram[`rust-asahi-nvram`]. + +=== Emulation + +We provide the https://src.fedoraproject.org/rpms/box64[`box64`] emulator, which is able to run some x86_64 programs. + +=== Apple ecosystem integration + +We maintain packages for the https://libimobiledevice.org/[libimobiledevice] stack, which implements protocols and tools to communicate with Apple devices. Among other things, this includes https://src.fedoraproject.org/rpms/idevicerestore[`idevicerestore`], which can be used to DFU an Apple Silicon laptop from another Linux system (instead of having to rely on another Mac with Apple Configurator 2). Other components of this stack are https://src.fedoraproject.org/rpms/libimobiledevice[`libimobiledevice`], https://src.fedoraproject.org/rpms/libimobiledevice-glue[`libimobiledevice-glue`], https://src.fedoraproject.org/rpms/libplist[`libplist`], https://src.fedoraproject.org/rpms/libusbmuxd[`libusbmuxd`], and https://src.fedoraproject.org/rpms/usbmuxd[`usbmuxd`]. + +We also maintain a handful of ecosystem-related tools: https://src.fedoraproject.org/rpms/apfs-fuse[`apfs-fuse`] is a work-in-progress read-only userspace driver for APFS filesystems, and https://src.fedoraproject.org/rpms/uxplay[`uxplay`] is an AirPlay2 implementation. + +=== Remix-specific plumbing ⚠️ + +We maintain a number of packages that are specific to the implementation of the Remix: + +- `asahi-platform-metapackage` (https://pagure.io/fedora-asahi/asahi-platform-metapackage[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides a metapackage that declares all other Asahi platform package dependencies +- `asahi-repos` (https://pagure.io/fedora-asahi/asahi-repos[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides Yum repository definitions for our copr-provided packages +- `calamares-firstboot-configs` (https://pagure.io/fedora-asahi/calamares-firstboot-config[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides the Calamares configuration used for the first-boot installer on the KDE edition +- `fedora-asahi-remix-appstream-metadata` (https://pagure.io/fedora-asahi/fedora-asahi-remix-appstream-metadata[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides the Remix-specific AppStream metadata that is required to support updates between major releases via PackageKit +- `fedora-asahi-remix-release` (https://pagure.io/fedora-asahi/fedora-asahi-remix-release[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-branding/[copr]) provides the distribution branding for the Remix; Fedora Asahi Remix has https://pagure.io/Fedora-Council/tickets/issue/432[trademark approval] from the Fedora Council to use its current name and branding (which includes the use of the Fedora logo) +- `fedora-asahi-remix-scripts` (https://pagure.io/fedora-asahi/fedora-asahi-remix-scripts[source], https://copr.fedorainfracloud.org/coprs/g/asahi/fedora-remix-scripts/[copr]) provides various utility scripts and systemd services used in the Remix + +== Infrastructure + +=== Installation images ⚠️ + +Fedora Asahi Remix installation images are built using https://osinside.github.io/kiwi/[Kiwi] from our https://pagure.io/fedora-asahi/kiwi-descriptions[Kiwi descriptions] and are outside of Fedora Infrastructure. + +We are using AWS EC2 instances to perform builds and upload them to AWS S3, triggered by an https://gitlab.com/fedora/sigs/asahi/overseer[AWS Lambda] (which runs daily). We use another https://gitlab.com/fedora/sigs/asahi/manifest-generator[Lambda] to generate the manifest for these https://fedora-asahi-remix.org/builds.html[daily builds] to be consumed by the Asahi Installer. + +We host our https://fedora-asahi-remix.org/[website] on AWS S3, fronted by AWS Cloudfront; we use https://gitlab.com/fedora/sigs/asahi/cdn-invalidation[another Lambda] to handle CDN invalidation. The website is automatically deployed from its https://gitlab.com/fedora/sigs/asahi/website[GitLab repository]; this is also where the manually-maintained https://gitlab.com/fedora/sigs/asahi/website/-/blob/main/installer_data_stable.json[installer manifest] for release images (as opposed to dailies) lives. The Lambdas are also automatically deployed via Gitlab Pipelines using AWS Chalice. + +=== Documentation + +Our https://docs.fedoraproject.org/en-US/fedora-asahi-remix/[documentation site] is generated with https://antora.org[Antora] from its https://pagure.io/fedora-asahi/docs-site[repository], and is https://gitlab.com/fedora/docs/docs-website/docs-fp-o/-/commit/afc54fdc8d8a94ba72ab35f1d65ea535055f7b87[integrated] into the Fedora Docs Website. + +=== Project and bug tracking + +We maintain a https://pagure.io/fedora-asahi/project[project planning tracker] and a https://pagure.io/fedora-asahi/remix-bugs[bugs tracker].