asahi-installer is the upstream project used to install Fedora Asahi Remix on Apple Silicon systems. It's composed of two parts: a tool that runs on macOS to perform the actual install, and a Python module to extract firmware and put it in the right place. The latter is already packaged in Fedora, and I've put up https://src.fedoraproject.org/rpms/asahi-installer/pull-request/2 to start packaging the former as well (note that PR depends on https://src.fedoraproject.org/rpms/m1n1/pull-request/5 to build the m1n1 stage1 in Fedora as well).
The installer is a Python program that needs to run both on the running macOS system and in recoveryOS (which you can think of as a cut-down version of macOS). To do that, it needs a Python interpreter that can run on macOS and a copy of libffi (which is needed for ctypes to work in recoveryOS, as it's not shipped there). In that PR we're using a pre-built Python from the upstream Python project, and a prebuilt libffi from Homebrew. Python is under the Python license, and libffi is MIT, so both should be fine to redistribute. While we would love to build these from source, that's not feasible in practice, as building macOS programs from Linux isn't something that's doable without a VM, and I don't see that being viable in Koji.
ctypes
We would like to request an exception to https://docs.fedoraproject.org/en-US/packaging-guidelines/what-can-be-packaged/#prebuilt-binaries-or-libraries to be able to ship these two prebuilt binaries (python-3.9.6-macos11.pkg and libffi-3.4.6-macos.tar.gz) in Fedora, so that we can build the installer on Fedora infrastructure and ship it to users, instead of using the prebuilt one from the Asahi Linux project (which is what we're currently doing). We specifically want to do this because it will allow us to ship to users an m1n1 stage1 that is also built in Fedora (the Asahi Linux installer ships its own prebuit one).
python-3.9.6-macos11.pkg
libffi-3.4.6-macos.tar.gz
Unfortunately, we don't have a way to build macOS binaries within Fedora yet, so I think this is probably fine, since from our perspective, macOS is "firmware-ish". These are FOSS and we could do our own builds if we had a macOS cross-toolchain, we just lack one currently.
+1
-1 I think this could benefit from a discussion in a meeting.
Also, I wonder if you could cross compile libffi using clang and musl. If this was possible, you could in theory cross complie python too, but I'm not sure what other dependencies it has other than a libc.
musl is a Linux libc, not a macOS libc.
Metadata Update from @ngompa: - Issue tagged with: meeting
I don't believe clang can target a macOS binary when building on Linux. The only reliable way I know of producing macOS binaries from a Linux system involves spinning up macOS and Xcode in a VM, which is obviously not practical (even setting aside the policy/legal issues involved). Down the road something like Darling could be worth exploring, but I don't believe it's in a usable state yet (which is also why we haven't packaged it for now).
As for what's involved in building Python, https://github.com/python/cpython/blob/main/Mac/README.rst explains how the process works for the macOS release (which is the artifact we're using here).
@dcavalca clang can target macOS (or any other supported OS) binaries building on Linux, but the challenge is getting the sysroot with the correct headers and libraries to be able to build any thing useful.
I am not a FESCo member anymore, but I would probably be against this on principle if I still were. If we allow shipping pre-built stuff for macOS in RPMs today, this can easily spiral down to drop the "everything is built from source" mantra. Also, how do we know the macOS binaries don't contain some proprietary macOS/Xcode bits?
Also, I am curious -- why Python 3.9.6? That seems like an ancient Python version (both 3.9 in general and 3.9.6 specifically -- that bugfix release is 3 years old).
(Perhaps this would also benefit a discussion in a more appropriate place, such as the mailing list or discourse.)
Can you do that with a fully opensource toolchain though? I've seen stuff like https://github.com/tpoechtrager/osxcross but these all rely on the Xcode SDK, which I very strongly doubt we can redistribute (I haven't run https://www.apple.com/legal/sla/docs/xcode.pdf by legal but I doubt it'd pass muster).
Also, how do we know the macOS binaries don't contain some proprietary macOS/Xcode bits?
I haven't personally audited them, but they are coming from official sources and they are redistributable. Generally speaking the usage of proprietary compilers/toolchains does not preclude redistribution (it's often spelled out explicitly in license agreements, as otherwise you wouldn't be able to use the compiler for much of anything).
I suspect it's just historical and could probably be updated, @marcan might know if there's a specific reason for picking this version.
To clarify: we are already shipping these prebuilt artifacts in the Fedora Asahi Remix by virtue of using an upstream build of the Asahi Linux installer. One of the benefits of moving this into Fedora proper would be reducing the number of prebuilt components (as it'd allow us to ship a built-from-source m1n1 stage 1 and installer, leaving only python and libffi as the prebuilt blobs).
Would making our own builds externally (similar to what we do for Fedora Media Writer) and pulling that in as bundled artifacts work?
The Python version is just what was current when the installer was first developed, and we've never had any reason to update it (since it's a single-purpose vendored dependency anyway and it's not like the installer has much of an attack surface, other than fetching stuff via HTTPS, nor do we use any external libraries/dependencies). It should be easy to update to whatever is current these days, but there isn't really any motivation for us to actively track the latest release.
The main issue here is the lack of an unencumbered platform SDK (header files and shared library stubs) for macOS like what the mingw-headers are for W32/W64 targets. The rest of the osxcross toolchain looks acceptably-licensed to me (some use bizarre licenses such as the Apple Public Source License 2.0 used at least by cctools, but the FSF says that that, unlike earlier versions, is a Free Software license), but unfortunately you do not get anywhere without a platform SDK. And the XCode EULA attempts to ban you from even installing any parts of XCode on a non-Apple-branded computer (section 2.2A) – whether that is actually enforcible or not probably depends on the jurisdiction, but I would assume it is enforcible in the USA. It is technically possible to extract the headers without even running macOS, but legally that can be an obstacle. And you also need an Apple Developer account to download XCode legally.
Yes, though there has been slow progress on getting around that by using the pieces that Apple open sourced, reimplementations from GNUStep, etc.
All of that is how Darling is being built today.
This was discussed during the FESCo meeting on May 20th: AGREED: FESCo will permit the inclusion of binaries provided by upstream Python and FFI exclusively for the purposes of loading the installer on MacOS since we have no reasonable way to cross-compile for that platform at this time. (+5, 0, -4)
Metadata Update from @zbyszek: - Issue untagged with: meeting - Issue close_status updated to: Accepted - Issue status updated to: Closed (was: Open)
Are the FFI binaries actually provided by the FFI upstream?
They come from Homebrew: https://formulae.brew.sh/formula/libffi
You can see the logic to download them here: https://github.com/AsahiLinux/asahi-installer/blob/main/build.sh#L61-L82 and the way homebrew builds it here: https://github.com/Homebrew/homebrew-core/blob/a6178d4d3b7f6e00b2c338695f426367f2235791/Formula/lib/libffi.rb
Exactly. Which is not what was approved on the meeting.
The provenance of the binaries was clarified during the meeting, and at least I was aware that the ffi binaries are from homebrew when we were voting. You are right that the text of the approval is not very clear on this, but I think the intent was to approve the inclusion of the homebrew ffi binaries, since this is what was explicitly discussed before.
Log in to comment on this ticket.