I am working on bringing support to Fedora for running confidential virtual machines (hereafter called 'CVM') using Intel's TDX technology. At a high level a CVM is just like any traditional VM, but the guest RAM is encrypted, such that no host/hypervisor software can read guest memory, only the physical hardware and guest software can see guest memory in plaintext.

Integrating software to enable the launch of confidential virtual machines is straightforward, however, one of the use cases for CVMs requires a remote user / service to be able to "prove" that a CVM is running in the confidential hardware environment it claims to be - this is called "remote attestation".

For current generations of TDX, the attestation processes builds on top of the Intel SGX technology.
At a high level SGX provides a way for a userspace process to carve off a portion of RAM to be used to execute code in an "enclave". Trusted code is loaded into this enclave and runs some computation tasks, guarantee safe from interference from untrusted code in the regular userspace process or OS kernel. The enclave code exchanges data with its owning process. Code running inside the enclave does NOT link to any of the OS libraries. There are completely separate embedded C & C++ runtime libraries (and some supporting libs for crypto & data exchange) for SGX enclaves to link against, which cannot be substituted out with glibc or any other standard distro library. We're still compiling for x86_64 using regular GCC/binutils x86 targets, but a dedicated runtime (ie gcc -nostdlib -standalone flags are used) as you would have in firmware or kernel or other embedded scenarios. The code to be run inside an SGX enclave will have a signature attached from the vendor of the code, which is validated when the enclave is launched.

It is possible to verify that an application provided enclave is running code signed by its particular vendor's certificate. This involves using some standard / fundamental SGX enclaves built and signed by Intel with a well known certificate, tieing the root of trust to the Intel hardware platform:

To support remote attestation of CVMs using Intel TDX, thus requires that we package much of the SGX software stack in Fedora. Packaging SGX software in Fedora requires packaging the small set of standard enclaves built and signed by Intel - id_enclave.so le.so le.so.orig pce.so pve.so qe.so qe3.so qve.so tdqe.so.

While the enclaves are built into ELF so files, this is just a convenient format to package the code and signature together. These .so files are not loadable by Linux userspace ld-linux.so. The SGX host software will extract the executable code from the ELF file and place it into the enclave memory and verify the signature. The prebuilt binary enclave .so files are available at

• https://download.01.org/intel-sgx/sgx-linux/2.24/prebuilt_ae_2.24.tar.gz
• https://download.01.org/intel-sgx/sgx-dcap/1.21/linux/prebuilt_dcap_1.21.tar.gz

which were respectively created from enclave sources in

• https://github.com/intel/linux-sgx/
• https://github.com/intel/SGXDataCenterAttestationPrimitives

You can see from git that the code for these enclaves is all open source, and comprises a mixture of licenses, all of which are approved for inclusion in Fedora. It is possible to build the enclaves on Fedora, however, it would not be possible to use the result since the binaries would lack the required Intel cryptographic signatures the establish the root of trust back to the hardware.

This creates a packaging guideline compliance problem for us, given the normal requirement to build from source and not ship pre-built binaries in Fedora.

The binaries in the tarballs above were built from corresponding git tags in the repos above using a toolchain and some supporting packages from the NixOS distro that enables byte-perfect reproducibility:

https://github.com/intel/linux-sgx/blob/main/linux/reproducibility/Dockerfile#L24

NB, that dockerfile is derived from Ubuntu, but that's just for bootstrapping the NixOS installation, which is what the enclaves actually build with.

I've considered / evaluated a number of options for SGX enclave packaging in Fedora

1. Declare that the standard Intel signed SGX enclaves get a general exception from the requirement to build from source. The maintainer would promise to run the reproducible build process under NixOS before importing any new releases, to prove they come from the source they claim to come from. This verification would not involve any Fedora toolchain/software though, and would be a manual process outside the context of Koji. The licenses don't require providing the source (no copyleft code goes into the enclaves), but we could still provide the source tarballs if desired, even if not building from them. This is similar conceptually to how we grant pre-built firmware blobs a general exception, with the added verification of the build process, since we are fortunate to actually have the source available.

2. Build the standard SGX enclaves with normal Fedora toolchain and package dependencies. Send the resulting enclaves to Intel to be verified and signed by them. Receive back signed enclaves which we package. This is similar conceptually to how we roundtrip 'shim' through Microsoft for signing. The difference is that with Shim there's value in that each distro vendor is building shim with their own set of certificates embedded. With SGX enclaves each distro would be building the exact same source with no local modifications - only the gcc/binutils would really differ. I've several spent months in discussion with Intel on the possibility of them signing our builds, but this has ultimately been rejected as something they're unwilling to do. It is not scalable for them to audit and verify the full toolchain and build process used across countless Linux distros, and across multiple releases of each distro, in order to sign their respective enclave builds. From an end user POV it would have no functional benefit either, since distros would not be permitted to make any source changes. Effectively the only differences would be what gcc/binutils versions are used. At that point there's no upside to signing something again from their POV, only risk of undermining the root of trust by mistakenly signing a bad build from a distro, a scenario that is difficult to recover from.

3. Add custom RPM builds of binutils, gcc (& a couple of other pkgs), using the exact same versions as present in Intel's NixOS environment. Build the standard SGX enclaves with this alternative toolchain in Fedora. Bundle Intel's existing pre-built & signed enclave binaries in the RPM as additional "sources". Verify that the freshly built enclaves are byte-for-byte identical with Intel's pre-built enclaves from NixOS as part of the RPM build %check step. This is effectively (1), but with all toolchain needed for verification packaged up in Fedora so we're not relying on NixOS / Ubuntu containers, or manual maintainer verification. The verification of signed pre-built enclaves would be done automatically inside Koji as part of the RPM build process. This gives us a much stronger guarantee that we're shipping what we claim to, along with its corresponding source (even though the licenses don't require that in this case).

4. Don't ship any SGX enclaves at all in Fedora. Require anyone who wants to run TDX CVMs to manually download the pre-built enclaves from Intel's GitHub download site. Possibly add a script to Fedora to download this automatically on boot or first use. I include this for completeness, as it is the fallback if all the other options above are rejected. I think it would be an unfriendly solution for Fedora's users though, so I'd like to avoid it.

With all this in mind, I am asking FPC to consider granting packaging exception to allow option (3):

• Pre-built binaries for the standard / fundamental SGX enclaves, signed and distributed by Intel, can be packaged Fedora, with the pre-condition that their payload is verified to be byte-for-byte identical to unsigned binaries fully packaged and built from source in koji using the designated SGX toolchain and runtime for reproducible builds.

The main downside to this exception is that while the source is open, there's no practical way to build your own versions / make changes in context of Fedora, as while you can create builds, you can't sign them and thus can't use them. The only way to get changes is to submit them back upstream to Intel and hope they'll be accepted for the next upstream release.

In case it wasn't clear, all of the above applies only to the SGX enclave code. The SGX software stack includes a lot of other code that is just built and run in normal OS distro context. This host OS software part satisfies Fedora packaging guidelines without issue.

Also there other SGX enclaves app developers have created for various tasks. I am not proposing for these to be covered by the exception. Only the fundamental SGX enclaves created & signed by Intel that bootstrap the platform should be covered by the exception.

BTW, you may have read that SGX is deprecated/removed from Intel CPUs. This only applies to desktop class 'Core' CPUs. Server class 'Xeon' CPUs retain SGX, and TDX is present only in the latter as a result. So this is an relevant technology for the forseeable future.

Sorry for the long read, I wanted to give as much background detail as practical, to make an informed decision, since I expect most people are unlikely to be very familiar with Intel's SGX or TDX technologies.