MIT Kerberos libraries consider RC4 cipher insecure and when system-wide policy disables RC4, they do not allow applications to operate on RC4 at all. At the same time, RC4 is not available in FIPS mode either. Samba 4.14 (and backports to Fedora 33+ and RHEL 8.4+) started to block use of weak crypto as well.

The cipher and hashes using RC4 are essential for two operations in SMB protocol:

• setting up trust between domains in Active Directory environment: RC4 is used to encrypt trusted domain object credentials as an application-specific material in accordance with MS-LSAD specification;

• in a secure channel based on AES session key communicating between SMB domain member and domain controller: as an input to set up a secure channel.

Extend FreeIPA code that establishes trust to Active Directory to be able to run in FIPS mode

In technical terms, the use of RC boils down to

• LSA CreateTrustedDomainEx2. For any AuthInfo type, (0x00000001 (rc4hmac key) and 0x00000002 (plain text password) are the most common ones) the content in AuthenticationInformation is considered an application level payload because it is encrypted with a session key negotiated with the transport. Covered by MS-LSAD section 3.1.4.7.10 and MS-LSAD section 5.1.1

• ServerAuthenticate3 call needs an AES session key which is calculated based on an rc4hmac of a machine account credential according to MS-NRPC section 3.1.4.3.1

Samba 4.13 and backports to 4.12 in Fedora 33+ and RHEL 8.4+ now provide a helper that wraps LSA RPC call CreateTrustedDomainEx2. This helper ensures that in FIPS mode we first check that LSA session key is AES before allowing RC4 use internally in Samba bindings. Thus, it becomes possible to establish trust to Active Directory forest in FIPS mode.

Adopt FreeIPA code to use the helper provided by Samba when it is available.