#2062 F30 Change: libcrypt.so.1 (compatibility library for POSIX): Let encrypt, encrypt_r, setkey, setkey_r, and fcrypt return ENOSYS instead of performing any real operation
Closed: Accepted 5 years ago by churchyard. Opened 5 years ago by bcotton.

Remove real functionality from encrypt, encrypt_r, setkey, setkey_r, and fcrypt from the libxcrypt.so.1 compatibility library and let those functions set "errno" to "ENOSYS" when invoked.


Since POSIX explicitly states those functions shall set "errno" to "ENOSYS", if the functionality is not available nor implemented, third-party application that start to malfunction are broken by design.

Someone wrote an application, it worked on Fedora. Were they supposed to ship their own version of encrypt() as well in case one day encrypt() suddenly started returning ENOSYS?

I'm trying to figure out what the expected user experience is here:

  • you have a very old Motif-based password-management app that you got a binary of somewhere, you upgrade to Fedora 30
  • you type in your master password, and a dialog pops up saying "encryption does not work on this system"
  • You google around, find this change proposal
  • you downgrade to Fedora 29, get your passwords back, switch to a different password program, then upgrade to Fedora 30.

Security has been improved, arguably, but the user experience was very bad. (And this is the best case - probably the return code wasn't checked at all, and the user got gibberish or the application crashed.)

I think making it hard or impossible to use these functions in new code makes sense, but I don't see the virtue of stubbing the functions out like this. The only argument that I could see is if we think nobody is using the functions, and we get some distro-wide benefit from eliminating the existence of the DES code on the system compared to just making it hard or impossible to use from new code.

Someone wrote an application, it worked on Fedora.

They could not build it since Fedora 28. At this point we started to make the named functions unavailable for the linker. Other distributions shipping glibc >= 2.28 have the same behaviour.

Were they supposed to ship their own version of encrypt() as well in case one day encrypt() suddenly started returning ENOSYS?

Quoting from the IEEE Std 1003.1 (POSIX) description of encrypt and setkey:

The encrypt() function shall fail if:

[ENOSYS]
The functionality is not supported on this implementation.

Quoting from the IEEE Std 1003.1 (POSIX) description of ENOSYS, see chapter 2.3:

[ENOSYS]
Functionality not supported. An attempt was made to use optional
functionality that is not supported in this implementation.

So from my POV that particular application should have been prepared when the underlying system does not support DES-based encryption operation or suddenly drops support for it due to abitrary reasons.

If an application starts to crash because it gets gibberish from the encrypt function and does not properly check errno, I consider this a severe bug in the code as this has always been mandatory by the POSIX definition of that function.

When adding this option to libxcrypt, there was a brief discussion between Alexander Peslyak (aka Solar Designer) and Zack Weinberg (glibc developer, computer security researcher at the Carnegie Mellon University, Pittsburgh, PA) with the conclusion:

Given that any use of either encrypt or setkey in a production program nowadays
is in and of itself a bug, though, I think we can live with the risk.

and

Yeah, IMHO any program that still needs to use DES for whatever reason
should include its own implementation (or use e.g. OpenSSL's implementation).
You already can't link new programs against encrypt/setkey if the system
libcrypt is libxcrypt or glibc >=2.28.

I think making it hard or impossible to use these functions in new code makes sense, but I don't see the virtue of stubbing the functions out like this. The only argument that I could see is if we think nobody is using the functions, and we get some distro-wide benefit from eliminating the existence of the DES code on the system compared to just making it hard or impossible to use from new code.

We went through https://codesearch.debian.net/ looking for uses of encrypt and/or setkey and we did find a single program: cairo-dock appears to be using the DES encryption functions to store passwords for email accounts on disk. (We already have reported this as a bug in cairo-dock.)

I'm trying to figure out what the expected user experience is here

IMHO, users should be alerted in a (maybe forceful) way that they are using applications, which do not store their passwords in a safe way. As long as we do not start to make such applications to be broken in some way, users won't switch to a more reasonable solution.

Since people aren't using encrypt() to any major extent, it doesn't really
matter much here what we do (and we don't have examples to help us figure
out the right path), but i would consider hiding under the posix language
to be poor form, considering that the crypto export restrictions that the
ENOSYS was supposed to handle were lifted in 2000.

I would hope that if we go this route and significant problems show up
before the final release we would revert this change than considering the
problems to be "notification to the user"

Owen

I would hope that if we go this route and significant problems show up before the final release we would revert this change than considering the problems to be "notification to the user"

We can revert this at any time, since we only need to change one configure switch in the libxcrypt spec file.

7 days, 3 pluses. APPROVED (+3, 0, -0)

Metadata Update from @churchyard:
- Issue tagged with: pending announcement

5 years ago

Metadata Update from @churchyard:
- Issue untagged with: pending announcement
- Issue close_status updated to: Accepted
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata