This issue tracks the release note for the following F26 Change:
https://fedoraproject.org/wiki/Changes/python3_c.utf-8_locale
If you want to write this release note, then:
Note that sometimes, the contact people are unresponsive. Try to do your best, but if nobody gets back to you, it's not the end of the world.
Once you're done with the above, make sure to either commit the relnote to an appropriate section of the Release Notes book, or, if you're not familiar with Git, DocBook, or whatever else, just add it to this issue as a comment and let pbokoc[1] know that you're done with this one and you'd like the note included. Be sure to do this at least one day before the final release (so the limit is July 10 according to the current schedule). Also make sure to do this even for relnotes that haven't been checked by the change owner.
[0] You can do that by asking the change owner listed on the wiki page; alternatively you can infer it by checking the tracker bug (linked in Wiki) in Bugzilla and looking at its status; see bug comments for details. Ask someone on the mailing list or on IRC if you're not sure. [1] In #fedora-docs on FreeNode (UTC+1 timezone, online mostly during the day on weekdays), or pbokoc@redhat.com if you can't get a hold of me on IRC.
@cstratak and @ncoghlan could you help us distill this change into a release note to guide users?
@bex The upstream context is that this is a backport of Python 3.7's PEP 538 changes: https://docs.python.org/dev/whatsnew/3.7.html#pep-538-legacy-c-locale-coercion
Adapting that What's New entry for Fedora 26 would require something along the lines of: - omitting anything about other platforms - omitting anything about locales other than C.UTF-8 - omitting anything about PEP 11 (since that's CPython's platform support policy
Deleting all of that would give something like:
An ongoing challenge within the Python 3 series has been determining a sensible default strategy for handling the “7-bit ASCII” text encoding assumption currently implied by the use of the default C locale. Starting with Fedora 26, the Fedora system Python includes a backport of Python 3.7's upcoming (locale coercion)[https://docs.python.org/dev/whatsnew/3.7.html#pep-538-legacy-c-locale-coercion] feature, which means the Python 3 stack will automatically coerce the C locale to C.UTF-8 by setting the LC_CTYPE environment variable (if neither it nor LC_ALL are already set) before configuring the process locale. Automatically setting LC_CTYPE this way means that both the core interpreter and locale-aware C extensions (such as readline) will assume the use of UTF-8 as the default text encoding, rather than ASCII. As part of this change, the default error handler for sys.stdin and sys.stdout is now surrogateescape (rather than strict) when using the C.UTF-8 locale. The default error handler for sys.stderr continues to be backslashreplace, regardless of locale. Locale coercion is silent by default, but to assist in debugging potentially locale related integration problems, explicit warnings (emitted directly on stderr) can be requested by setting PYTHONCOERCECLOCALE=warn. This setting will also cause the Python runtime to emit a warning if the legacy C locale remains active when the core interpreter is initialized (e.g. when running Python embedded inside another application which doesn't implement the automatic locale coercion behaviour).
An ongoing challenge within the Python 3 series has been determining a sensible default strategy for handling the “7-bit ASCII” text encoding assumption currently implied by the use of the default C locale.
Starting with Fedora 26, the Fedora system Python includes a backport of Python 3.7's upcoming (locale coercion)[https://docs.python.org/dev/whatsnew/3.7.html#pep-538-legacy-c-locale-coercion] feature, which means the Python 3 stack will automatically coerce the C locale to C.UTF-8 by setting the LC_CTYPE environment variable (if neither it nor LC_ALL are already set) before configuring the process locale. Automatically setting LC_CTYPE this way means that both the core interpreter and locale-aware C extensions (such as readline) will assume the use of UTF-8 as the default text encoding, rather than ASCII.
LC_CTYPE
LC_ALL
readline
As part of this change, the default error handler for sys.stdin and sys.stdout is now surrogateescape (rather than strict) when using the C.UTF-8 locale. The default error handler for sys.stderr continues to be backslashreplace, regardless of locale.
sys.stdin
sys.stdout
surrogateescape
strict
sys.stderr
backslashreplace
Locale coercion is silent by default, but to assist in debugging potentially locale related integration problems, explicit warnings (emitted directly on stderr) can be requested by setting PYTHONCOERCECLOCALE=warn. This setting will also cause the Python runtime to emit a warning if the legacy C locale remains active when the core interpreter is initialized (e.g. when running Python embedded inside another application which doesn't implement the automatic locale coercion behaviour).
stderr
PYTHONCOERCECLOCALE=warn
That may be too long for the release notes, but it covers all the details that I'd expect to be potentially relevant to Fedora users.
I'm not sure if it makes sense to mention it in the release notes, but I also made sure that the upstream PEP covered the prospect of redistributors backporting the change to 3.6: https://www.python.org/dev/peps/pep-0538/#backporting-to-python-3-6-x
Metadata Update from @sclark: - Issue tagged with: stubbed
Metadata Update from @sclark: - Issue assigned to sclark
Thank you, @ncoghlan. I agree with you that your full text is probably too long for the Release Notes but I have created an entry (Commit e6ca892) that uses your first two paragraphs as an introduction to the topic and then links to PEP-538 for the full details.
Metadata Update from @sclark: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.