#7646 plugable.py:491:bootstrap:SystemEncodingError: System encoding must be UTF-8, 'ANSI_X3.4-1968' is not supported.
Opened a year ago by abiagion. Modified 5 months ago

Ticket was cloned from Red Hat Bugzilla (product Red Hat Enterprise Linux 7): Bug 1598044

This is the report we received from ipa-upgrade tests

time:           Tue 03 Jul 2018 07:08:22 PM IST
package:        ipa-server-4.6.4-2.el7
reason:         plugable.py:491:bootstrap:SystemEncodingError: System encoding must be UTF-8, 'ANSI_X3.4-1968' is not supported. Set LC_ALL="C.UTF-8", or LC_ALL="" and LC_CTYPE="C.UTF-8".
cmdline:        /usr/bin/python2 /usr/libexec/ipa/ipa-httpd-kdcproxy
executable:     /usr/libexec/ipa/ipa-httpd-kdcproxy
component:      ipa
uid:            0
username:       root
hostname:       vm-idm-001.aa1239.test
os_release:     Red Hat Enterprise Linux Server release 7.3 (Maipo)
architecture:   x86_64
kernel:         3.10.0-514.39.1.el7.x86_64
abrt_version:   2.1.11

Full Backtrace:
plugable.py:491:bootstrap:SystemEncodingError: System encoding must be UTF-8, 'ANSI_X3.4-1968' is not supported. Set LC_ALL="C.UTF-8", or LC_ALL="" and LC_CTYPE="C.UTF-8".

Traceback (most recent call last):
  File "/usr/libexec/ipa/ipa-httpd-kdcproxy", line 215, in <module>
    sys.exit(main())
  File "/usr/libexec/ipa/ipa-httpd-kdcproxy", line 190, in main
    log=None, debug=debug)
  File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 491, in bootstrap
    raise errors.SystemEncodingError(encoding=fse)
SystemEncodingError: System encoding must be UTF-8, 'ANSI_X3.4-1968' is not supported. Set LC_ALL="C.UTF-8", or LC_ALL="" and LC_CTYPE="C.UTF-8".

Local variables in innermost frame:
fse: 'ANSI_X3.4-1968'
level: 20
overrides: {'confdir': '/etc/ipa', 'debug': False, 'log': None, 'context': 'server'}
parser: <optparse.OptionParser instance at 0x7f87e156d170>
root_logger: <logging.RootLogger object at 0x7f87eb31d690>
handler: <logging.StreamHandler object at 0x7f87e1b2f150>
self: <ipalib.API object at 0x7f87e15c5e10>
match: None
attr: 'xmlrpc_uri'

Additional information:
environ:
:LANG=C
:PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
:KDCPROXY_CONFIG=/etc/ipa/kdcproxy/kdcproxy.conf
:KRB5CCNAME=/var/run/httpd/ipa/krbcache/krb5ccache

Metadata Update from @abiagion:
- Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1598044

a year ago

Seems related to https://pagure.io/freeipa/issue/5887 and commit
e1bd827. See also BZ
https://bugzilla.redhat.com/show_bug.cgi?id=1596161.

If system locale specifies encoding other than utf-8, various programs are breaking
including KDC proxy and certmonger helpers & pre/post scripts.

@cheimes any ideas how we should resolve this? I feel like updating all things that
invoke IPA programs to explicitly set the locale in env is a bit much...

Unfortunately there are only two options to fix the mess:

  • set the env vars
  • update to Python 3.7

See https://www.python.org/dev/peps/pep-0538/ and https://www.python.org/dev/peps/pep-0540/ . Apparently there was a plan to backport the change to Fedora. I don't know if it is still on the table.

Or drop this patch in ipa-4-6 branch given it is is typically used in a python 2 environment.

4.6 is fully Python 3 on Fedora 27.

To remedy the issue, we could either turn the fatal error into a warning or only warn with Python 3. For pure ASCII file names, command line arguments and related values, the ANSI encoding is good enough. I put the error into place, because C encoding breaks non-ASCII cases.

$ python2 -c "import sys; print(sys.argv)" föö
['-c', 'f\xc3\xb6\xc3\xb6']
$ LC_ALL=C python2 -c "import sys; print(sys.argv)" föö
['-c', 'f\xc3\xb6\xc3\xb6']

$ python3 -c "import sys; print(sys.argv)" föö
['-c', 'föö']
$ LC_ALL=C python3 -c "import sys; print(sys.argv)" föö
['-c', 'f\udcc3\udcb6\udcc3\udcb6']

I think demoting to warning is reasonable.

Fedora 27 is EOL and the last release to use 4.6. I don't know about Debian/Ubuntu but I believe they are on 4.7.x as well.

Login to comment on this ticket.

Metadata