#5700 UnicodeEncodeError while creating an OTPtoken
Closed: Fixed None Opened 8 years ago by alich.

[root@ipabox rpms]# ipa user-add --first=IPAtest1 --last=User1 ipatestuser1
-------------------------
Added user "ipatestuser1"
-------------------------
  User login: ipatestuser1
  First name: IPAtest1
  Last name: User1
  Full name: IPAtest1 User1
  Display name: IPAtest1 User1
  Initials: IU
  Home directory: /home/ipatestuser1
  GECOS: IPAtest1 User1
  Login shell: /bin/sh
  Kerberos principal: ipatestuser1@ALICH.WORK
  Email address: ipatestuser1@alich.work
  UID: 1192000003
  GID: 1192000003
  Password: False
  Member of groups: ipausers
  Kerberos keys available: False
[root@ipabox rpms]# ipa passwd ipatestuser1
New Password: 
Enter New Password again to verify: 
----------------------------------------------
Changed password for "ipatestuser1@ALICH.WORK"
----------------------------------------------
[root@ipabox rpms]# kinit ipatestuser1
Password for ipatestuser1@ALICH.WORK: 
Password expired.  You must change it now.
Enter new password: 
Enter it again: 
[root@ipabox rpms]# kswitch -p admin
[root@ipabox rpms]# ipa otptoken-add --type=totp --owner=ipatestuser1 --desc="New soft token"
------------------
Added OTP token ""
------------------
  Unique ID: e69cce40-c575-400f-a09a-8d636009fa49
  Type: TOTP
  Description: New soft token
  Owner: ipatestuser1
  Key: U8uQGucWNbuWmx0IsMswZoIR1tc=
  Algorithm: sha1
  Digits: 6
  Clock offset: 0
  Clock interval: 30
  URI: otpauth://totp/ipatestuser1@ALICH.WORK:e69cce40-c575-400f-a09a-8d636009fa49?digits=6&secret=KPFZAGXHCY23XFU3DUELBSZQM2BBDVWX&period=30&algorithm=SHA1&issuer=ipatestuser1%40ALICH.WORK


ipa: ERROR: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1345, in run
    sys.exit(api.Backend.cli.run(argv))
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1110, in run
    rv = cmd.output_for_cli(self.api.Backend.textui, result, *args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/plugins/otptoken.py", line 367, in output_for_cli
    qr.print_ascii(tty=True)
  File "/usr/lib/python2.7/site-packages/qrcode/main.py", line 235, in print_ascii
    out.write(codes[pos])
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
ipa: ERROR: an internal error has occurred



[root@ipabox rpms]# kswitch -p ipatestuser1
[root@ipabox rpms]# klist | grep -i "default principal"
Default principal: ipatestuser1@ALICH.WORK
[root@ipabox rpms]# ipa otptoken-add
------------------
Added OTP token ""
------------------
  Unique ID: b947e632-3f64-4b96-98b3-b17369db45f6
  Type: TOTP
  Owner: ipatestuser1
  Manager: ipatestuser1
  Algorithm: sha1
  Digits: 6
  Clock interval: 30
  URI: otpauth://totp/ipatestuser1@ALICH.WORK:b947e632-3f64-4b96-98b3-b17369db45f6?digits=6&secret=2G25NFXTLQUA2MKKKDFNVXVBSKM6GPIA&period=30&algorithm=SHA1&issuer=ipatestuser1%40ALICH.WORK


ipa: ERROR: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1345, in run
    sys.exit(api.Backend.cli.run(argv))
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1110, in run
    rv = cmd.output_for_cli(self.api.Backend.textui, result, *args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/plugins/otptoken.py", line 367, in output_for_cli
    qr.print_ascii(tty=True)
  File "/usr/lib/python2.7/site-packages/qrcode/main.py", line 235, in print_ascii
    out.write(codes[pos])
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
ipa: ERROR: an internal error has occurred


[root@ipabox rpms]# ipa --version
VERSION: 4.3.90.201602241341GITb9c27b6, API_VERSION: 2.163

Seems like a configuration issue (see below):

# export LC_ALL=C
[root@master1 ~]# ipa otptoken-add ttoken2
-------------------------
Added OTP token "ttoken2"
-------------------------
  Unique ID: ttoken2
  Type: TOTP
  Owner: admin
  Manager: admin
  Key: CvkUpvwdt7hfgb7pxWfJHK+sxeY=
  Algorithm: sha1
  Digits: 6
  Clock offset: 0
  Clock interval: 30
  URI: otpauth://totp/admin@IPA.TEST:ttoken2?digits=6&secret=BL4RJJX4DW33QX4BX3U4KZ6JDSX2ZRPG&period=30&algorithm=SHA1&issuer=admin%40IPA.TEST


ipa: ERROR: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1355, in run
    sys.exit(api.Backend.cli.run(argv))
  File "/usr/lib/python2.7/site-packages/ipalib/cli.py", line 1120, in run
    rv = cmd.output_for_cli(self.api.Backend.textui, result, *args, **options)
  File "/usr/lib/python2.7/site-packages/ipalib/plugins/otptoken.py", line 367, in output_for_cli
    qr.print_ascii(tty=True)
  File "/usr/lib/python2.7/site-packages/qrcode/main.py", line 235, in print_ascii
    out.write(codes[pos])
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2588' in position 0: ordinal not in range(128)
ipa: ERROR: an internal error has occurred

A workaround is to always use UTF-8 enabled locale (like "C.UTF-8") when adding tokens in CLI.

More environment info:
Fedora 23
Python 2.7.10
glibc-2.22-10.fc23

$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE=UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Simo is right that if we are going to fail this operation, we need to do so before token creation.

Fortunately, python-qrcode has a terminal output mode which doesn't use UTF-8. However, it only really works if connected to a terminal that is >80 characters in width. Is it possible to detect terminal width?

If yes, I propose the following (pseudocode):

if (!no_qrcode && !utf8 && (!is_terminal || !wide_enough))
    error("QR Code output needs UTF-8 or a wide terminal.");

uri = add_token();

if (!no_qrcode) {
    if (utf8)
        print_utf8_qrcode(uri);
    else if (is_terminal && wide_enough)
        print_term_qrcode(uri);
}

This logic has a few nice properties.

First, we don't create an unusable token when QR code printing fails.

Second, we still work in some non-UTF-8 cases.

Third, the user can always override this detection by simply disabling the qrcode output (no-qrcode=True).

master:

  • 7febe56 otptoken-add: improve the robustness of QR code printing

ipa-4-3:

  • 77e9d31 otptoken-add: improve the robustness of QR code printing

Metadata Update from @alich:
- Issue assigned to mbabinsk
- Issue set to the milestone: FreeIPA 4.3.1

7 years ago

Login to comment on this ticket.

Metadata