This issue was moved here from https://bugzilla.redhat.com/show_bug.cgi?id=689448

Miroslav Suchý 2011-03-21 10:51:50 EDT

Current newt-python does not accept unicode for text parameters.

E.g. if you modify peanut.py from example:

t = TextboxReflowed(25, "Some text which needs to be wrapped at a good place.")

to

-- coding: utf-8 --

t = TextboxReflowed(25, u"žžž")

you will get:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)

It will work if you pass as parameter
"žžž"
i.e. object string encoded as utf-8 - but that is different from object unicode, see:
http://docs.python.org/howto/unicode.html#the-unicode-type
and:

type(u"žžž")
<type 'unicode'>
type("žžž")
<type 'str'>

And it will work if you pass as parameter:
u"žžž".encode('utf-8')

So this can be work arounded for sure. But it will be nice if newt-python can accept true unicode type.

Comment 1 Miroslav Suchý 2011-03-21 16:01:38 EDT

I spent some time investigating this. And the cause is best described here:
http://docs.python.org/c-api/arg.html

s (string or Unicode) [const char ]...Unicode objects are converted to C strings using the default encoding*...
(emphasis is mine)

And problem with default encoding is best described here:

http://fedoraproject.org/wiki/Features/PythonEncodingUsesSystemLocale
And in mailing thread:
http://thread.gmane.org/gmane.comp.python.devel/109914

So one solution is to put into snackmodule.c:
PyUnicode_SetDefaultEncoding("utf-8");
but that is probably wrong approach.

Second option is to put at the begging each function of snack.py this:
if isinstance(text, unicode):
text = text.encode('utf-8')
for every text parameter.