[PATCH 1 of 2 v2] kallithea_config: properly handle Unicode characters in .ini template

[Thomas De Schampheleire]

On Thu, Aug 6, 2015 at 10:44 PM, Mads Kiilerich <mads at kiilerich.com> wrote:
> On 08/06/2015 02:53 PM, Thomas De Schampheleire wrote:
>>
>> # HG changeset patch
>> # User Thomas De Schampheleire <thomas.de.schampheleire at gmail.com>
>> # Date 1438854177 -7200
>> #      Thu Aug 06 11:42:57 2015 +0200
>> # Node ID 36872d5ea39382ccd6413d2d373d6f899fbd837d
>> # Parent  88a5fb51d83744befbdb2aaeae63d8c5e735749a
>> kallithea_config: properly handle Unicode characters in .ini template
>>
>> The kallithea_config.py script did not allow Unicode characters in the
>> configuration file template template.ini.mako. Following error would be
>> given:
>>      Traceback (most recent call last):
>>        File
>> "/home/tdescham/repo/contrib/kallithea-fixes/kallithea/bin/kallithea_config.py",
>> line 141, in _run
>>          f.write(tmpl)
>>      UnicodeEncodeError: 'ascii' codec can't encode character u'\u2002' in
>> position 2087: ordinal not in range(128)
>>
>> diff --git a/kallithea/bin/kallithea_config.py
>> b/kallithea/bin/kallithea_config.py
>> --- a/kallithea/bin/kallithea_config.py
>> +++ b/kallithea/bin/kallithea_config.py
>> @@ -132,13 +132,13 @@ def _run(argv):
>>               tmpl_file = args.template
>>             with open(tmpl_file, 'rb') as f:
>> -            tmpl_data = f.read()
>> +            tmpl_data = f.read().decode('utf-8')
>
>
> It seems a bit unfortunate to hardcode utf-8 here. The user could be using
> another encoding for their .ini file .
>
> Would an update to a later version of the ini file reader "solve" this
> problem? It seems wrong to work around it here.

Which ini file reader are you referring to?
Unicode in the ini file is only a problem when running
kallithea_config to _generate_ an ini file based on a template with
unicode. The parsing of ini files with unicode during normal Kallithea
operation works fine.

>
> I guess it is less relevant without the error mail subject prefix? Perhaps
> ask upstream here too and wait for a better fix?

This is indeed less urgent without the error_subject_prefix changes,
but I added them because it looked like a general bug. But let's leave
it out for now.

We should also make a stance on using either 'paster make-config' _or_
kallithea_config: having to support both, which generate different
output, is annoying. IMO the advantage of kallithea_config is that you
have more control over how to generate the file -- not so with paster
(at least I couldn't find it).
Disadvantage of kallithea_config is that it leaves out 'unused' parts:
the resulting ini file will contain either gunicorn stuff or waitress,
but not both. This is a bit confusing IMO: it's useful to see _all_
available settings even if commented. This can probably be fixed in
kallithea_config itself.

Are there other relevant items in this discussion?