[PATCH 1 of 2 v2] kallithea_config: properly handle Unicode characters in .ini template

[Mads Kiilerich]

On 08/06/2015 10:55 PM, Thomas De Schampheleire wrote:
> On Thu, Aug 6, 2015 at 10:44 PM, Mads Kiilerich <mads at kiilerich.com> wrote:
>> On 08/06/2015 02:53 PM, Thomas De Schampheleire wrote:
>>> # HG changeset patch
>>> # User Thomas De Schampheleire <thomas.de.schampheleire at gmail.com>
>>> # Date 1438854177 -7200
>>> #      Thu Aug 06 11:42:57 2015 +0200
>>> # Node ID 36872d5ea39382ccd6413d2d373d6f899fbd837d
>>> # Parent  88a5fb51d83744befbdb2aaeae63d8c5e735749a
>>> kallithea_config: properly handle Unicode characters in .ini template
>>>
>>> The kallithea_config.py script did not allow Unicode characters in the
>>> configuration file template template.ini.mako. Following error would be
>>> given:
>>>       Traceback (most recent call last):
>>>         File
>>> "/home/tdescham/repo/contrib/kallithea-fixes/kallithea/bin/kallithea_config.py",
>>> line 141, in _run
>>>           f.write(tmpl)
>>>       UnicodeEncodeError: 'ascii' codec can't encode character u'\u2002' in
>>> position 2087: ordinal not in range(128)
>>>
>>> diff --git a/kallithea/bin/kallithea_config.py
>>> b/kallithea/bin/kallithea_config.py
>>> --- a/kallithea/bin/kallithea_config.py
>>> +++ b/kallithea/bin/kallithea_config.py
>>> @@ -132,13 +132,13 @@ def _run(argv):
>>>                tmpl_file = args.template
>>>              with open(tmpl_file, 'rb') as f:
>>> -            tmpl_data = f.read()
>>> +            tmpl_data = f.read().decode('utf-8')
>>
>> It seems a bit unfortunate to hardcode utf-8 here. The user could be using
>> another encoding for their .ini file .
>>
>> Would an update to a later version of the ini file reader "solve" this
>> problem? It seems wrong to work around it here.
> Which ini file reader are you referring to?
> Unicode in the ini file is only a problem when running
> kallithea_config to _generate_ an ini file based on a template with
> unicode. The parsing of ini files with unicode during normal Kallithea
> operation works fine.

Ok, got it. Makes sense. I will push it later.

> We should also make a stance on using either 'paster make-config' _or_
> kallithea_config: having to support both, which generate different
> output, is annoying. IMO the advantage of kallithea_config is that you
> have more control over how to generate the file -- not so with paster
> (at least I couldn't find it).
> Disadvantage of kallithea_config is that it leaves out 'unused' parts:
> the resulting ini file will contain either gunicorn stuff or waitress,
> but not both. This is a bit confusing IMO: it's useful to see _all_
> available settings even if commented. This can probably be fixed in
> kallithea_config itself.

'paster make-config' is a pylons feature that has limited feature set 
but I think it would be weird to _not_ have it. I guess it will go away 
or turn into something else if we change framework.

kallithea_config is more powerful and we can use templating to generate 
config files exactly as we want them. We could change the template to be 
less clever or to keep disabled parts.

I appreciate having config files where I don't have to scroll through 
big sections that not only is documentation or default values but also 
is for areas or features I don't use at all. I like having config files 
that only tell me in what way this system is special. There are many 
ways to skin that cat ...

I will post a script that can generate template and the two ini files 
from the mako template. That was used to create / validate the changes I 
just pushed and will reduce the overhead of maintaining them.

/Mads