Issue #340: 0.4.0: dispatch.wsgi terminates due to LC_ALL or LANG encoding. (conservancy/kallithea)

[Edmund Wong]

New issue 340: 0.4.0: dispatch.wsgi terminates due to LC_ALL or LANG encoding.
https://bitbucket.org/conservancy/kallithea/issues/340/040-dispatchwsgi-terminates-due-to-lc_all

Edmund Wong:

I upgraded 0.3.x to 0.4.0 and came across a LC_ALL/LANG issue.

`
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.696 DEBUG [tg.appwrappers.caching] Caching enabled: True -> {'data_dir': '/var/www/kali/srv/data/cache/data', 'cache_regions': {'long_term': {'expire': 36000, 'data_dir': '/var/www/kali/srv/data/cache/data', 'type': 'memory', 'enabled': True, 'lock_dir': '/var/www/kali/srv/data/cache/lock', 'key_length': 256}, 'sql_cache_short': {'expire': 10, 'data_dir': '/var/www/kali/srv/data/cache/data', 'type': 'memory', 'enabled': True, 'lock_dir': '/var/www/kali/srv/data/cache/lock', 'key_length': 256}, 'short_term': {'expire': 60, 'data_dir': '/var/www/kali/srv/data/cache/data', 'type': 'memory', 'enabled': True, 'lock_dir': '/var/www/kali/srv/data/cache/lock', 'key_length': 256}}, 'expire': None, 'log_file': None, 'type': 'memory', 'lock_dir': '/var/www/kali/srv/data/cache/lock'}
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.696 DEBUG [tg.appwrappers.mingflush] MingSessionFlush enabled: False -> {'autoflush': False}
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.696 DEBUG [tg.appwrappers.transaction_manager] TransactionManager enabled: False -> {'attempts': 1, 'enabled': False, 'commit_veto': None}
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.696 DEBUG [tg.appwrappers.errorpage] ErrorPageApplicationWrapper enabled: True -> {'content_types': ['text/html', None], 'handle_exceptions': True, 'path': '/error/document', 'status_codes': [400, 401, 403, 404], 'enabled': True}
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.697 ERROR [kallithea.config.app_cfg] Cannot encode Unicode paths to file system encoding 'ANSI_X3.4-1968'
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.697 ERROR [kallithea.config.app_cfg] Note: Environment variable LANG is 'en_US.UTF-8' - perhaps change it to some other value from 'locale -a', like 'C.UTF-8' or 'en_US.UTF-8'
[Mon Apr 01 09:58:06 2019] [error] 2019-04-01 09:58:06.697 ERROR [kallithea.config.app_cfg] Terminating ...

`

Looking at the code in app_cfg.py, I noticed it's testing out `u\xe9'.encode(sys.getfilesystemencoding())` 

But it ends up failing and Terminating.

I go into python and get the following results:

`
import sys

print(sys.getfilesystemencoding())
 
`

prints out 'UTF-8'.

So I copy part of the code form setup-configuration(app) as follows:

`
import os
import sys
 try:
     u'\xe9'.encode(sys.getfilesystemencoding())
 except UnicodeEncodeError:
     print("Cannot encode Unicode paths to file system encoding %r", sys.getfilesystemencoding())
     for var in ['LC_CTYPE', 'LC_ALL', 'LANG']:
         if var in os.environ:
             val = os.environ[var]
             print("%s is %r" % (var, val))
             break
         else:
             print("No locale setting found...")
     print("Terminating...")
`

prints : '\xc3\xa9'

so I add the following before the try: block:

`
print(sys.getfilesystemencoding())
`

and it prints :
`
sys : 'ANSI_X3.4-1968'

`

So I add:

os.environ["LC_ALL"] = "en_US.UTF-8" 
os.environ["LANG"] = "en_US.UTF-8"

to the dispatch.wsgi.

But I still get the same ANSI_X3.4-1968 result.

These are the settings I have in my.ini:

i18n.enabled = false
lang =

I think I might have screwed up the upgrade... though I'm not sure where as I followed the upgrade instructions..  (though I hazard a guess I missed some points somewhere...)