SV: SV: Bug in MySQL code?

[Lars Skjærlund]

Hi Dominik,

That exactly matches our problems.

I tried following your suggestion, setting sqlalchemy.db1.convert_unicode = false. I then restored the database in correct Unicode format.

Kallithea still break, but the error is different:

Module kallithea.controllers.home:2 in index
Module kallithea.lib.auth:800 in __wrapper
>>  return func(*fargs, **fkwargs)
Module kallithea.controllers.home:66 in index
>>  admin=False)
Module kallithea.model.repo:239 in get_repos_as_dict
>>  "desc": desc(repo.description),
Module kallithea.model.repo:210 in desc
>>  return h.urlify_text(desc, truncate=60, stylize=c.visual.stylify_metatags)
Module kallithea.lib.helpers:1263 in urlify_text
>>  return literal(s)
Module markupsafe:74 in __new__
>>  return text_type.__new__(cls, base)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 10: ordinal not in range(128)
CGI Variables
CONTENT_TYPE	'; charset="utf-8"'
CONTEXT_DOCUMENT_ROOT	'/var/www/html'
DOCUMENT_ROOT	'/var/www/html'
GATEWAY_INTERFACE	'CGI/1.1'
HTTP_ACCEPT	'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
HTTP_ACCEPT_ENCODING	'gzip, deflate'
HTTP_ACCEPT_LANGUAGE	'da,en-US;q=0.7,en;q=0.3'
HTTP_CACHE_CONTROL	'max-age=0'
HTTP_CONNECTION	'keep-alive'
HTTP_COOKIE	'kallithea=415a7ec891c25bff957a23aabfed10b1f59bfb56b938e778f79f4c63b2460b4538fb6e0b'
HTTP_DNT	'1'
HTTP_HOST	'las-kallithea.dbc.dk'
HTTP_USER_AGENT	'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0'
PATH_INFO	'/'
PATH_TRANSLATED	'/var/lib/kallithea/kallithea.wsgi/'
REMOTE_ADDR	'192.168.30.1'
REMOTE_PORT	'58030'
REQUEST_METHOD	'GET'
REQUEST_SCHEME	'https'
REQUEST_URI	'/'
SCRIPT_FILENAME	'/var/lib/kallithea/kallithea.wsgi'
SERVER_ADDR	'192.168.30.138'
SERVER_ADMIN	'webmaster at dbc.dk'
SERVER_NAME	'las-kallithea.dbc.dk'
SERVER_PORT	'443'
SERVER_PROTOCOL	'HTTP/1.1'
SERVER_SIGNATURE	'<address>Apache/2.4.10 (Debian) Server at las-kallithea.dbc.dk Port 443</address>\n'
SERVER_SOFTWARE	'Apache/2.4.10 (Debian)'
SSL_TLS_SNI	'las-kallithea.dbc.dk'
WSGI Variables
apache.version	(2, 4, 10)
application	<kallithea.lib.middleware.sessionmiddleware.SecureSessionMiddleware object at 0x7f4a1cac8950>
beaker.get_session	<bound method SecureSessionMiddleware._get_session of <kallithea.lib.middleware.sessionmiddleware.SecureSessionMiddleware object at 0x7f4a1cac8950>>
beaker.session	{'_accessed_time': 1447938481.609899, '_creation_time': 1447938397.118032}
mod_wsgi.application_group	'git.dbc.dk|'
mod_wsgi.callable_object	'application'
mod_wsgi.daemon_connects	'1'
mod_wsgi.daemon_restarts	'0'
mod_wsgi.daemon_start	'1447938481607046'
mod_wsgi.enable_sendfile	'0'
mod_wsgi.handler_script	''
mod_wsgi.input_chunked	'0'
mod_wsgi.listener_host	''
mod_wsgi.listener_port	'443'
mod_wsgi.process_group	'kallithea'
mod_wsgi.queue_start	'1447938481606786'
mod_wsgi.request_handler	'wsgi-script'
mod_wsgi.request_start	'1447938481606617'
mod_wsgi.script_reloading	'1'
mod_wsgi.script_start	'1447938481607112'
mod_wsgi.version	(4, 3, 0)
paste.registry	<paste.registry.Registry object at 0x7f4a1c0af850>
paste.throw_errors	True
pylons.action_method	<bound method HomeController.index of <kallithea.controllers.home.HomeController object at 0x7f4a1c091d50>>
pylons.controller	<kallithea.controllers.home.HomeController object at 0x7f4a1c091d50>
pylons.environ_config	{'session': 'beaker.session', 'cache': 'beaker.cache'}
pylons.log_debug	True
pylons.pylons	<pylons.util.PylonsContext object at 0x7f4a1c0af8d0>
pylons.routes_dict	{'action': u'index', 'controller': u'home'}
routes.route	<routes.route.Route object at 0x7f4a1e256c10>
routes.url	<routes.util.URLGenerator object at 0x7f4a1c0afad0>
webob._parsed_query_vars	(GET([]), '')
wsgi process	'Multi process AND threads (?)'
wsgi.file_wrapper	<type 'mod_wsgi.FileWrapper'>
wsgiorg.routing_args	(<routes.util.URLGenerator object at 0x7f4a1c0afad0>, {'action': u'index', 'controller': u'home'})




Med venlig hilsen
Lars Skjærlund
DevOps
Tlf.: 44 86 77 77
DBC as

www.dbc.dk
las at dbc.dk


-----Oprindelig meddelelse-----
Fra: kallithea-general [mailto:kallithea-general-bounces at sfconservancy.org] På vegne af Dominik Ruf
Sendt: 19. november 2015 13:36
Til: kallithea-general at sfconservancy.org
Emne: Re: SV: Bug in MySQL code?

Hi,

we use MySql as well so took a quick look at this.
As a test I added German Umlaut äöü to a repo description.
Although it is displayed correctly in on the web, in the DB it is not.

I haven't verified it but I bet this is because of sqlalchemy.db1.convert_unicode = true in the production.ini.

I guess this is a sqlalchemy setting that allows you to use Umlauts etc. 
in non unicode tables.

Maybe if you set this to false it works as expected.

-Dominik
_______________________________________________
kallithea-general mailing list
kallithea-general at sfconservancy.org
http://lists.sfconservancy.org/mailman/listinfo/kallithea-general