Issue #325: Upgrade to SHA hashing (conservancy/kallithea)

Wed Jun 27 05:29:34 UTC 2018

New issue 325: Upgrade to SHA hashing
https://bitbucket.org/conservancy/kallithea/issues/325/upgrade-to-sha-hashing

James Shewey:

The [FIPS 140-2](https://en.wikipedia.org/wiki/FIPS_140-2) standard, published by NIST [disables](https://access.redhat.com/solutions/137833) support in the kernel and in OpenSSL. Basically, this breaks md5 hashing support through OpenSSL libraries. Thus, if a system sets the kernel flag fips=1, it breaks Kallithea. The service will start, but upon accessing the page, the stack trace below is recieved:

```
#!python

2018-06-26 18:47:59.806 ERROR [waitress] Exception when serving /favicon.ico

Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/channel.py", line 337, in service

    task.service()

  File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 173, in service

    self.execute()

  File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 392, in execute

    app_iter = self.channel.server.application(env, start_response)

  File "/usr/lib/python2.7/site-packages/paste/gzipper.py", line 38, in __call__

    response.gzip_start_response)

  File "/usr/lib/python2.7/site-packages/paste/cascade.py", line 130, in __call__

    return self.apps[-1](environ, start_response)

  File "/usr/lib/python2.7/site-packages/paste/registry.py", line 379, in __call__

    app_iter = self.application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/wrapper.py", line 43, in __call__

    return self.application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 312, in __call__

    return self._handle_request(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/simplegit.py", line 68, in _handle_request

    return self.application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 312, in __call__

    return self._handle_request(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/simplehg.py", line 73, in _handle_request

    return self.application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/pylons/middleware.py", line 177, in __call__

    self.app, new_environ, catch_exc_info=True)

  File "/usr/lib/python2.7/site-packages/pylons/util.py", line 50, in call_wsgi_application

    app_iter = application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/weberror/errormiddleware.py", line 156, in __call__

    return self.application(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/sessionmiddleware.py", line 62, in __call__

    return self.wrap_app(environ, session_start_response)

  File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__

    response = self.app(environ, start_response)

  File "/usr/lib/python2.7/site-packages/pylons/wsgiapp.py", line 103, in __call__

   response = self.dispatch(controller, environ, start_response)

  File "/usr/lib/python2.7/site-packages/pylons/wsgiapp.py", line 313, in dispatch

    return controller(environ, start_response)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 439, in __call__

    session.get('authuser'),

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 424, in _determine_auth_user

    return AuthUser()

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/auth.py", line 488, in __init__

    self.anonymous_user = User.get_default_user(cache=True)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 649, in get_default_user

    user = User.get_by_username(User.DEFAULT_USER, cache=cache)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 559, in get_by_username

    "get_user_%s" % _hash_key(username)

  File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 68, in <lambda>

    _hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips
```

The basic fix here is to switch to using a sha1 or better hash instead of an MD5 hash. A cursory grep of the source code results in the following files that would need to be swapped to a stronger hash:

```
#!python
./kallithea/lib/celerylib/__init__.py:from hashlib import md5

./kallithea/lib/celerylib/__init__.py:        md5(func_name + '-' + '-'.join(map(safe_str, params))).hexdigest()

./kallithea/tests/other/test_libs.py:        _md5 = lambda s: hashlib.md5(s).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_0_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_0_1.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_5_2.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_2_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_6_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_7_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_0_2.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_8_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_4_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_2_3.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_3_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_2_1_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/lib/dbmigrate/schema/db_1_5_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/model/db.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()

./kallithea/model/db.py:    DEFAULT_GRAVATAR_URL = 'https://secure.gravatar.com/avatar/{md5email}?d=identicon&s={size}'

./kallithea/lib/helpers.py:    return 'C-%s-%s' % (short_id(raw_id), hashlib.md5(safe_str(path)).hexdigest()[:12])

./kallithea/lib/helpers.py:        _md5 = lambda s: hashlib.md5(s).hexdigest()

./kallithea/lib/helpers.py:                   .replace('{md5email}', _md5(safe_str(email_address).lower())) \

./kallithea/lib/markup_renderer.py:        from hashlib import md5

./kallithea/lib/markup_renderer.py:            digest = md5(matchobj.group(0)).hexdigest()

```
Upgrading this hashing will allow companies who undergo NIST auditing and require NIST compliance to use Kallithea, however I could see that this might cause an issue for an upgrade to newer versions and hashes might need to be converted or regenerated somehow.