[PATCH] tests: hide database setup queries with pytest

[Thomas De Schampheleire]

On Mon, Jul 13, 2015 at 7:52 PM, Mads Kiilerich <mads at kiilerich.com> wrote:
> On 07/11/2015 09:06 PM, Thomas De Schampheleire wrote:
>>
>> # HG changeset patch
>> # User Thomas De Schampheleire <thomas.de.schampheleire at gmail.com>
>> # Date 1435831639 -7200
>> #      Thu Jul 02 12:07:19 2015 +0200
>> # Node ID a71b7c1c0efa80456ed9e85dcbba9990eaec154f
>> # Parent  53d68f201e4602d3f2ccfcd27107d2ebea2deea2
>> tests: hide database setup queries with pytest
>>
>> Unlike nosetest, pytest would show the database setup queries at the
>> beginning of the test run, which don't bring added value.
>
>
> What is the root cause of this? What is nosetest doing that pytest isn't?
> Are we solving the problem correctly or adding a hack that might bite us
> later?
>
> Would it perhaps be better to change the log level inside test.ini?

This patch only change the log level for the setup of the database.
The logging of the rest of the tests is untouched.

I honestly don't know if there are better ways to do this, and why
pytest is different than nosetest in this matter. However, I want to
make progress in making pytest the test suite, because now that topic
has stalled. Even if later there turns out to be a better way to do
this, we can still change it. I don't think this current patch can
really bite us hard.

>
>> Hide them by disabling logging during this time.
>>
>> Note that this does not provide an easy method of enabling the logging on
>> demand (what could be done with the -s switch in nosetest).
>>
>> diff --git a/kallithea/tests/conftest.py b/kallithea/tests/conftest.py
>> --- a/kallithea/tests/conftest.py
>> +++ b/kallithea/tests/conftest.py
>> @@ -5,9 +5,12 @@ import pkg_resources
>>   from paste.deploy import loadapp
>>   import pylons.test
>>   from pylons.i18n.translation import _get_translator
>> -
>> +import logging
>>     def pytest_configure():
>> +
>> +    logging.disable(logging.CRITICAL)
>
>
> It seems like this is a noop ... except giving improved logging for a while?
>
>> +
>>       path = os.getcwd()
>>       sys.path.insert(0, path)
>>       pkg_resources.working_set.add_entry(path)
>> @@ -26,4 +29,6 @@ def pytest_configure():
>>       translator = _get_translator(pylons.config.get('lang'))
>>       pylons.translator._push_object(translator)
>>   +    logging.disable(logging.NOTSET)
>
>
> This is what actually disable the logging? But it will remain disabled for
> all tests? I guess that will make it harder to debug test failures?

No, your understanding is incorrect:

----------------
https://docs.python.org/2/library/logging.html
logging.disable(lvl)

Provides an overriding level lvl for all loggers which takes
precedence over the logger’s own level. When the need arises to
temporarily throttle logging output down across the whole application,
this function can be useful. Its effect is to disable all logging
calls of severity lvl and below, so that if you call it with a value
of INFO, then all INFO and DEBUG events would be discarded, whereas
those of severity WARNING and above would be processed according to
the logger’s effective level. If logging.disable(logging.NOTSET) is
called, it effectively removes this overriding level, so that logging
output again depends on the effective levels of individual loggers.
----------------

So, by calling it with logging.CRITICAL, _all_ logging is temporarily
disabled, until the NOTSET call. The code that does the database setup
magic is between both calls.

I'm obviously open to other approaches to fixing this problem, but I
think we should move forward so that we can start benefiting from
pytest for real by writing tests that use its features.

/Thomas