[PATCH] tests: hide database setup queries with pytest

[Mads Kiilerich]

On 07/13/2015 09:11 PM, Thomas De Schampheleire wrote:
> On Mon, Jul 13, 2015 at 7:52 PM, Mads Kiilerich <mads at kiilerich.com> wrote:
>> On 07/11/2015 09:06 PM, Thomas De Schampheleire wrote:
>>> # HG changeset patch
>>> # User Thomas De Schampheleire <thomas.de.schampheleire at gmail.com>
>>> # Date 1435831639 -7200
>>> #      Thu Jul 02 12:07:19 2015 +0200
>>> # Node ID a71b7c1c0efa80456ed9e85dcbba9990eaec154f
>>> # Parent  53d68f201e4602d3f2ccfcd27107d2ebea2deea2
>>> tests: hide database setup queries with pytest
>>>
>>> Unlike nosetest, pytest would show the database setup queries at the
>>> beginning of the test run, which don't bring added value.
>>
>> What is the root cause of this? What is nosetest doing that pytest isn't?
>> Are we solving the problem correctly or adding a hack that might bite us
>> later?
>>
>> Would it perhaps be better to change the log level inside test.ini?
> This patch only change the log level for the setup of the database.
> The logging of the rest of the tests is untouched.
>
> I honestly don't know if there are better ways to do this, and why
> pytest is different than nosetest in this matter. However, I want to
> make progress in making pytest the test suite, because now that topic
> has stalled. Even if later there turns out to be a better way to do
> this, we can still change it. I don't think this current patch can
> really bite us hard.
>
>>> Hide them by disabling logging during this time.
>>>
>>> Note that this does not provide an easy method of enabling the logging on
>>> demand (what could be done with the -s switch in nosetest).
>>>
>>> diff --git a/kallithea/tests/conftest.py b/kallithea/tests/conftest.py
>>> --- a/kallithea/tests/conftest.py
>>> +++ b/kallithea/tests/conftest.py
>>> @@ -5,9 +5,12 @@ import pkg_resources
>>>    from paste.deploy import loadapp
>>>    import pylons.test
>>>    from pylons.i18n.translation import _get_translator
>>> -
>>> +import logging
>>>      def pytest_configure():
>>> +
>>> +    logging.disable(logging.CRITICAL)
>>
>> It seems like this is a noop ... except giving improved logging for a while?
>>
>>> +
>>>        path = os.getcwd()
>>>        sys.path.insert(0, path)
>>>        pkg_resources.working_set.add_entry(path)
>>> @@ -26,4 +29,6 @@ def pytest_configure():
>>>        translator = _get_translator(pylons.config.get('lang'))
>>>        pylons.translator._push_object(translator)
>>>    +    logging.disable(logging.NOTSET)
>>
>> This is what actually disable the logging? But it will remain disabled for
>> all tests? I guess that will make it harder to debug test failures?
> No, your understanding is incorrect:
>
> ----------------
> https://docs.python.org/2/library/logging.html
> logging.disable(lvl)
>
> Provides an overriding level lvl for all loggers which takes
> precedence over the logger’s own level. When the need arises to
> temporarily throttle logging output down across the whole application,
> this function can be useful. Its effect is to disable all logging
> calls of severity lvl and below, so that if you call it with a value
> of INFO, then all INFO and DEBUG events would be discarded, whereas
> those of severity WARNING and above would be processed according to
> the logger’s effective level. If logging.disable(logging.NOTSET) is
> called, it effectively removes this overriding level, so that logging
> output again depends on the effective levels of individual loggers.
> ----------------
>
> So, by calling it with logging.CRITICAL, _all_ logging is temporarily
> disabled, until the NOTSET call. The code that does the database setup
> magic is between both calls.
>
> I'm obviously open to other approaches to fixing this problem, but I
> think we should move forward so that we can start benefiting from
> pytest for real by writing tests that use its features.

Right, thanks. My testing was incorrect.

I will push it with more narrow disabling and some comments:

+    # Disable INFO logging of test database creation, restore with NOTSET
+    logging.disable(logging.INFO)
      pylons.test.pylonsapp = loadapp('config:test.ini', relative_to=path)
+    logging.disable(logging.NOTSET)

(and please put imports of standard modules together.)

/Mads