pytest adoption: current status (beginning of May)

[Thomas De Schampheleire]

On Mon, May 4, 2015 at 9:01 PM, Thomas De Schampheleire
<patrickdepinguin at gmail.com> wrote:
> Hi all,
>
> So the official part of 'pytest adoption month' is over, so it's high
> time to look back.
>
> The main achievement is that py.test now can be used as test runner!
> This has been the major accomplishment of the month, thanks to the
> efforts of Marc and Ronny. Thanks to both of you to make this happen.
>
> Visually, the output of py.test feels much cleaner and dense than
> nosetests. However, the initial database setup code used to be silent
> with nosetests, but is now spitting out a lot of data. Is it possible
> to silence this?
>
> Another point: it seems that py.test is currently not in Kallithea's
> setup.py dependency list, meaning that nosetests is installed by
> default but py.test is not. Shouldn't we change this?
>
>
> Unfortunately, due to the amount of unforeseen issues in making this
> happen, there has not been much other work in the test area.
> Marc, Ronny: are you still interested in contributing in the test area
> of Kallithea?
> On https://bitbucket.org/conservancy/kallithea/wiki/TestingImprovements
> I had listed some items, copy pasted-below for convenience:
>
> 1. One aspect I recently inquired about myself is related to fixtures
> and the creation/modification of a repository for the execution of a
> test. See http://lists.sfconservancy.org/pipermail/kallithea-general/2015q1/000232.html
> and Mads' reply:
> http://lists.sfconservancy.org/pipermail/kallithea-general/2015q1/000272.html.
> Fixing this would probably require both help from pytest as internal
> knowledge of Kallithea.
>
> 2. Improving existing tests to write them in a more pytest way
>
> 3. Misc improvements
>
>   a. do not dump all test files in /tmp but rather in a subdirectory
>   b. check why parallel test execution seems to fail (at least on
> nosetests it does)
>   c. investigate/profile why each test takes relatively long to
> complete, while really most tests are making a simple http request and
> check some output.
>
>
> I guess that 3a is an easy fix for py.test experts but would render my
> /tmp directory much more readable.
>
> Item 1 has the highest value for me: I got blocked trying to write
> such test, and it would be great to being able to continue here.
>
> Item 3c and/or 3b could help in case it is possible to speed up the
> overall test execution. Currently, on my laptop, the tests take about
> 900 seconds, which is pretty long IMO.
>



On a related note: Mads reported that the amount of tests collected by
pytest outnumbered the amount of tests collected by nose. I compared
the output of 'nosetests --collect-only' and 'py.test --collect-only',
and conclude that this difference is due to the fact that pytest
counts a parametrized test with n parameter sets as n+1 (including the
unparametrized test), while nose counts it as n.

So according to me, there is no issue and the same tests are run with
either test runner.

Best regards,
Thomas