Making pytest the standard test suite (instead of nose)

[Brianna Laugher]

...And you don't get failures from running
KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 nosetests
kallithea/tests/functional/
? Because I do.

Ran 383 tests in 125.998s
FAILED (SKIP=5, errors=10, failures=19)

I put some print statements into load_environment and it looks to me that
running under pytest, it is respecting those settings.

cheers
Brianna


On 4 June 2015 at 21:20, Thomas De Schampheleire <patrickdepinguin at gmail.com
> wrote:

> Hi Brianna, all,
>
> On Tue, Jun 2, 2015 at 6:34 PM, Thomas De Schampheleire
> <patrickdepinguin at gmail.com> wrote:
> > On June 2, 2015 4:10:19 PM CEST, Brianna Laugher <
> brianna.laugher at gmail.com> wrote:
> >>On 31 May 2015 at 12:57, Thomas De Schampheleire
> >><patrickdepinguin at gmail.com
> >>> wrote:
> >>
> >>> Hi,
> >>>
> >>> I've been using pytest for a while now, and think we should consider
> >>> making it the default test suite instead of nose.
> >>>
> >>> As I see it, there are following points to do:
> >>>
> >>> 1. silence the database setup code by default. Currently, the initial
> >>> step of the tests sets up the database and all commands are passing
> >>by
> >>> on the terminal. This should be silent, just like with nose.
> >>> However, I don't know how to fix this correctly, and I hope Marc,
> >>> Ronny or Brianna could help here.
> >>
> >>
> >>So I downloaded kallithea and had a bit of a poke around. My impression
> >>is
> >>that the database setup is done in kallithea/tests/conftest.py
> >>pytest_configure, specifically loadapp. While under nosetests, more or
> >>less
> >>the same is done at pylons.tests PylonsPlugin.begin. So I think the
> >>Pylons
> >>nose plugin is suppressing all output (and in fact I couldn't make it
> >>deliberately show me the output).
> >
> > IIRC, 'nosetests -s' shows that output.
> >
> >>
> >>Whereas with pytest, I think it is not expecting test setup to be done
> >>in
> >>pytest_configure, and so the options for controlling output have no
> >>effect
> >>on what happens in this function.
> >>
> >>So 1) Maybe pytest should be changed to allow suppression of output
> >>from
> >>pytest_configure
> >>or 2) Maybe this functionality belongs under a different hook function.
> >
> > Any hook is fine for us, as long as it works and we can hide the output
> of the setup code. :)
> >
> >>
> >>With pytest style fixtures I don't think it would be a problem. However
> >>maybe there still would need to be something at this level, I don't
> >>fully
> >>understand the implications of what is written here re nosetest, maybe
> >>the
> >>same problems would occur with pytest.
> >>http://pylons-webframework.readthedocs.org/en/latest/testing.html
> >>
> >
> > I have no idea about this, I guess it needs to be tried to know for sure.
> >
> > Thanks for looking at this!
> >
>
> Another thing that is not yet working properly with pytest is the
> feature that runs the setup code once, and then directly starts
> testing, as mentioned here:
> http://kallithea.readthedocs.org/en/latest/contributing.html#running-tests
>
> The result output of:
> KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test
> kallithea/tests/functional/
> is:
>
> Results (239.58s):
>      383 passed
>       18 failed
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:84:
> AssertionError: no repo vcs_test_git_new in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:175:
> AssertionError: no repo ingroup in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:324:
> AssertionError: no repo ingroup_inherited_git in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:267:
> AssertionError: no repo ingroup in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:124:
> AssertionError: no repo vcs_test_git_newąęł in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/__init__.py:221:
> AssertionError: msg `Error creating repository vcs_test_git_new` not
> found - session has no flash
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:399:
> AssertionError: no repo vcs_test_new_to_delete_git in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:451:
> AssertionError: no repo vcs_test_git_newąęł in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:84:
> AssertionError: no repo vcs_test_hg_new in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:175:
> AssertionError: no repo ingroup in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:324:
> AssertionError: no repo ingroup_inherited_hg in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:267:
> AssertionError: no repo ingroup in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:124:
> AssertionError: no repo vcs_test_hg_newąęł in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/__init__.py:221:
> AssertionError: msg `Error creating repository vcs_test_hg_new` not
> found - session has no flash
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:399:
> AssertionError: no repo vcs_test_new_to_delete_hg in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_admin_repos.py:451:
> AssertionError: no repo vcs_test_hg_newąęł in filesystem
>          -
> /home/tdescham/repo/contrib/kallithea-venv/local/lib/python2.7/site-packages/WebTest-1.4.3-py2.7.egg/webtest/app.py:445:
> IndexError: Body does not contain string 'var data = {"totalRecords":
> 16'
>          -
> /home/tdescham/repo/contrib/kallithea-venv/local/lib/python2.7/site-packages/WebTest-1.4.3-py2.7.egg/webtest/app.py:445:
> IndexError: Body does not contain string '{"totalRecords": 0, "sort":
> null, "startIndex": 0, "dir": "asc", "records": []};'
>
>
> Fixing this sounds like a prerequisite for pytest-as-default-test-runner
> too.
> I have no idea whether this is difficult...
>
> Best regards,
> Thomas
>



-- 
They've just been waiting in a mountain for the right moment:
http://modernthings.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20150604/c6121259/attachment.html>