Unstable nosetests wrt login

[Mads Kiilerich]

On 02/22/2015 12:29 PM, Thomas De Schampheleire wrote:
> Hi,
>
> When running nosetests, I sometimes see a few tests failing,
> inconsistently. The errors are:
>
> ======================================================================
> ERROR: test_index_with_anonymous_access_disabled
> (kallithea.tests.functional.test_home.TestHomeController)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "/home/tdescham/repo/contrib/kallithea/kallithea/tests/functional/test_home.py",
> line 42, in test_index_with_anonymous_access_disabled
>      status=302)
>    File "/home/tdescham/repo/contrib/kallithea/dist/v/local/lib/python2.7/site-packages/WebTest-1.4.3-py2.7.egg/webtest/app.py",
> line 759, in get
>      expect_errors=expect_errors)
> ...
> ----------------------------------------------------------------------
> Ran 1507 tests in 700.694s
>
> FAILED (SKIP=5, errors=4)
>
>
> So it looks like an access that is supposed to be blocked is allowed anyhow.
>
> Running the tests again typically 'solves' the problem.
> I tried running the offending tests over and over again, but it does
> not reproduce.
>
> I have seen this issue several times already, but very inconsistently.
>
> Is anyone else seeing this?
> Is it a test problem or rather a Kallithea bug?

I might have seen it a few times but not so often or reproducible that I 
can reproduce it.

The tests are not unit tests and are not isolated and cannot be run in 
parallel ... but we are always running them in the same sequence so it 
should be stable.

-Mads