Turbogears2 migration: tests: "cannot import name meta"

[Thomas De Schampheleire]

On Sun, Sep 25, 2016 at 5:06 PM, Thomas De Schampheleire
<patrickdepinguin at gmail.com> wrote:
> On Sep 25, 2016 16:49, "Mads Kiilerich" <mads at kiilerich.com> wrote:
>>
>> On 09/25/2016 04:37 PM, Thomas De Schampheleire wrote:
>>>
>>> On Sun, Sep 25, 2016 at 4:10 PM, Mads Kiilerich <mads at kiilerich.com>
>>> wrote:
>>>>>
>>>>> (It should thus perhaps not be in lib but in model?)
>>>>
>>>> I think the web framework, like tg2 does, may expect helpers to be in
>>>> lib.
>>>> We can tell it about another location, but I don't know if that's worth
>>>> it.
>>>>
>>>>
>>>> Hmm. That suggests that my idea of lib as low level and model as more
>>>> high
>>>> level doesn't work. That model perhaps should be considered an internal
>>>> part
>>>> of lib ... or something like that.
>>>
>>> In a traditional MVC model-view-controller split, 'model' is one of
>>> the three pillars. In that view, I think it shouldn't be 'stuffed
>>> away' in lib.
>>>
>>> I don't know if you should try to put model and lib on the same scale.
>>> I think they are in some way orthogonal, especially things like
>>> lib.{helpers,utils}. For me, the core of 'model' is the definition of
>>> the data model, thus the database tables, and perhaps very related
>>> set-up code.
>>>
>>> 'lib' is currently (but maybe logically) a 'trashbin', a collection of
>>> code that does not really fit elsewhere, but can be considered
>>> 'libraries', i.e. each being standalone and doing some job that helps
>>> someone else. 'utils'/'helpers' is even more a trashbin collection
>>> that did not even fit in logically named files.
>>
>>
>> Does that mean that lib neither is "below" nor "above" model
>
> Yes
>
>> ...but it is ok that they have mutual dependencies? That seems
>> unfortunate.
>
> In any case I think mutual dependencies are bad and should be eliminated.
> But I don't think we should treat 'lib' as one thing and dictate that it
> should only have inward or only have outward dependency arrows. Rather, I'd
> look at the individual entities inside lib, e.g. Annotate.py,
> paster_commands, helpers, vcs, ...
> For each of them, we should decide whether it can depend on model , or if
> model can depend on it (but not both).
>
>>
>> While I agree that the model is special and "very different", I still
>> think we have to decide whether it is below or above lib. Or perhaps lib
>> should be split up in low and high level parts.
>
> Yes, I think this lines up with what I wrote above.
>
>>
>> It also sounds like your investigation and experiments show a different
>> than my initial idea. Great! :-)
>
> So far, all I did was the two patches I sent to the list, and theorize with
> you via email. :-)

With my recent helpers.py cleanup patches, and two other patches I
still have to send, the below circular import problem is solved.

> E     File "/home/madski/kallithea-tg2/kallithea/tests/conftest.py", line 12, in <module>
> E       from kallithea.model.user import UserModel
> E     File "/home/madski/kallithea-tg2/kallithea/model/__init__.py", line 48, in <module>
> E       from kallithea.model import meta
> E     File "/home/madski/kallithea-tg2/kallithea/model/meta.py", line 21, in <module>
> E       from kallithea.lib import caching_query
> E     File "/home/madski/kallithea-tg2/kallithea/lib/__init__.py", line 27, in <module>
> E       from . import helpers
> E     File "/home/madski/kallithea-tg2/kallithea/lib/helpers.py", line 56, in <module>
> E       from kallithea.lib.utils import repo_name_slug, get_custom_lexer
> E     File "/home/madski/kallithea-tg2/kallithea/lib/utils.py", line 50, in <module>
> E       from kallithea.model import meta

However, in comes the next one:

E   ConftestImportFailure: ImportError('cannot import name UserModel',)
E     File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-tg-clean/lib/python2.7/site-packages/_pytest/assertion/rewrite.py",
line 207, in load_module
E       py.builtin.exec_(co, mod.__dict__)
E     File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-tg-clean/lib/python2.7/site-packages/py/_builtin.py",
line 221, in exec_
E       exec2(obj, globals, locals)
E     File "<string>", line 7, in exec2
E     File "/home/tdescham/repo/contrib/kallithea/kallithea-tg-clean/kallithea/tests/conftest.py",
line 12, in <module>
E       from kallithea.model.user import UserModel
E     File "/home/tdescham/repo/contrib/kallithea/kallithea-tg-clean/kallithea/model/user.py",
line 40, in <module>
E       from kallithea.lib.utils2 import safe_str, generate_api_key,
get_current_authuser
E     File "/home/tdescham/repo/contrib/kallithea/kallithea-tg-clean/kallithea/lib/__init__.py",
line 27, in <module>
E       from . import helpers
E     File "/home/tdescham/repo/contrib/kallithea/kallithea-tg-clean/kallithea/lib/helpers.py",
line 781, in <module>
E       from kallithea.lib.auth import HasPermissionAny, \
E     File "/home/tdescham/repo/contrib/kallithea/kallithea-tg-clean/kallithea/lib/auth.py",
line 50, in <module>
E       from kallithea.model.user import UserModel


While the first one could easily be solved, this one is less easy: the
imports from auth of HasPermissionAny and others are directly expected
by templates, so they can use h.HasPermissionAny.
And I cannot say that there is a clear violation of a policy in the
above stack trace: due to the special nature of helpers, it importing
auth is not 'bad'.  And auth depending on model does not seem so wrong
either.

So I can only conclude that the root of all evil is that 'helpers' is
imported whenever you take something from lib, due to the
TG2-introduced 'from . import helpers'.  I already had a patch that
got rid of those, and I think I should just reapply it.

/Thomas