[PATCH 1 of 6] setup.py: allow Paste 3.0.x

[Thomas De Schampheleire]

El sáb., 29 dic. 2018 a las 15:15, Mads Kiilerich
(<mads at kiilerich.com>) escribió:
>
> On 12/29/18 2:52 PM, Thomas De Schampheleire wrote:
> > El sáb., 29 dic. 2018 a las 0:58, Mads Kiilerich
> > (<mads at kiilerich.com>) escribió:
> >>
> >> # HG changeset patch
> >> # User Thomas De Schampheleire <thomas.de_schampheleire at nokia.com>
> >> # Date 1546030612 -3600
> >> #      Fri Dec 28 21:56:52 2018 +0100
> >> # Node ID 43b3bbf90a48adfcfb4837655cffee3a939bf48f
> >> # Parent  445d6875c2eefdd8d10676e761cfe1e82581f78a
> >> setup.py: allow Paste 3.0.x
> >>
> >>
> >> My main concern for whitelisting new major versions of dependencies is whether they are sufficiently backwards compatible. We should try to mitigate that with some amount of testing and review, and preferably a brief mentioning of it in the commit message.
> >>
> >> What can be said about this upgrade?
> > Paste has a new maintainer and moved to github, after years of
> > inactivity (March 2016 -> Oct 2018). There have AFAICS not been
> > incompatible changes.
>
>
> Thanks.
>
>
> >> Note that we also *could* upgrade and support:
> >>
> >> celery==4.2.1
> >> Markdown==3.0.1
> >> Pygments==2.3.1
> >> pytest==4.0.2
> >> pytest-localserver==0.5.0
> >> Routes==2.4.1
> >> Sphinx==1.8.3
> >>
> >> - but I know that some of these break things and are non-trivial ...
> > I generally think that we should try to stay more or less up-to-date
> > with the versions of external components. Staying on old versions will
> > cause problems at some point.
> > But, this desire should be weighed against time/effort factors.
>
>
> Sure. If the upgrade really is easy and low risk, then it is easy to do.
>
>
> > From
> > the list above, I think celery could be more important to investigate
> > than the others.
>
>
> I think they changed the API. Or we just use it in an odd way. If
> upgrading, we should perhaps drop support for < 4.

I agree that we should not try to support 3.x and 4.x at the same time.

I'm not really sure, but I think we use it in an odd way.
At least, I don't immediately see our usage pattern in their docs. But
I admin I haven't read it very thoroughly.

>
>
> > (One specific issue I have, but probably related to how we call
> > celery, rather than a problem inside celery itself, is that if the
> > smtp server responds with "Try again later" then the message is not
> > queued and thus lost).
>
>
> Ouch.

My expectation is that celery knows how to deal with requeueing /
reattempting tasks, but may need some help from the caller to identify
such 'retry' tasks.
But I haven't had/taken time to dig into this problem.

>
> (Also, "celery" is just an implementation detail. And a quite complex
> and fragile one - especially if using rabbitmq. But at our scale, it
> would probably be better to just use a sql database. And most
> immportant: Our commands and documentation should perhaps just talk
> about "worker process".)

Can you elaborate on the following, please?

- "especially if using rabbitmq" -- in which way is rabbitmq a bigger
problem than other brokers?
- what do you mean with 'use a sql database'? To replace what,
rabbitmq? And how exactly? If you meant to replace celery, then I'm
totally lost.

Celery is a 'distributed task queue', while in Kallithea we probably
just need something to asynchronously handle tasks. Unless Kallithea
would be deployed at Github scale, maybe?
Do you suggest a different approach?

>
>
> > Do you have more details about which of these upgrades you know break things?
>
>
> I think Routes also changed alot.

We might get rid of routes if we used more Turbogears2 style routing.
At least, that's feedback we got from Alessandro Molina.

>
>
> > Ideally, our test suite would catch any such breakage: if the tests
> > pass, the upgrade should be fine. If that is not the case, then we
> > need to improve our test suite.
>
>
> Yes. If we assess that an area has good test coverage, then a safe
> upgrade is easy.

The problem is to make a statement about test coverage about external
components. What is our coverage of 'Paste' ?

/Thomas