horizontal scaling using uwsgi and celery

[Mads Kiilerich]

On 11/10/2016 04:14 PM, Jan Heylen wrote:
>
>     > Is this true for uwsgi and how could I accomplish this with one
>     shared kallithea.ini config file? (is it possible to have uwsgi
>     start a separate celercy e.g.?)
>
>     All wsgi instances should (in my opinion) send to the same queue
>     and should thus use the same .ini settings.
>
>     The celery worker has to be launched and managed somehow. It is
>     possible that uwsgi can do that is good at it - I don't know. But
>     the point with celery is that it is 100% decoupled from the web
>     serving and only communicate through the queue (and the database
>     and the file system). It is thus not obvious that it should be
>     managed through uwsgi.
>
>
> Ok, thx for the info, we will probably have paster still start celery, 
> and use uwsgi to scale Kallithea horizontally it's instances.

Yes, I don't think it is feasible to "run celeryd" without paster. More 
precisely, Kallithea implements a paster command (soon gearbox) that 
runs Kallithea code that uses the celery library. The question is how to 
run this paster command. It can be run from a init.d script, from 
systemd, from some other service wrapper ... or perhaps from uwsgi.

/Mads
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20161110/0dd7c07b/attachment.html>