Kallithea Performance Tuning

[Tim Downey]

Hi Mads,

We're using postgres for our db.  I didn't mention it because it doesn't
seem to be playing any real role in the overall utilization.  We're also
using celery and rabbitmq.  We did have a problem for a while that our
Jenkins CI system was overrunning the Kallithea with all kinds of pull
requests that were polling for any CI changes.  I've switched that over to
use Mercurial hooks to let Jenkins know when something changes.  That seems
to have made a big difference.

I think now the issue may be overall clone & pull speed, but maybe I'm
wrong.  Do any of the settings inside production.ini have any affect on
that, or are we talking raw Mercurial performance once pulls and clones
take over?  Also, would there be any benefit to having my CI systems use
SSH for their activity rather than HTTP?

Sorry for all of the questions, but I'm not quite sure where to look....

Tim

On Mon, Jan 26, 2015 at 4:30 PM, Mads Kiilerich <mads at kiilerich.com> wrote:

> On 01/26/2015 10:01 PM, Tim Downey wrote:
>
>> Hi folks,
>>
>> I'm running a Kallithea instance with about 100 repositories and healthy
>> amount of CI activity across several branches on each repository.  I'm
>> looking for some tips on tuning performance.  I'm running using waitress
>> (at least I think I am) and see what look like several knobs in the
>> production.ini file that can be turned, but I'm not really sure what I'm
>> looking at.
>>
>> See below for my settings.  Can anyone point towards what I can be
>> looking at or changing?  I appear to be CPU bound, not io or mem.  Most of
>> my activity is the result of my CI activity, not users through the web
>> interface.
>>
>
> Normal Python was not designed for multi core CPUs. Neither was Mercurial
> ... even though hgweb should handle multi threading correctly. (I use
> largefiles which do horrible hacks with live monkey patching of modules and
> I would not dare to use that multithreaded.)
>
> I would suggest a setup with multiple worker processes. I don't know how
> feasible that is with the Python web servers. It seems like people then put
> another web server / proxy in front of them - which in my opinion gives a
> much more complicated setup.
>
> I am a relatively happy user of apache + mod_wsgi. It scales nicely with
> multiple single-threaded worker processes but might require more memory. (I
> might however investigate uwsgi.)
>
> I am surprised you don't mention your database. Could it be that you are
> using sqlite and you need something that is less light?
>
> /Mads
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20150126/64fc6f34/attachment.html>