Current caching

[Dominik Ruf]

Thomas De Schampheleire <patrickdepinguin at gmail.com> schrieb am So., 4.
Feb. 2018 um 13:04 Uhr:

>
>
> On Feb 4, 2018 12:32, "Dominik Ruf" <dominikruf at gmail.com> wrote:
>
>
>
> On Sun, Feb 4, 2018, 12:03 Mads Kiilerich <mads at kiilerich.com> wrote:
>
>> On 02/03/2018 10:12 PM, Dominik Ruf wrote:
>>
>>
>> Mads Kiilerich <mads at kiilerich.com> schrieb am Sa., 3. Feb. 2018 um
>> 19:32 Uhr:
>>
>>> On 02/01/2018 12:50 AM, Dominik Ruf wrote:
>>> > Hi all,
>>> >
>>> > I'm currently looking at the caching Kallithea does. And I'm a
>>> > bit...baffled.
>>>
>>> Yes, it is quite baffling and not very efficient.
>>>
>>> I think it very rarely gives any real benefit - it might even make
>>> things slower. In real world setups with multiple repositories served by
>>> each worker process, cache hits are quite rare.
>>
>> For large git repositories we certainly do need caching.
>>
>>
>> Perhaps. But it is probably a very different kind of caching we need.
>>
> Displaying a changelog page with 100 changesets currently takes 54 seconds
> even with the current caching for the Linux Kernel.
> I'm about to make some changes that will get this down to about 2.5
> seconds. But of course this also depends on caching.
>
>
> This is with a hot cache, ie not just invalidated?
>
Yes with 'hot cache'. Without cache it is about 70 seconds.

>
> Do you see the same for a similar mercurial repository, or is it specific
> to git? What makes the difference?
>
I don't know any hg repositories with 900000+ revisions.

>
> Is this on an SSD disk or spinning disk?
>
SSD

>
> In our old installation we had performance problems with a large
> (mercurial) repo with many heads (review repo where nothing ever gets
> merged), particularly after invalidating of the cache. After upgrading
> Kallithea we were able to bump mercurial (which was supposed to have
> improvements to dealing with many heads), and additionally convert our
> repos to the generaldelta storage format. Now performance is relatively
> fine. Loading a changelog page definitely does not take 54 seconds, with
> both cold and hot cache.
>
Here is what I did so far.
https://kallithea.dominikruf.com/kallithea/kallithea-domruf/changeset/89934e26b1fe4e4c5ef36416e1afbb22299233e7
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20180204/d0c78569/attachment.html>