[PATCH 2 of 6] scripts: docs-headings: distribute over available CPU cores

[Thomas De Schampheleire]

El dom., 30 dic. 2018 a las 11:47, Thomas De Schampheleire
(<patrickdepinguin at gmail.com>) escribió:
>
> El dom., 30 dic. 2018 a las 0:45, Mads Kiilerich
> (<mads at kiilerich.com>) escribió:
> >
> > On 12/29/18 10:50 PM, Thomas De Schampheleire wrote:
> > > # HG changeset patch
> > > # User Thomas De Schampheleire <thomas.de_schampheleire at nokia.com>
> > > # Date 1546110972 -3600
> > > #      Sat Dec 29 20:16:12 2018 +0100
> > > # Node ID a7df630cfe21e5e66c555b9fa88ef2c3930870b1
> > > # Parent  6caed3c13cb8d631430371b8e1141a724c4c4cae
> > > scripts: docs-headings: distribute over available CPU cores
> > >
> > > This script is only relevant for contributors, and the fact that it is quite
> > > slow is normally not a big problem.
> > > However, when running it in iteration on different commits, as preparation
> > > to sending out a series, its slowness becomes annoying.
> >
> >
> > Yeah, the script takes something like 30s to run for me.
>
> I guess you don't have an SSD?
> On my side the original script takes about 8 seconds.
>
> >
> > But most of the time is spent running 'hg diff' once for each file. Just
> > the part of your change that moves it to a simple diff call takes it
> > down to less than one second.
> >
> > The parallelization doesn't seem to save additional time. Perhaps
> > because I don't have many cores. But is it really worth the extra
> > complexity? Then let's assess the parallelization separate from the
> > single hg invocation.
>
> Good catch, I hadn't done any profiling :-s
>
> I see the same: just reducing the 'hg diff' invocations brings my 8
> seconds to 0.5 seconds, and parallelizing doesn't improve it further.
> I'll send a new version.
>

Just one additional note: I wouldn't call this solution complex. It
just moves the logic in a function, and replaces the for loop with
three lines of boilerplate Pool code.
If it had given a measurable improvement, I would have gone for it. Am
using similar things in other code where it does make a difference.

/Thomas