[PATCH] git: move non-kallithea hooks and execute other hooks

[Mads Kiilerich]

https://kallithea-scm.org/repos/kallithea/changeset/Thanks for the 
reminder of this discussion. Sorry for dropping it.


As spin-off from this discussion and Tim Ooms' patch proposal, we landed 
some changes in the default branch, heading for 0.7 :

https://kallithea-scm.org/repos/kallithea/changeset/6a90b1ebea2c
hooks will now always be written atomically and will never follow symlinks

https://kallithea-scm.org/repos/kallithea/changeset/1089fac66e81
detect hooks that are symlinks as having been changed by the sysadmin, 
and they will thus (by default) not be overwritten

https://kallithea-scm.org/repos/kallithea/changeset/7b809e4a1ea5
use current umask when writing hooks file

https://kallithea-scm.org/repos/kallithea/changeset/5de682ee0b06
introduce cli repo-scan options --install-git-hooks and 
--overwrite-git-hooks

https://kallithea-scm.org/repos/kallithea/changeset/92894fccc615
recommend always running repo-scan to reinstall Git hooks after upgrading

That's a start.



As next step, I propose the changes in 
https://kallithea-scm.org/repos/kallithea/pull-request/306/_/hooks .

Some of it took a slightly different direction than Tim proposed. And 
cleaning up many other related things. Please let us know if you 
disagree with any of it.



Finally, with these things out of the way, the core problem of 
custom/multiple Git hooks is more clear.

The way Kallithea use Git, I see no alternative to Kallithea installing 
the post-receive hook to be able to detect what has been pushed. (But it 
is possible we could come of with another scheme, and thus completely 
avoid this problem?)

I don't like the idea of Kallithea having its own custom way of allowing 
admins to install hooks. If admins want to install extra hooks, they 
should do it in a "standard" way. Tim mentioned a couple of examples of 
how it could be done, but nothing that was ready for use, covered 
relevant concerns such as having a scheme that didn't pick up backup 
files or non-executable files, with a convincing cross platform story 
(covering Windows), or approximated a "standard".

I think the best (and preferred) way forward for having custom and 
multiple hooks for Git would be to have a "standard" dispatcher that can 
be installed in hooks/HOOKNAME . That hook should be a separate project, 
independent of Kallithea, and with a fair chance of being used widely, 
also when not using Kallithea. Ideally, it should be upstreamed in Git.

The scheme could be that it runs all executable hooks/HOOKNAME.d/*.hook 
(or some other scheme, such as hooks/HOOKNAME.*.hook). It should have 
some meaningful handling of stdin, stdout, exit code, and how they 
should be shared/chained when multiple hooks run and might fail. And it 
should support both Linux, Mac, and Windows (where there might be 
cmd/exe/whatever extensions and no exe flag).

When there is a good 3rd party solution to the problem, I will implement 
support for it in Kallithea: detect if a hook from that project is 
installed in hooks/HOOKNAME, and then write the Kallithea hook to 
something like hooks/HOOKNAME.d/kallithea.hook instead.

I suggest you team up to create that project. We will support you, but 
Kallithea would not be a good owner of it.

Does that make sense?


/Mads



On 1/8/21 11:50 PM, Lance Edgar wrote:
> I'm fairly new to Kallithea but also encountered this problem with my 
> setup.  I'm not clear where things are at with this discussion now, 
> but it seems unresolved so wanted to share my perspective.
>
> I need the same thing others have mentioned:  to effectively have 
> multiple post-receive hooks for certain git repos.  Possible solutions 
> appear to be:
>
> A) Kallithea hook gets built-in support for running additional hooks, 
> discovered by globbing
>
> B) admin must create a Kallithea extension which somehow does that 
> same thing
>
> C) admin installs custom hook script which can run the Kallithea hook, 
> plus additional (per glob)
>
> Thus far I've gone with option C, because option A doesn't exist and 
> option B seems wrong to me.  The only thing I don't really like about 
> C is that Kallithea will no longer update its own hook script.  If I'm 
> being honest I think option A should exist and if so I'd use it.
>
>
> On 11/4/20 3:14 AM, Tim Ooms wrote:
>> On 24/10/2020 13:05, Mads Kiilerich wrote:
>>> On 10/23/20 9:30 AM, Tim Ooms wrote:
>>>>> It might have rough edges to bypass Kallithea when accessing 
>>>>> Kallithea-managed repos. It is perhaps better to go with the 
>>>>> assumption that Kallithea fully owns the repositories it manage.
>>>>
>>>> I tried to not bypass Kallithea by letting it execute the other 
>>>> hooks. I know the mercurial hooks are more integrated, but as the 
>>>> git hooks don't work the same way I think this is the next best 
>>>> thing. Although there may be better ways, I'm not sure.
>>>
>>>
>>> If you only access the repos through Kallithea, I think it would be 
>>> better to run your hooks as Kallithea extensions. I'm not sure 
>>> that's possible right now, but we can perhaps fix that.
>
>
> It seems to me that the patch submitted by Tim Ooms will solve for 
> option A.  The argument here appears to be that keeping things 
> "inside" of Kallithea is the ideal goal, but in my mind that's exactly 
> what the patch does.
>
> I think most folks who encounter this problem, will already have a 
> script which is perfectly suitable for use as post-receive hook. Their 
> problem is how to get Kallithea to invoke it.  Trying to shoehorn 
> their existing script into a Kallithea extension seems like overkill.
>
> So what Kallithea needs is some way to invoke these scripts, and the 
> patch does that fine AFAICT.  Kallithea has no reason to assume that 
> any extra hook scripts will work properly etc. but I don't think that 
> is enough to "void the warranty" in general, it should just run them 
> in good faith.  Presumably the admin installed them so consequences 
> will fall to that person.
>
>
>>>> Can you say more about your use case? What other hooks are 
>>>> installed? What precautions do they use to avoid clobbering the 
>>>> Kallithea hooks?
>>>>
>>>> Use cases I think of:
>>>> post:
>>>> * trigger CI build (like jenkins)
>>>> * notify bug tracker (like the bug genie)
>>>> pre:
>>>> * check for commit message
>>>> * verify syntax/codestyle
>>>> Other examples: 
>>>> https://scriptrunner.adaptavist.com/stash/3.0.14-beta1/docs/pre_receive_hooks/ 
>>>>
>>>> These hooks are really independent and don't touch the repository. 
>>>> If they fail, it has no effect on the other hooks. A hook can mess 
>>>> up if it just hangs and does not return. But you will always have 
>>>> this if people are using custom hooks, even in mercurial.
>>>
>>>
>>> Thanks. Some good examples of "hooks" into the default process.
>>>
>>> I understand you install these hooks yourself - it is not some 3rd 
>>> party system doing it. Your purpose with installing the hooks is to 
>>> customize the Kallithea process.
>>>
>>> In that case, would it perhaps be better to hook into Kallithea 
>>> "extensions" instead of hooking in as Git hooks? If we don't have 
>>> the right kind of extension hooks, perhaps we should introduce that?
>>
>> If installing an extension is as easy as installing the git hook 
>> (meaning: simply put the provided hook code somewhere) this can 
>> indeed be a good solution (together with something to allow custom 
>> hooks).
>
>
> My guess is that the use cases above apply to nearly all who encounter 
> this problem.  And although can't be sure, I would expect this to be a 
> somewhat common problem.  (Anyone who is moving to Kallithea but 
> already had hooks in their previous git repos...)
>
> If Kallithea has built-in support then it potentially removes a lot of 
> frustration.  Not clear to me why requiring each admin to make an 
> extension is the "better" way if it is a common problem.
>
>
>>>> It must be a general challenge with Git to fit multiple hooks into 
>>>> the single hook slot. Is there any prior art for solving / working 
>>>> around that problem?
>>>>
>>>> Yes, there is:
>>>> * solution using "pee": 
>>>> https://git.seveas.net/using-multiple-post-receive-hooks.html
>>>
>>>
>>> Nice. But depends on perl. It would be nice to have it as Python. 
>>> But I guess Git already depends on perl ...
>>>
>>>
>>>> * using bash built-ins: 
>>>> https://serverfault.com/questions/909153/how-to-add-multiple-post-receive-hooks-to-a-git-repository 
>>>>
>>>
>>>
>>> It seems like the hooks can have multiple lines on stdin. This batch 
>>> script will invoke the hooks multiple times in that case. That might 
>>> be unfortunate. And it also doesn't detect failures.
>>>
>>> And unfortunately, it seems like they both just describe how it 
>>> *can* be done. It is not a standard way of doing it.
>
>
> I was also surprised at first, to find that this problem even 
> existed.  Why doesn't git support multiple post-receive hooks 
> natively?  I think I understand more now, why they would want to just 
> stay out of that game.  Sort of unfortunate that Kallithea must bear 
> the brunt of that decision here.
>
>
>>>>> It would be nice if there was a "standard" dispatcher (perhaps a 
>>>>> simple shell script) that could be installed in The Hook Location 
>>>>> - something that picked up multiple hook files. Instead of having 
>>>>> our hook call others, it would perhaps be better to install a 
>>>>> generic-ish hook and let it dispatch to both our hook and the 
>>>>> existing one.
>>>>
>>>> I would indeed like a shell solution (I used the bash solution till 
>>>> now, but till now that causes problems when doing an update), but 
>>>> that ain't portable and I think Kallithea can run on Windows. It 
>>>> can have advantages to let Kallithea run the other hooks: they can 
>>>> be managed (in a later stage) by Kallithea, can be logged by 
>>>> Kallithea and admins can be notified if a hook fails to run.
>>>> Also, running directly the Kallithea hook, Kallithea -being the one 
>>>> managing the repository- is sure that its hook is correctly called 
>>>> and it's shebang is correctly set and executable.
>>>>
>>>> This can also be done using something generic-ish (and I like the 
>>>> idea), but you add another layer that can fail and another file to 
>>>> keep up to date like the existing hook.
>>>
>>>
>>> As a step in the right direction, we can perhaps let Kallithea 
>>> create/overwrite post-receive.d/kallithea.hook *if* post-receive 
>>> already exists *and* isn't a kallithea hook, *and* if post-receive.d 
>>> exists.
>>>
>>> That will leave it to you to install the dispatcher when installing 
>>> your hooks.
>>
>> This would indeed be a step in the right direction. The admin can 
>> always search in the docs how to create the dispatcher if she/he 
>> wants to keep a single custom hook.
>> The globbing with pre-&postfix or the subdirectory (cleaner, but can 
>> break the script) both have there (dis)advantages.
>
>
> +1 for this.  If option C is used (as it is for me) then currently 
> Kallithea will not update its own script because I've moved it. So 
> would love to see this land even if option A does too.
>
> Thanks for reading!
>
> Lance
>
> _______________________________________________
> kallithea-general mailing list
> kallithea-general at sfconservancy.org
> https://lists.sfconservancy.org/mailman/listinfo/kallithea-general