Kallithea Setup Docs

[David Griffin]

Hi Mads,

I think there may be some misconceptions about uwsgi. uwsgi appears to be
designed as something more akin to a runtime for uwsgi applications, which
then interacts with a compatible webserver. If you were hosting multiple
uwsgi applications, then it would normally be preferable to have each one
be hosted on its own uwsgi server. The webserver then redirects to each of
the uwsgi application servers as required. Therefore I don't agree that
Kallithea could not supply a useful uwsgi server configuration file,
because the configuration file only has to describe how to run Kallithea.
If someone wants to run multiple uwsgi applications, they should most
likely be running multiple uwsgi servers (even if those servers are just
running on different sockets of the same machine) - if they don't, they're
giving up a lot uwsgi's scalable design choices. Similarly, given uwsgi's
nature as something like a runtime, I'd argue that running a uwsgi server
is quite a lot simpler than some of the instructions you have on your setup
page because Kallithea can offload all the interfacing to the actual
webserver to uwsgi, regardless of exactly what that server is.

Therefore, taking into account what you've said, as well as my own research
into the topic, I think my specific proposed change would be to change the
"preferred" method to run Kallithea behind another webserver to be via
uwsgi. This has a bunch of positives for maintaining Kallithea:

1) It follows best practice for deploying uwsgi apps. Your docs have an
example of running behind nginx with http forwarding, which is not an ideal
way of running a uwsgi app.
2) It offloads the integration with web servers to the uwsgi project,
meaning that if something changes upstream, Kallithea doesn't need to
update its instructions / way of doing thigns. For example, for Apache,
mod_uwsgi has fallen out of favour and mod_proxy_uwsgi seems to be
preferred, or at least according to the uwsgi docs. (Note: this also means
that Apache no longer has special instructions for running uwsgi
applications)
3) Similar to the previous point, this would expand support to other web
servers without needing any extra effort in Kallithea.
4) Also similarly to 3, this would simplify the documentation - Kallithea
would only need to document setting up the uwsgi app, and then point users
to the uwsgi docs for integrating the uwsgi app with their preferred
webserver. This would substantially reduce the size of the setup
instructions, being able to remove all sections on specific servers (i.e.
Apache, nginx), and thus reduce the maintenance burden.
5) This would address the potential confusion between uwsgi as an HTTP
server and uwsgi as a uwsgi server by adding a simple note to the HTTP
instructions that if the user wants to run behind an HTTP server, they
should follow the uwsgi server instructions instead.
6) Potential to remove untested/unused templates from the codebase, as
there would be a preferred method to replace them.

Doing this would require mostly changes to the documentation, I think. The
only potential change to the code might be the addition of a uwsgi server
setup template for config-create, which might require a little bit of work,
as well as the potential removal of any untested / unnecessary templates.
If this (or some variant after feedback) seems like a good idea, I'd be
happy to spend some time on it.

One aside: manage-script-name seemed to be necessary in my setup. Some of
the environ variables that Kallithea depends on (If memory serves,
PATH_INFO) were not being set at all, which obviously broke things.
However, while setting manage-script-name fixed the issue, I'm not entirely
sure if the issue was caused by lighttpd not following the uwsgi spec
correctly - this is something I should perhaps test when I can. As far as
uwsgi-socket goes, it seems to be just a synonym for socket.

All the best,
- David

On Mon, 12 Dec 2022 at 15:12, Mads Kiilerich <mads at kiilerich.com> wrote:

> Hi David
>
> The Kallithea docs aim to help getting a very basic setup with the
> essentials. Something that perhaps can be used directly, but mainly
> serve as a starting point for further setup which is outside the scope
> of Kallithea. It is important to keep the configuration examples focused
> without introducing unnecessary concepts, or even worse: mixing up
> different concepts. We must assume that those who want to use advanced
> features (of uWSGI or other very configurable servers like Apache or
> Ngingx) will know how to use these or find the information elsewhere.
>
> The uWSGI template *is* for setting up an uWSGI server. And yes, that
> uWSGI server is serving the HTTP protocol directly, not the uwsgi
> protocol. That seems like a fine setup for Kallithea, per
>
> https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html#can-i-use-uwsgi-s-http-capabilities-in-production
> . I assume you are asking for clarification that the template is serving
> the HTTP protocol and not the uwsgi protocol?
>
> The first lines of the generated uWSGI section mention HTTP basics and
> configure http-socket . uWSGI is mentioned in the documentation, both
> overview and setup, but only very clearly in the context of web/http
> server. That all seems quite clear to me. Mentioning the uwsgi protocol
> doesn't seem helpful when the goal is to help people focus on the
> essentials to get something working, and enumerating things that are
> outside scope is out of scope.
>
> We do for convenience put an [uwsgi] section inside the Kallithea .ini
> where the uwsgi binary with one of the --ini-paste options can pick it
> up. The section name is mandated by uWSGI. In a bigger setup that use
> the uwsgi protocol, there will probably be several layers of servers
> with different configuration, and you will not be using the Kallithea
> .ini file.
>
> The --ini-paste-logged option might be a bit of an odd uWSGI feature
> that doesn't scale to bigger setups. There could *perhaps* be a point in
> giving an example or hinting towards more complex setups with a separate
> uwsgi.ini file without relying heavily on the paste configuration.
>
> I have no doubt that uWSGI can do great things, also with the uwsgi
> protocol. As far as I can see, that can be as simple and trivial as
> using "socket" instead of "http-socket". (I can not find any
> uwsgi-socket option, and manage-script-name only seems relevant when
> using mount points.) But when using uwsgi protocol you need another
> server in front that can serve it as http. That seems like a more
> complex setup, where I doubt even less that one size fits all. I'm sure
> there are many guides and documentation that can help with that. Or is
> there something particularly relevant for Kallithea setups?
>
> It is indeed possible to "mount" several WSGI applications inside most
> HTTP/WSGI servers (or directly in paste), but that is a more complex
> (for example because manage-script-name becomes relevant). New users
> shouldn't have to read and understand that just to get started. But that
> seems unrelated to the uwsgi protocol.
>
> We already have some (old and possibly outdated) mentioning of setups
> with apache and ngingx etc around
> https://kallithea.readthedocs.io/en/default/setup.html#proxy-setups and
> random setup files in
> https://kallithea-scm.org/repos/kallithea/files/stable/init.d .
> Something more elaborate for uWSGI with some examples and qualified
> recommendations could fit in there.
>
> With this context in mind, can you clarify what changes you would propose?
>
> /Mads
>
>
> On 27/11/2022 19:25, David Griffin wrote:
> > Hello all,
> >
> > I just set up Kallithea and there's one area of the docs that could
> > use clarification: emphasizing that setting up Kallithea with the
> > uwsgi template sets it up to use uwsgi as an HTTP server, and not a
> > uwsgi server. The name "uwsgi" is not particularly clear about this,
> > because the uwsgi server application can operate multiple protocols -
> > perhaps it would be better to name it as "uwsgi-http" to make it clear
> > which protocol the configuration is for.
> >
> > Incidentally, Kallithea appears to work great running under uwsgi as a
> > uwsgi server (with the additional configuration option of
> > manage-script-name = true, and setting uwsgi-socket instead of
> > http-socket). This might be a better option for running behind nginx /
> > lighttpd than the proxy_pass method you've got on your docs currently.
> > I can write this up if it's useful.
> >
> > All the best,
> > - David
> >
> > _______________________________________________
> > kallithea-general mailing list
> > kallithea-general at sfconservancy.org
> > https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
>
>
> _______________________________________________
> kallithea-general mailing list
> kallithea-general at sfconservancy.org
> https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20221212/fc0f7b86/attachment-0001.html>