Kallithea internals overview

[Thomas De Schampheleire]

Hi,

On Thu, Jan 29, 2015 at 11:50 AM, Thomas De Schampheleire
<patrickdepinguin at gmail.com> wrote:
...
>>> What would be useful to me (and probably to other new contributors) is
>>> a technical overview of Kallithea's internals.

Since my first attempt, I've been reading documentation of pylons and
mako, which was very enlightening to understanding Kallithea.

http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/
http://docs.makotemplates.org/en/latest/

Below I am updating parts of this overview with the gathered
knowledge. I removed the things that are no longer relevant or
accurate.

Framework:
----------
The pylons framework forms the core of the Kallithea application. It
defines a lot more than I had imagined, including the directory tree
and the set of additional Python packages being used. The concepts
models, views, controllers also stem from this framework.

When creating a pylons application, the skeleton structure is:

├── config
│   ├── deployment.ini_tmpl
│   ├── environment.py
│   ├── __init__.py
│   ├── middleware.py
│   └── routing.py
├── controllers
│   ├── error.py
│   └── __init__.py
├── __init__.py
├── lib
│   ├── app_globals.py
│   ├── base.py
│   ├── helpers.py
│   └── __init__.py
├── model
│   └── __init__.py
├── public
│   ├── bg.png
│   ├── favicon.ico
│   ├── index.html
│   └── pylons-logo.gif
├── templates
├── tests
│   ├── functional
│   │   └── __init__.py
│   ├── __init__.py
│   └── test_models.py
└── websetup.py

Which is precisely what Kallithea has for structure. See
http://docs.pylonsproject.org/projects/pylons-webframework/en/latest/gettingstarted.html#creating-a-pylons-project
for details.

The principle is as follows:
- URL paths are coupled to 'controllers' through the routes packages.
A controller is basically a python class with some methods. Based on
these routes, a specific method of a certain controller will be called
to handle a request.
- The controller may need some info from the db to fulfill the
request. Access to the database is handled through the 'model', a
python representation of the tables in the database (one class per
table).
- The controller then renders the page (the 'view') by using a
template. Template handling is done through the mako package.  These
templates can refer to python variables, and additionally can contain
typical web stuff like CSS and Javascript / jQuery.
- In addition to requests for a complete page, parts of a page can be
updated in a similar way through AJAX calls.

>
> Frontend:
> ---------
>
[..]
>
> Question: what is the reasoning to both use jQuery and YUI? Can't we
> select one framework and do everything in that?
> Answer: according to the wiki 'future' list, YUI should be killed in
> favor of jQuery.
>
> Related question: a bird told me that it may be better to use a
> higher-level framework like AngularJS instead of directly scripting
> stuff either in plain Javascript or jQuery. Has there been any thought
> about this before?

After reading further, I have the impression that AngularJS is not
really necessary as we already have the mako template library, and
most of the rendering of a page should be done at that (server-side)
level rather than doing it in (client-side) Javascript.

>
> On the 'future' list, I see "move code from templates to controllers
> and from controllers to libs or models". Can you clarify this a bit
> more?

I assume the first aspect (templates->controllers) refers to the fact
that the templates are currently containing a lot of stuff that is not
just about displaying the data (the main purpose of the template) and
that most of this logic should really be in controllers.
This has the nice side effect of making things more easy to test.

Not sure what type of code should move from controllers to libs or
models, though...

>
> Question: how is the relation between the REST API (controllers/api)
> and the AJAX calls? Is the API used internally in some way, or only
> provided for external usage?
>
> The codemirror JS library (http://codemirror.net/) is used to display
> code files.
>
> The mergely JS library (http://mergely.com) is used to display diffs
> of code. Mergely is based on codemirror.
>
> fontello is a library/service to create a font of symbols (instead of
> using icons in image files). Did we create our own font or are we
> using an existing one?
> 'font awesome' is a specific symbol font. How does it relate to fontello?
>
> pygments: syntax highlighting
>
> formencode: form validation  (also recommended by Pylons)
>
> excanvas?
>
> mousetrap?
>
> native.history.js?
>
>
> Backend:
> --------
>
> Revision information is obtained from the VCS library (lib/vcs), which
> in case of Mercurial interacts directly with the Mercurial python
> classes, and in case of git uses the dulwich python module.
>
> paste: the deployment tool helping in starting up, configuring, etc. the application
>
> bcrypt: password hashing
>
> celery: distributed task queue  (how is this used?)
>
> whoosh: code indexing/search
>
> Question: what is all this WSGI stuff? If you start Kallithea
> according to the base instructions, it's hosting it's own web
> interface (without WSGI?) How does WSGI work, what are the
> advantages/disadvantages? Related to this, I see people running
> Kallithea under Apache, advantages/disadvantages?
>