Docker update (looking for testers)

[Thomas De Schampheleire]

2017-11-14 22:55 GMT+01:00 Dominik Ruf <dominikruf at gmail.com>:
>
>
> Thomas De Schampheleire <patrickdepinguin at gmail.com> schrieb am Di., 14.
> Nov. 2017 um 21:32 Uhr:
>>
>> Hi Dominik,
>>
>> 2017-09-20 20:52 GMT+02:00 Dominik Ruf <dominikruf at gmail.com>:
>> > Hi all,
>> >
>> > like I mentioned before, I'd like to add a Dockerfile and
>> > docker-compose.yml
>> > to make it really easy to get started with kallithea.
>> > I did some more work on my docker setup and it now also includes ssh
>> > support.
>> > And like before it generates the css file from less.
>> > So since these 2 things are not merged yet, I'd like to wait before
>> > creating
>> > a PR.
>> >
>> > But I wanted to ask if some people would volunteer to already test it.
>> > I'd like to hear from others if it helps them setting up a Full Stack
>> > installation.
>> > And if there are unclear things about how to use it.
>> >
>> > I upload my images to https://hub.docker.com/r/domruf/kallithea-domruf/
>> > and you can find the docker-compose.yml at
>> >
>> > https://kallithea.dominikruf.com/kallithea/kallithea-domruf/raw/26ccf792b435/docker/docker-compose.yml
>> >
>>
>> Sorry, I kind of forgot about this mail.
>>
>> I have recently deployed a kallithea instance with docker based on
>> your repo, but I was unaware that the version on Bitbucket is not up
>> to date with your latest developments.
>> As a result, I was still working on top of the latest commit there:
>>
>> https://bitbucket.org/domruf/kallithea-docker/commits/36265eeea7a94a3538dbdb2e9cf22f0f56252965
>> I would need to reevaluate your changes on dominikruf.com. For the
>> text below, I have not yet done that. It is possible that you already
>> solved some problems.
>
> I did some experiments and some back and forth, so the history of my repo is
> a bit ugly. Sorry.
>>
>>
>> The instance we run is using uwsgi and postgresql.
>>
>> Some feedback:
>> - Since we have some custom changes to Kallithea, I build the image
>> myself. In order to solve the reproducibility and traceability
>> problem, I have added a label to the image, which is visible with
>> 'docker inspect <image>'. There apparently is a schema project that
>> provides a standard proposed set of labels, I took some of them from
>> there:
>>
>> ARG BUILD_DATE=unknown
>>
>> LABEL org.label-schema.build-date=$BUILD_DATE \
>>       org.label-schema.name="Kallithea image" \
>>       org.label-schema.description="Kallithea
>> (https://kallithea-scm.org/)" \
>>       org.label-schema.vcs-ref=$REV \
>>       org.label-schema.vcs-url=$URL \
>>       org.label-schema.schema-version="1.0"
>
> Okay, I didn't know about that. I'll look into it, thanks.
>>
>>
>> - In order to easily provide the build date and other args, I added a
>> makefile with a build target:
>>
>> NAME = foobar/kallithea
>> REV = 8142c5f0fd894a91dfd3f7e8e4484a028c5bfa96
>> URL = http://foobar:8000
>>
>> build:
>>        docker build \
>>                --build-arg REV=$(REV) \
>>                --build-arg URL=$(URL) \
>>                --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` \
>>                --tag $(NAME) \
>>                .
>>
>> - As we want uwsgi, I updated the dockerfile to install uwsgi and
>> uwsgitop with pip. I changed the CMD to:
>> CMD ["uwsgi", "--ini-paste-logged", "../conf/kallithea.ini"]
>> And in the kallithea-entry-point.sh script, I changed:
>> -if [ "$2" == "serve" ]; then
>> +if [ "$1" == "uwsgi" ]; then
>
> I did something similar.
>>
>>
>>
>> - I had to change ownership of the kallithea-entry-point.sh script to
>> the kallithea user, adapting the chmod line to also contain:
>>     chown -R kallithea $KALLITHEA_PATH/src/kallithea-entry-point.sh
>
> Why was this necessary? I mean it is probarbly a good idea but I don't see
> why it' be required.

I don't remember by heart. I need to try it again without that change,
I'll come back later.

>>
>>
>> - Also to improve reproducibility, I added the dockerfile in the image:
>>
>> ADD Dockerfile $KALLITHEA_PATH
>
> So the idea is that somebody who downloaded the image could  extract the
> Dockerfile and
> make some modifications?

Yes. We need to have a situation where a given docker image can be
reproduced exactly by someone else.
So we need to know how exactly an image is built.
Based on your kallithea-docker repo, we had two repos involved: the
kallithea source repo, and the separate docker repo. In order to
reproduce the docker image, you need to know which kallithea source
revision was used (can be solved with the label) and which dockerfile
and arguments were used. To know that, I am using image tags of the
form vX-YYYYYYYYY-k0.3.3-ZZZZZZZZZZ   where X is a sequential number,
YYYY is the hash of kallithea-docker, and ZZZZ is the hash of
kallithea sources.
But ideally, as you did  in your own repo now, the dockerfile is part
of the kallithea source repository. Then, all you need to know is the
kallithea source revision, which we can add in the image label. The
only reason one could still add the Dockerfile inside the image, is to
catch any non-committed changes to it. But if we create a clear
process on how to create official docker images, then that risk can be
mitigated and the explicit ADD of the dockerfile is not actually
necessary.


>>
>>
>>
>> Below things refer to docker-compose.yml:
>>
>> - removed references/names to 'domruf' as we use our own images.
>> - removed explicit references to 172.17.0.1
>> - added log rotation to all services (kallithea, celery, postgres,
>> etc.), for example:
>>     logging:
>>       driver: "json-file"
>>       options:
>>         max-size: "10M"
>>         max-file: "10"
>>
>> - removed explicit container_name statements. It was unclear to me why
>> you did that.
>>
>> - removed the explicit references to kallithea_net network. As far as
>> I see it is not necessary for a basic setup like this. All services
>> are connected to the same network anyway.
>>
>> - removed celerybeat. Can you clarify why it is needed? Is it used by
>> Kallithea mainline?
>
> No, it is used for an experimental feature (cron like tasks i.e. pull tasks)
>>
>>
>> - I changed named volumes for kallithea_conf, kallithea_repos and db
>> data to bind mounts, because it seemed easier for backup to me. But
>> that is probably not 'standard' in docker world.
>>
>> - how do you solve the problem of user permissions between the host
>> system and the docker world? Files created by docker, like the db data
>> directory, are owned by a non-existing user on the host system. To
>> change them, I need to be root, or do changes from inside docker.
>
> I usually use sudo
>>
>>
>>
>> With respect to mainlining docker-related files, and some random thoughts:
>> - we can avoid some duplication by splitting docker-compose.yml, for
>> example the reference to the right image for kallithea, celery, and
>> celerybeat could be done in docker-compose-base.yml similar to what is
>> done here: https://github.com/cabotapp/docker-cabot/
>
> Yes, maybe. I'll look into it.
>>
>>
>>
>> - we need to find a good way to cover different use cases. mysql
>> versus postgres is currently handled by uncommenting some lines, but
>> perhaps we could provide several files instead. Perhaps each of the
>> choices could be in separate files, and a master docker-compose can
>> include the relevant bits based on the user choice. Then the user
>> should only need to change the include statements in the master
>> docker-compose file.
>> Also the choice of web server (gearbox, uwsgi, external reverse proxy)
>> would need to be covered nicely. I'm no expert, but I guess we can
>> learn from other projects.
>>
>> - in the cabot app example I linked above (which is based on Django)
>> they use alpine images for postgres and rabbitmq, which are generally
>> more lightweight. I'm not sure if there are downsides to doing this.
>> It could be something to consider.
>>
>>
>> Best regards,
>> Thomas