Docker update (looking for testers)

Wed Nov 15 17:09:49 UTC 2017

Thomas De Schampheleire <patrickdepinguin at gmail.com> schrieb am Mi., 15.
Nov. 2017 um 08:50 Uhr:

> 2017-11-14 22:55 GMT+01:00 Dominik Ruf <dominikruf at gmail.com>:
> >
> >
> > Thomas De Schampheleire <patrickdepinguin at gmail.com> schrieb am Di., 14.
> > Nov. 2017 um 21:32 Uhr:
> >>
> >> Hi Dominik,
> >>
> >> 2017-09-20 20:52 GMT+02:00 Dominik Ruf <dominikruf at gmail.com>:
> >> > Hi all,
> >> >
> >> > like I mentioned before, I'd like to add a Dockerfile and
> >> > docker-compose.yml
> >> > to make it really easy to get started with kallithea.
> >> > I did some more work on my docker setup and it now also includes ssh
> >> > support.
> >> > And like before it generates the css file from less.
> >> > So since these 2 things are not merged yet, I'd like to wait before
> >> > creating
> >> > a PR.
> >> >
> >> > But I wanted to ask if some people would volunteer to already test it.
> >> > I'd like to hear from others if it helps them setting up a Full Stack
> >> > installation.
> >> > And if there are unclear things about how to use it.
> >> >
> >> > I upload my images to
> https://hub.docker.com/r/domruf/kallithea-domruf/
> >> > and you can find the docker-compose.yml at
> >> >
> >> >
> https://kallithea.dominikruf.com/kallithea/kallithea-domruf/raw/26ccf792b435/docker/docker-compose.yml
> >> >
> >>
> >> Sorry, I kind of forgot about this mail.
> >>
> >> I have recently deployed a kallithea instance with docker based on
> >> your repo, but I was unaware that the version on Bitbucket is not up
> >> to date with your latest developments.
> >> As a result, I was still working on top of the latest commit there:
> >>
> >>
> https://bitbucket.org/domruf/kallithea-docker/commits/36265eeea7a94a3538dbdb2e9cf22f0f56252965
> >> I would need to reevaluate your changes on dominikruf.com. For the
> >> text below, I have not yet done that. It is possible that you already
> >> solved some problems.
> >
> > I did some experiments and some back and forth, so the history of my
> repo is
> > a bit ugly. Sorry.
> >>
> >>
> >> The instance we run is using uwsgi and postgresql.
> >>
> >> Some feedback:
> >> - Since we have some custom changes to Kallithea, I build the image
> >> myself. In order to solve the reproducibility and traceability
> >> problem, I have added a label to the image, which is visible with
> >> 'docker inspect <image>'. There apparently is a schema project that
> >> provides a standard proposed set of labels, I took some of them from
> >> there:
> >>
> >> ARG BUILD_DATE=unknown
> >>
> >> LABEL org.label-schema.build-date=$BUILD_DATE \
> >>       org.label-schema.name="Kallithea image" \
> >>       org.label-schema.description="Kallithea
> >> (https://kallithea-scm.org/)" \
> >>       org.label-schema.vcs-ref=$REV \
> >>       org.label-schema.vcs-url=$URL \
> >>       org.label-schema.schema-version="1.0"
> >
> > Okay, I didn't know about that. I'll look into it, thanks.
> >>
> >>
> >> - In order to easily provide the build date and other args, I added a
> >> makefile with a build target:
> >>
> >> NAME = foobar/kallithea
> >> REV = 8142c5f0fd894a91dfd3f7e8e4484a028c5bfa96
> >> URL = http://foobar:8000
> >>
> >> build:
> >>        docker build \
> >>                --build-arg REV=$(REV) \
> >>                --build-arg URL=$(URL) \
> >>                --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` \
> >>                --tag $(NAME) \
> >>                .
> >>
> >> - As we want uwsgi, I updated the dockerfile to install uwsgi and
> >> uwsgitop with pip. I changed the CMD to:
> >> CMD ["uwsgi", "--ini-paste-logged", "../conf/kallithea.ini"]
> >> And in the kallithea-entry-point.sh script, I changed:
> >> -if [ "$2" == "serve" ]; then
> >> +if [ "$1" == "uwsgi" ]; then
> >
> > I did something similar.
> >>
> >>
> >>
> >> - I had to change ownership of the kallithea-entry-point.sh script to
> >> the kallithea user, adapting the chmod line to also contain:
> >>     chown -R kallithea $KALLITHEA_PATH/src/kallithea-entry-point.sh
> >
> > Why was this necessary? I mean it is probarbly a good idea but I don't
> see
> > why it' be required.
>
> I don't remember by heart. I need to try it again without that change,
> I'll come back later.
>
> >>
> >>
> >> - Also to improve reproducibility, I added the dockerfile in the image:
> >>
> >> ADD Dockerfile $KALLITHEA_PATH
> >
> > So the idea is that somebody who downloaded the image could  extract the
> > Dockerfile and
> > make some modifications?
>
> Yes. We need to have a situation where a given docker image can be
> reproduced exactly by someone else.
> So we need to know how exactly an image is built.
> Based on your kallithea-docker repo, we had two repos involved: the
> kallithea source repo, and the separate docker repo. In order to
> reproduce the docker image, you need to know which kallithea source
> revision was used (can be solved with the label) and which dockerfile
> and arguments were used. To know that, I am using image tags of the
> form vX-YYYYYYYYY-k0.3.3-ZZZZZZZZZZ   where X is a sequential number,
> YYYY is the hash of kallithea-docker, and ZZZZ is the hash of
> kallithea sources.
> But ideally, as you did  in your own repo now, the dockerfile is part
> of the kallithea source repository. Then, all you need to know is the
> kallithea source revision, which we can add in the image label. The
> only reason one could still add the Dockerfile inside the image, is to
> catch any non-committed changes to it. But if we create a clear
> process on how to create official docker images, then that risk can be
> mitigated and the explicit ADD of the dockerfile is not actually
> necessary.
>
Bare in mind that we (currently) do not pin the versions of the pypi or deb
packages,
so a newly generated images may not be exactly the same.

>
>
> >>
> >>
> >>
> >> Below things refer to docker-compose.yml:
> >>
> >> - removed references/names to 'domruf' as we use our own images.
> >> - removed explicit references to 172.17.0.1
> >> - added log rotation to all services (kallithea, celery, postgres,
> >> etc.), for example:
> >>     logging:
> >>       driver: "json-file"
> >>       options:
> >>         max-size: "10M"
> >>         max-file: "10"
> >>
> >> - removed explicit container_name statements. It was unclear to me why
> >> you did that.
> >>
> >> - removed the explicit references to kallithea_net network. As far as
> >> I see it is not necessary for a basic setup like this. All services
> >> are connected to the same network anyway.
> >>
> >> - removed celerybeat. Can you clarify why it is needed? Is it used by
> >> Kallithea mainline?
> >
> > No, it is used for an experimental feature (cron like tasks i.e. pull
> tasks)
> >>
> >>
> >> - I changed named volumes for kallithea_conf, kallithea_repos and db
> >> data to bind mounts, because it seemed easier for backup to me. But
> >> that is probably not 'standard' in docker world.
> >>
> >> - how do you solve the problem of user permissions between the host
> >> system and the docker world? Files created by docker, like the db data
> >> directory, are owned by a non-existing user on the host system. To
> >> change them, I need to be root, or do changes from inside docker.
> >
> > I usually use sudo
> >>
> >>
> >>
> >> With respect to mainlining docker-related files, and some random
> thoughts:
> >> - we can avoid some duplication by splitting docker-compose.yml, for
> >> example the reference to the right image for kallithea, celery, and
> >> celerybeat could be done in docker-compose-base.yml similar to what is
> >> done here: https://github.com/cabotapp/docker-cabot/
> >
> > Yes, maybe. I'll look into it.
> >>
> >>
> >>
> >> - we need to find a good way to cover different use cases. mysql
> >> versus postgres is currently handled by uncommenting some lines, but
> >> perhaps we could provide several files instead. Perhaps each of the
> >> choices could be in separate files, and a master docker-compose can
> >> include the relevant bits based on the user choice. Then the user
> >> should only need to change the include statements in the master
> >> docker-compose file.
> >> Also the choice of web server (gearbox, uwsgi, external reverse proxy)
> >> would need to be covered nicely. I'm no expert, but I guess we can
> >> learn from other projects.
> >>
> >> - in the cabot app example I linked above (which is based on Django)
> >> they use alpine images for postgres and rabbitmq, which are generally
> >> more lightweight. I'm not sure if there are downsides to doing this.
> >> It could be something to consider.
> >>
> >>
> >> Best regards,
> >> Thomas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20171115/f56c2fba/attachment-0001.html>