ISPConfig: running a Python Flask Docker container as a jailed Shell User
The method described requires you can be root, meaning it is not universal but may be sufficient if you are the system administrator.
I run a server with ISPConfig with some 50 sites. The sites are static or PHP. I am developing Python Flask applications now and also want to run them on the ISPConfig server. You can create virtual environments on the ISPConfig server and run your application from here. But some time ago I have chosen to use Docker for development, staging and production. It takes time to set this up but it is really worth it. Docker is way to go.
I will use a jailed Shell User to run the container. The reason is that when the container breaks access is limited to the jailed Shell User rights, is it? See also Summary below.
My ISPConfig system:
- ISPConfig3 3.1.13
- Debian 9 (Stretch)
- MariaDB 10.3
- Nginx 1.10.3
My Docker container on ISPConfig
For staging and production the Python Flask Docker container, based on Alpine and containing Python, the application and the Gunicorn WSGI webserver, uses a 'volumes' mapping to log files, session files and cache files. In addition it contains a 'volumes' mapping to the static directory.
For development, staging, and production I use a .dockerignore file to exclude the static folder from the Docker image. It grows fast with all the images. For development we do not need it anyway as we serve everything outside the Docker container. For staging and production we also do not want the static folder in the container. Here we serve the static items not with Gunicorn but directly with Nginx.
The Docker Python Flask container does not contain a database, etc. but uses the ISPConfig services for easy configuration and management:
- The ISPConfig domain and sites management (including Letsencrypt SSL)
- The host database (MariaDB), connect via a socket
- The host mail (Postfix), connect via port 25
- The host webserver (Nginx), reverse proxy and serving static
Note that I build the Docker image on my local machine using:
docker save ...
The resulting tar file is copied to the ISPConfig server and unpacked, see below.
To configure ISPConfig for our site we do the usual:
- Install Docker and Docker-compose (one time only)
- Add domain
- Add website, set Letsencrypt SSL
- Add database user and database
- Add a jailed (!) Shell User, Chroot Shell: Jailkit
Directories on ISPConfig, user and group
When we added the site (and created the shell user) ISPConfig created a (Linux) user for it. My Shell User:
- Username: peterpepyco
The linux user and group, see ISPConfig -> Shell User -> Options, in my case:
- Web Username: web73
- Web Group: client2
You can also see this by logging in with the Shell User and walking through some directories, doing a 'ls -n'.
There is a difference between a jailed Shell User and non-jailed Shell User. In both cases the base directory is:
/var/www/clients/client2/web73
The home directory is:
/var/www/clients/client2/web73/home/peterpepyco
and the web directory is:
/var/www/clients/client2/web73/web
When the Shell User is jailed, the file system root changes to the base directory.To get the group of the Shell User type:
groups
which returns in my case:
client2
To run the Docker as a different user we need the user id, UID, and group id, GID. To get the UID type:
id -u
which returns in my case 5055, and:
id -g
which returns in my case 5006. There are many ways to get UID and GID. You can also type:
cat /etc/passwd
which returns:
root:x:0:0:root:/root:/bin/bash
peterpepyco:x:5055:5006:::/bin/bash
and
cat /etc/group
which returns:
root:x:0:
client2:x:5006:
You can also create a file, 'echo "" > a', and then do 'ls -n', etc.
Modifying the Nginx configuration
In ISPConfig go to the site and select the Options tab. In the Nginx Directives section paste:
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 1M;
}
location /static {
alias /var/www/clients/client2/web73/web/static;
}
Note that reverse proxy passes requests to port 8000. This is the port the Gunicorn WSGI server in the container is listening on.
Copying the files
In the Shell User home I create a directory docker where I copy the compressed container, environment variables file, the docker-compose files and the database. After the copy the directory looks like:
.
└── docker
├── .env
├── docker-compose_base_1.283_production.yml
├── docker-compose_production.yml
├── docker-volumes
│ ├── cache
│ │ ├── other
│ │ ├── query_result
│ │ └── render_template
│ ├── flask_session
│ └── log
├── peterspython2.dump_20191017
├── peterspython_image_web_1.283.tar
└── project
├── Dockerfile
└── requirements.txt
To load the database:
mysql -upeterspythonuser -p peterspython2 < peterspython2.dump_20191017
To load the docker image, requires you are root:
docker load -i peterspython_image_web_1.283.tar
Next step is copying the static folder from my local system to the ISPConfig '/web' folder, see also above.
Add user, UID, and group, GID, to docker-compose and Dockerfile
I use Docker-compose to start and stop the container, map volumes, etc. The .env file holds a number of configuration variables we pass to docker-compose. First part of this file:
# production environment vars
PROJECT_NAME=peterspython
FLASK_CONFIG=production
# docker-compose, docker
# peterpepyco:client2
CONTAINER_USER=peterpepyco
CONTAINER_UID=5055
CONTAINER_GROUP=client2
CONTAINER_GID=5006
...
The first part of the compose file:
# docker-compose_base.yml
version: '3.2'
services:
web:
image: ${PROJECT_NAME}_image_web:1.283
container_name: ${PROJECT_NAME}_container_web
env_file:
- ./.env
restart: always
build:
context: ./project
dockerfile: Dockerfile
args:
- CONTAINER_USER=${CONTAINER_USER}
- CONTAINER_UID=${CONTAINER_UID}
- CONTAINER_GROUP=${CONTAINER_GROUP}
- CONTAINER_GID=${CONTAINER_GID}
ports:
- "${SERVER_PORT_HOST}:${SERVER_PORT_CONTAINER}"
volumes:
...
and then in the Dockerfile:
...
# create and set working directory
RUN mkdir -p /home/flask/project
WORKDIR /home/flask/project
# copy app code into container
COPY . ./
# create group and user used in this container
ARG CONTAINER_USER
ARG CONTAINER_UID
ARG CONTAINER_GROUP
ARG CONTAINER_GID
RUN addgroup -g $CONTAINER_GID $CONTAINER_GROUP && \
adduser -D -H -G $CONTAINER_GROUP -u $CONTAINER_UID $CONTAINER_USER && \
chown -R $CONTAINER_USER:$CONTAINER_GROUP /home/flask
USER $CONTAINER_USER
Starting the container
Again this requires you are root:
docker-compose -f docker-compose_base_1.283_production.yml -f docker-compose_production.yml up -d
The result is:
Creating network "docker_default" with the default driver
Creating peterspython_container_web ... done
If the container does not start check logs, messages. If it runs but you get errors you can enter the running container, first get the container id:
docker ps
which returns:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
292aa9bcecaf peterspython_image_web:1.283 "/usr/local/bin/guni…" 18 hours ago Up 18 hours 0.0.0.0:8000->8000/tcp peterspython_container_web
Then enter the running container:
docker exec -it 292aa9bcecaf sh
Note that we start sh and not bash because bash is not in the Alpine image.
Summary
It is not really difficult once you understand (part of) Docker and (part of) ISPConfig. Now you can run anything you want on an ISPConfig server.
I used a maximum of ISPConfig services, I am happy using MariaDB but some people may complain that for example PostgreSQL is not supported. It would be nice if ISPConfig would add PostgreSQL as an option. That would be better than adding a PostgreSQL service to the container, increasing the size of the container.
A problem is that we need to be root when loading the Docker image and starting and stopping the container, in fact this is required for every Docker command. This means this method is not suitable for just some random clients. It would be nice if ISPConfig would support a method to allow per-site Docker and Docker-Compose commands. Also containers expose ports which may conflict with existing ones. This can be solved by assigning a per-site port range.
Is the setup on ISPConfig secure enough using the jailed Shell User credentials to run the container? I see a possible problem with the use of the UID and starting the container being root. The UID of the jailed Shell User peterpepyco is the same as the UID of web73 meaning that the Docker container in fact runs as web73:client2 and not as peterpepyco:client2. I must look into this further. Perhaps namespace can be used. But for the moment I am happy it is running.
Links / credits
How do I add a user when I'm using Alpine as a base image?
https://stackoverflow.com/questions/49955097/how-do-i-add-a-user-when-im-using-alpine-as-a-base-image
Running Docker Containers as Current Host User
https://jtreminio.com/blog/running-docker-containers-as-current-host-user/
Compose file version 3 reference
https://docs.docker.com/compose/compose-file/
How to copy Docker images from one host to another without using a repository
https://stackoverflow.com/questions/23935141/how-to-copy-docker-images-from-one-host-to-another-without-using-a-repository
Isolate containers with a user namespace
https://docs.docker.com/engine/security/userns-remap/
Read more
Docker Docker-compose Flask ISPConfig
Recent
- Hiding database UUID primary keys of your web application
- Don't Repeat Yourself (DRY) with Jinja2
- SQLAlchemy, PostgreSQL, maximum number of rows per user
- Show the values in SQLAlchemy dynamic filters
- Secure data transfer with Public Key encryption and pyNaCl
- rqlite: a high-availability and distributed SQLite alternative
Most viewed
- Using Python's pyOpenSSL to verify SSL certificates downloaded from a host
- Using UUIDs instead of Integer Autoincrement Primary Keys with SQLAlchemy and MariaDb
- Connect to a service on a Docker host from a Docker container
- Using PyInstaller and Cython to create a Python executable
- SQLAlchemy: Using Cascade Deletes to delete related objects
- Flask RESTful API request parameter validation with Marshmallow schemas