Dockerize Django/Postgres Application

Introduction:

                   

                           In this article I will explain you about how to dockerize your django application for different environments ie.. development and master. Because we may need different containers in production and development stage. For example, you may use postgres for development at local and RDS at production environment, so for we need two containers (db and app) at development and only app container at production server.

What is Docker?

                         Docker is a program that performs virtualization by creating image for all required packages/libraries for your app, which is also known as "containerization". Docker runs various containers, for example one may run your web server and your app and another may run your database server which is used by your app.

Containers are light-weight executable packages which includes evrything needed to run an application. Once tha app is running fine after building the images, its going to run on all machines which has docker installed in it.

Dockerize your app:

                       The first step to dockerize any app is defining the components of the app, for which you need to create `Dockerfile` and a `docker-compose.yml` file. You can give any name you want, but if you are giving different name then you need to give `dockerfile: <path/to/Dockerfile>` in service details of `yml` file.

Docker file defines application's image content with different build commands. Add the following content to the docker file.

FROM python:3.6

ENV PYTHONUNBUFFERED 1

RUN mkdir /code

WORKDIR /code

ADD requirements.txt /code/

RUN pip install -r requirements.txt

ADD . /code/

COPY ./start.sh /start.sh

RUN sed -i 's/\r//' /start.sh

RUN chmod +x /start.sh

- Dockerfile starts with Python 3.6 image, which is modified by creating new app directory, now it is further modified by adding `requirements.txt` and installing all the dependencies from `requirements.txt`.

- Now you need to create another file `start.sh` in which you will write the commands to be executed or if you want we can write the commads. Add the following content in `start.sh`

#!/bin/sh

python manage.py migrate

while true

do

python manage.py runserver 0.0.0.0:8000

done

- In this file you are simply migrating and running the django server at 0.0.0.0:8000.

- Now you need to create `requirements.txt` file where you will keep all the requirements of the app and `docker-compose.yml` file which describes the services that makes your app, in these case we need app and database server.

- Content for requirements.txt

Django>=1.8,<2.0

psycopg2

- Content for `docker-compose.yml` file

version: '3'

volumes:

postgres_data_dev: {}

postgres_backup_dev: {}

services:

db:

build: ./dockerconfig/postgres

volumes:

- postgres_data_dev:/var/lib/postgresql/data

- postgres_backup_dev:/backups

environment:

- POSTGRES_USER=postgres

ports:

- "5432:5432"

web:

build: .

command: /start.sh

depends_on:

- db

volumes:

- .:/code

ports:

- "8000:8000"

links:

- db

- These yml file defines two services. The db service and the web service with mechanism for taking dump and restoring it while building again.

- In development mode developer need to build container n number of times, and killing the db container will drop the data which becomes difficult for developer to create db again and again, so we have added the logic to take the dump and restore it.

- You can create separate folder to keep your docker related data, like which all containers you want with their Dockerfile. So created a folder `dockerconfig` inside which again create `postgres` folder to keep postgres Dockerfile and other files which has commands to take dump and restore data, you can also create another folder inside dockerconfig to store your app related Dockerfile, for now I am creating them in base directory and creating only for Postgres to show both approaches.

- Now create `Dockerfile` inside `postrges` folder and add the following content.

FROM postgres:9.6

# add backup scripts

ADD backup.sh /usr/local/bin/backup

ADD restore.sh /usr/local/bin/restore

ADD list-backups.sh /usr/local/bin/list-backups

# make them executable

RUN chmod +x /usr/local/bin/restore

RUN chmod +x /usr/local/bin/list-backups

RUN chmod +x /usr/local/bin/backup

FROM postgres:9.6 will create parent image of postgres9.6 and follwoing files will be added.

Create `backup.sh` in postgres folder and add the following content.

#!/bin/bash

# stop on errors

set -e

if [ "$POSTGRES_USER" == "postgres" ]

then

echo "creating backup"

exit 1

fi

# export the postgres password so that subsequent commands don't ask for it

export PGPASSWORD=$POSTGRES_PASSWORD

echo "creating backup"

echo "==============="

FILENAME=backup_$(date +'%Y_%m_%dT%H_%M_%S').dump

pg_dump -Fc --no-acl --no-owner -h postgres -U $POSTGRES_USER > /backups/$FILENAME

echo "backup taken successfully - $FILENAME"

- This file basically exports the POSTGRES password and creates a dump file with name of combination of backup and timestamp where system takes pg_dump with above mentioned command.

- Create another file list-backups.sh and add the following content.

#!/bin/bash

echo "listing available backups"

echo "========================="

ls /backups/

- Create one more file restore.sh which has commands to restore the data from the dump file stored in last commands and add the following content.

#!/bin/bash

# stop on errors

set -e

if [ "$POSTGRES_USER" == "postgres" ]

then

echo "restoring backup"

exit 1

fi

# export the postgres password so that subsequent commands don't ask for it

export PGPASSWORD=$POSTGRES_PASSWORD

# check that we have an argument for a filename candidate

if [[ $# -eq 0 ]] ; then

echo 'usage:'

echo ' docker-compose run postgres restore <backup-file>'

echo ''

echo 'to get a list of available backups, run:'

echo ' docker-compose run postgres list-backups'

exit 1

fi

# set the backupfile variable

BACKUPFILE=/backups/$1

# check that the file exists

if ! [ -f $BACKUPFILE ]; then

echo "backup file not found"

echo 'to get a list of available backups, run:'

echo ' docker-compose run postgres list-backups'

exit 1

fi

echo "beginning restore from $1"

echo "-------------------------"

echo "deleting old database $POSTGRES_USER"

if dropdb -h postgres -U $POSTGRES_USER $POSTGRES_USER

then echo "deleted $POSTGRES_USER database"

else echo "database $POSTGRES_USER does not exist, continue"

fi

# create a new database

echo "creating new database $POSTGRES_USER"

createdb -h postgres -U $POSTGRES_USER $POSTGRES_USER -O $POSTGRES_USER

# restore the database

echo "restoring database $POSTGRES_USER"

pg_restore --verbose --clean --no-acl --no-owner -h postgres -U $POSTGRES_USER -d $POSTGRES_USER $BACKUPFILE

- Now lets create the django app with `docker run ` to start with example, `docker-compose run web django-admin.py startproject djangowithdocker`, now move all files and folders to base directory of project, here you may need to update the permission for this folder for your user.

- Next step is update your `requirements.txt` file with `django-environ==0.4.3` library to read environment variable and update your `settings.py` file with following content.

import environ

env = environ.Env()

DATABASES = {

'default': env.db('DATABASE_URL', default='postgres://<user_name>:<password>@db/<database_name>'),

}

- Create `.env` file in base directory to export `DATABASE_URL` with following content.

DATABASE_URL=postgres://postgres:postgres@db/postgres

- Now build it using `docker-compose build` and once its build you can run the app with `docker-compose up`, you can see the app is running at `localhost:8000`.

- If you want to run the containers in background run it with -d `docker-compose up -d`.

- Now, lets create a superuser for the django admin, now there are two ways to do that you can either create with `docker-compose run web python manage.py createsuperuser` or you can go into the container and simply run `python manage.py createsuperuser`, to go into the container you need to run `docker exec -it <ImageID> bash`.

- Now lets login with newly created superuser in django admin panel.

- Now, stop the containers with `docker-compose stop` and run it again and cross check if the user is present or not.

- Login again to admin panel and cross check if all the created users are there or not?

- Now, In case if you dont need postgres container at production server if you are using RDS, then you can create another yml file `docker-compose-rds.yml` with following content.

version: '3'

services:

web:

build: .

env_file: .env

command: /start.sh

ports:

- "8000:8000"

- Where you can define only web service, and RDS url can be kept in `.env` file. Now to build using this new file you need to give the path of `yml` file to build and run the docker containers.

`docker-compose -f docker-compose-rds.yml build` and then `docker-compose -f docker-compose-rds.yml up`.

Conclusion:

                         In this article you have learnt how to dockerize an app with different containers at different servers to run you web app and database server. You can find the code at github repo `https://github.com/pawan3103/dockerizedDjango` for your reference.