Host all your applications in one server

... like a PRO

 Posted on July 22, 2022  |  

Hello,

I own a VPS and like to use it to host my all side projects, including my tool applications that I host by myself.

I use docker to host my application servers since for me it is a no-brainer, quick and simple solution. I used to manually set up a reverse proxy (nginx) that negotiates the SSL connection and forward the request to the good application server. The certificate is generated using Let’s Encrypt, with their certbot tool. And since I own a domain, I usually do some advanced nginx configuration to generate the certificate and forward HTTPS requests.

This was very tedious for (a very lazy person as) me since :

  • I have to manually install certbot on my server.
  • I have to manage by myself the cron job to renew all of my certificates every three months
  • I stop my main nginx every time I renew certificates because certbot don’t know how to automatically handle my dockered nginx server, and thus should start its own web server to handle the acme validation.
  • Do some very advanced nginx configuration forwarding request based on the subdomain to the good docker container (containing the application server requested)
  • I have to do some docker networking between my main nginx server and the docker-compose stack networks of every application.

Portainer

So the first thing I did is to give portainer a try. I found that it definitely does accelerate container management by replacing all the commands I have to check and write every time with simple clicks. So I started by creating the portainer-ce container as follows

docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 \
    --name=portainer --restart=always \
    -v /var/run/docker.sock:/var/run/docker.sock \
    portainer/portainer-ce

Testing portainer a little bit, it seems a good tool for quickly testing my setup.

Multiple applications with reverse proxy

Now the funny part, I needed to setup, with the least possible effort and configuration, a reverse proxy with automatic certification for my subdomain.

Googling a little bit I found the fantastic nginx-proxy. I like opensource software, and every time I find exactly whay I need it a tool like this, I get even more excited about it.

So nginx-proxy is a docker container that automatically handle forwarding requests to other docker containers based of the requested domain. All this without any configuration file !

All I have to do is, firstly launch proxy :

docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro nginxproxy/nginx-proxy

And then for every application server, all I have to do is to add an env variable VIRTUAL_HOST containing the domain that it needs to serve, for example:

docker run -e VIRTUAL_HOST=foo.bar.com  hello-world
docker run -e VIRTUAL_HOST=boo.bar.com  hello-world

And we are done ! Supposing that you already configured your domain to point foo.bar.com and boo.bar.com to the same server, visiting the http://foo.bar.com would be redirected to the hello-world container, and http://boo.bar.com will be served by the second one

HTTPS

Now we would access our applications using https and not simple unencrypted HTTP. We will take advantage of the free service provided by Let’s Encrypt. But again, it should be dumb and simple to setup.

Googling a little bit I found another fantastic tool acme-companion

Now this is getting even more funny, because this works along with nginx-proxy. It will use the method to know about the proxied applications to handle, but now with the LETSENCRYPT_HOST variable that should contain the domain to request certification for, for every proxied container.

Since we will always be using the nginx-proxy for our application, but with SSL, there are some files that should be shared between the two containers (ie the ssl certifications). We will simple create docker volumes that will be shared between them. I’ll include the architecture picture I found in the github project

Shenandoah GC cycle

Creating the volumes:

Using Portainer it is very simple just navigate to the good tab and create theses volumes :

acme
certs
vhost
html

Changes to the nginx-proxy

The last command to launch the nginx proxy should be updated to use theses volumes :

docker run --detach \
    --name nginx-proxy \
    --publish 80:80 \
    --publish 443:443 \
    --volume certs:/etc/nginx/certs \
    --volume vhost:/etc/nginx/vhost.d \
    --volume html:/usr/share/nginx/html \
    --volume /var/run/docker.sock:/tmp/docker.sock:ro \
    nginxproxy/nginx-proxy

PS: I am doing this using portainer, the the same but using the UI.

launch acme-compagnion

Now launching acme-compagnion, using the same volumes as the nginx-proxy and not using DEFAULT_EMAIL asked by Let’s Encrypt when generating certificates.

docker run --detach \
    --name nginx-proxy-acme \
    --volumes-from nginx-proxy \
    --volume /var/run/docker.sock:/var/run/docker.sock:ro \
    --volume acme:/etc/acme.sh \
    --volume certs:/etc/nginx/certs \
    --volume vhost:/etc/nginx/vhost.d \
    --volume html:/usr/share/nginx/html \
    --env "DEFAULT_EMAIL=mail@yourdomain.tld" \
    nginxproxy/acme-companion

PS: I am doing this using portainer, the the same but using the UI.

Proxiyng our application :

Now the setup is done, just launch your applications as follow, using LETSENCRYPT_HOST to specify the certificate’s domain for each application:

docker run -e VIRTUAL_HOST=foo.bar.com LETSENCRYPT_HOST=foo.bar.com hello-world
docker run -e VIRTUAL_HOST=boo.bar.com LETSENCRYPT_HOST=boo.bar.com hello-world

Looking in the acme-proxy log, you will see that it received the start request of these two containers, and requested the certificates for each domain, nothing more needed. Just navigate in your browser to https://foo.bar.com and you’ll be up and running.

Bonus, portainer on HTTPS :

I have a little trick to run portainer also on https: it is very simple, you should launch portainer using the certt volume, as for a proxied application :

docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 \
    --name=portainer --restart=always \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v portainer_data:/data \
	-v certs:/certs \
    portainer/portainer-ce

In portainer create a dummy container, as done above and asking the request certification for the domain used to access portainer, acme-proxy will request a certificate and store it in the cert volume and do the job to configure it through nginx-proxy and there for access portainer.

Conclusion

Now you have a server with portainer to handle deployments. Adding new application to your server is as simple as adding to the container two env variables LETSENCRYPT_HOST and VIRTUAL_HOST pointing to the domain it should serve.