Splits nginx into static and static-git

README.md

@@ -23,11 +23,11 @@ vagrant up
 Wait for the provisioning to finish (~5mins), and go to your browser: https://indiehosters.dev
 If the process fails, for instance due to network problems, you can retry by running `vagrant provision`.
 
-### If you want to add another nginx instance apart from indiehosters.dev:
+### If you want to add another static-git instance apart from indiehosters.dev:
 
 ```bash
 vagrant ssh
-sudo sh /data/indiehosters/scripts/activate-user.sh example.dev nginx https://github.com/indiehosters/website-static.git
+sudo sh /data/indiehosters/scripts/activate-user.sh example.dev static-git https://github.com/indiehosters/website.git
 ```
 Check https://example.dev in your bowser!
 

deploy/add-placeholder-site.sh

@@ -15,6 +15,6 @@ fi
 echo "Adding $DOMAIN to $SERVER with cert from $PEMFILE"
 echo "Remote user is $USER"
 
-ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/nginx/data
+ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/static
 scp $PEMFILE $USER@$SERVER:/data/runtime/haproxy/approved-certs/$DOMAIN.pem
-ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN nginx
+ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN static

doc/architecture-overview.md

@@ -12,12 +12,12 @@ There is currently no similar side-kick for updating `/data/runtime/postfix/`, s
 
 The `scripts/setup.sh` takes care of setting up etcd, enabling and starting the haproxy and postfix service (as well as one haproxy backend, to serve the default site), and the haproxy-confd side-kick to listen for changes in the backends configuration in etcd, so that new backends are automatically added to the haproxy config as soon as their IP address is written into etcd.
 
-## HAProxy backends: nginx, wordpress
+## HAProxy backends: static, static-git, wordpress
 
-A per user process is a haproxy backend for a specific domain name. At the time of writing we have two applications available: nginx and wordpress.
+A per user process is a haproxy backend for a specific domain name. At the time of writing we have three applications available: static, static-git and wordpress.
 
 You will notice there are also some other units in the `unit-files/` folder of this repo, like the gitpuller and mysql ones. Whenever you start a wordpress unit, it requires a mysql service.
-Whenever you start an nginx unit, it wants a gitpuller unit. In all three cases, an -importer unit and a -discovery unit are required.
+Whenever you start a static-git unit, it wants a static-git-puller unit. In all three cases, an -importer unit and a -discovery unit are required.
 This works through a
 [`Requires=` directive](https://github.com/indiehosters/indiehosters/blob/0.1.0/unit-files/nginx@.service#L6-L7) which systemd interprets, so that if you start one service, its dependencies are also started (you can see that in `systemctl list-units`).
 

importers/nginx.sh

-#!/bin/bash -eux
-
-if [ ! -e "/data/domains/$DOMAIN/nginx/data/www-content/index.html" ]; then
-  if [ -e "/data/domains/$DOMAIN/nginx/data/GITURL" ]; then
-    git clone `cat /data/domains/$DOMAIN/nginx/data/GITURL` /data/domains/$DOMAIN/nginx/data/www-content
-    cd /data/domains/$DOMAIN/nginx/data/www-content && git checkout master
-  else
-    mkdir -p /data/domains/$DOMAIN/nginx/data/www-content
-    echo Hello $DOMAIN > /data/domains/$DOMAIN/nginx/data/www-content/index.html
-  fi
-fi

scripts/activate-user.sh

@@ -7,10 +7,10 @@ else
   echo "Usage: sh /data/indiehosters/scripts/activate-user.sh domain image [gitrepo]"
   exit 1
 fi
-mkdir -p /data/domains/$DOMAIN/$IMAGE/data
+mkdir -p /data/domains/$DOMAIN/$IMAGE
 if [ $# -ge 3 ]; then
   GITREPO=$3
-  echo $GITREPO > /data/domains/$DOMAIN/$IMAGE/data/GITURL
+  echo $GITREPO > /data/domains/$DOMAIN/$IMAGE/GITURL
 fi
 
 # Start service for new site (and create the user). This will also enable the git puller.

scripts/setup.sh

@@ -25,8 +25,8 @@ cp /data/indiehosters/unit-files/* /etc/systemd/system
 systemctl daemon-reload
 
 # Activate default domain
-sh /data/indiehosters/scripts/activate-user.sh $HOSTNAME nginx
-etcdctl set /services/default '{"app":"nginx", "hostname":"'$HOSTNAME'"}'
+sh /data/indiehosters/scripts/activate-user.sh $HOSTNAME static
+etcdctl set /services/default '{"app":"static", "hostname":"'$HOSTNAME'"}'
 
 # Configure and start HAproxy
 mkdir -p /data/runtime/haproxy/approved-certs

unit-files/nginx-gitpuller@.path

-[Unit]
-Description=Start the timer if there is a GITURL file
-
-# Dependency binding
-BindsTo=nginx@%i.service
-
-[Path]
-PathExists=/data/domains/%i/nginx/data/GITURL
-Unit=nginx-gitpuller@%i.timer

unit-files/nginx-importer@.service

-[Unit]
-Description=nginx importer
-
-# Dependency ordering
-Before=nginx@%i.service
-
-# Dependency binding
-BindsTo=mysql@%i.service
-
-[Service]
-Type=oneshot
-RemainAfterExit=yes
-Environment=DOMAIN=%i
-ExecStart=/data/indiehosters/importers/nginx.sh
-
-[Install]
-WantedBy=nginx@%i.service

unit-files/nginx-discovery@.service → unit-files/static-discovery@.service

@@ -6,10 +6,10 @@ Requires=etcd.service
 
 # Dependency ordering
 After=etcd.service
-After=nginx@%i.service
+After=static@%i.service
 
 # Dependency binding
-BindsTo=nginx@%i.service
+BindsTo=static@%i.service
 
 [Service]
 # Start

unit-files/static-git-discovery@.service

+[Unit]
+Description=%p for %i etcd registration
+
+# Requirements
+Requires=etcd.service
+
+# Dependency ordering
+After=etcd.service
+After=static-git@%i.service
+
+# Dependency binding
+BindsTo=static-git@%i.service
+
+[Service]
+# Start
+## Test whether service is accessible and then register useful information
+ExecStart=/bin/bash -euxc ' \
+  sleep 3; \
+  while true; do \
+    app=`echo %p | cut -d"-" -f1,2`; \
+    ip=`docker inspect --format \'{{.NetworkSettings.IPAddress}}\' $app-%i`; \
+    curl -f $ip; \
+    if [ $? -eq 0 ]; then \
+      etcdctl set /services/$app/%i \'{"ip":"\'$ip\'", "port":"80"}\' --ttl 60; \
+    else \
+      etcdctl rm /services/$app/%i; \
+    fi; \
+    sleep 50; \
+  done'
+
+# Stop
+ExecStop=/bin/bash -euxc ' \
+  app=`echo %p | cut -d"-" -f1`; \
+  /usr/bin/etcdctl rm /services/$app/%i

unit-files/static-git-importer@.service

+[Unit]
+Description=static importer
+
+# Dependency ordering
+Before=static-git@%i.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/bin/bash -euxc ' \
+  runtime_folder=/data/runtime/domains/%i/static-git; \
+  mkdir -p $runtime_folder; \
+  git_folder=$runtime_folder/www-content; \
+  if [ ! -d $git_folder/.git ]; then \
+    git clone `cat /data/domains/%i/static-git/GITURL` $git_folder; \
+  fi;'
+
+[Install]
+WantedBy=static-git@%i.service

unit-files/nginx-gitpuller@.service → unit-files/static-git-puller@.service

@@ -4,6 +4,6 @@ Description=git puller
 [Service]
 Type=oneshot
 ExecStart=/bin/bash -euxc ' \
-  app=`echo %p | cut -d"-" -f1`; \
-  cd /data/domains/%i/$app/data/www-content; \
+  app=`echo %p | cut -d"-" -f1,2`; \
+  cd /data/runtime/domains/%i/$app/www-content; \
   git pull'

unit-files/nginx-gitpuller@.timer → unit-files/static-git-puller@.timer

@@ -2,7 +2,7 @@
 Description=Run git puller service every 10 minutes
 
 # Dependency binding
-BindsTo=nginx-gitpuller@%i.path
+BindsTo=static-git@%i.service
 
 [Timer]
 OnUnitActiveSec=10min

unit-files/static-git@.service

+[Unit]
+Description=%p-%i
+
+# Requirements
+Requires=docker.service
+Requires=%p-importer@%i.service
+Requires=%p-discovery@%i.service
+Requires=%p-puller@%i.timer
+Requires=backup@%i.timer
+
+# Dependency ordering
+After=docker.service
+After=%p-importer@%i.service
+Before=%p-discovery@%i.service
+
+[Service]
+Restart=always
+ExecStartPre=-/usr/bin/docker kill %p-%i
+ExecStartPre=-/usr/bin/docker rm %p-%i
+ExecStart=/usr/bin/docker run \
+  --name %p-%i \
+  -v /data/runtime/domains/%i/%p/www-content:/app \
+  indiehosters/nginx
+ExecReload=/usr/bin/docker restart %p-%i
+ExecStop=/usr/bin/docker stop %p-%i
+
+[Install]
+WantedBy=multi-user.target

unit-files/static-importer@.service

+[Unit]
+Description=static importer
+
+# Dependency ordering
+Before=static@%i.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/bin/bash -euxc ' \
+  folder=/data/domains/%i/static/www-content; \
+  mkdir -p $folder; \
+  if [ ! -f $folder/index.html ]; then \
+    echo Hello %i > $folder/index.html; \
+  fi;'
+
+[Install]
+WantedBy=static@%i.service

unit-files/nginx@.service → unit-files/static@.service

@@ -5,7 +5,6 @@ Description=%p-%i
 Requires=docker.service
 Requires=%p-importer@%i.service
 Requires=%p-discovery@%i.service
-Requires=%p-gitpuller@%i.path
 Requires=backup@%i.timer
 
 # Dependency ordering
@@ -19,7 +18,7 @@ ExecStartPre=-/usr/bin/docker kill %p-%i
 ExecStartPre=-/usr/bin/docker rm %p-%i
 ExecStart=/usr/bin/docker run \
   --name %p-%i \
-  -v /data/domains/%i/%p/data/www-content:/app \
+  -v /data/domains/%i/%p/www-content:/app \
   indiehosters/nginx
 ExecReload=/usr/bin/docker restart %p-%i
 ExecStop=/usr/bin/docker stop %p-%i