clean up

README.md

 ## IndieHosters
 
 This repository contains the configuration and scripts we use to control our servers.
-It can run inside Vagrant or
-[deploy to a server](doc/getting-started-as-a-hoster.md) (FIXME: update those instructions to
-prescribe less folder structure, explain static https+smtp hosting, and check if they currently
-work).
+It can run inside Vagrant or [deploy to a server](doc/getting-started-as-a-hoster.md)
 
 ## Prerequisites to running this code with Vagrant:
 - [vagrant](http://www.vagrantup.com/)

data/server-wide/haproxy/approved-certs/indiehosters.dev.pem

------BEGIN CERTIFICATE-----
-MIIFljCCA34CCQDXgLjASWHpmDANBgkqhkiG9w0BAQUFADCBjDELMAkGA1UEBhMC
-UFQxETAPBgNVBAgTCFBvcnR1Z2FsMQ8wDQYDVQQHEwZMaXNib24xFTATBgNVBAoT
-DEluZGllSG9zdGVyczEZMBcGA1UEAxMQaW5kaWVob3N0ZXJzLmRldjEnMCUGCSqG
-SIb3DQEJARYYY29udGFjdEBpbmRpZWhvc3RlcnMubmV0MB4XDTE0MTAxMDE0MzY1
-NVoXDTE1MTAxMDE0MzY1NVowgYwxCzAJBgNVBAYTAlBUMREwDwYDVQQIEwhQb3J0
-dWdhbDEPMA0GA1UEBxMGTGlzYm9uMRUwEwYDVQQKEwxJbmRpZUhvc3RlcnMxGTAX
-BgNVBAMTEGluZGllaG9zdGVycy5kZXYxJzAlBgkqhkiG9w0BCQEWGGNvbnRhY3RA
-aW5kaWVob3N0ZXJzLm5ldDCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB
-AKBOylYEoL1P3q7skTJsRA8yQj6fVHWHS3kPg6tcVavZawc6tRxIiDc41/EWjL7i
-Owb6io2UbKaD/g8695CFER9FvcW1iukrC/tUV5/AVd0SDcvS3RnGUndKh82HCNrM
-rUDU/XH8smEpfjuXrq0YPuiGbY1zSLQKirjYTiasJODfGkxSbobNfjdL7aEo+3HX
-BQq5mGIj9A4PYmeyFGkHCN8tRvf4lY1KfPJoWtDL4kmO4SFNZ4FAehH9AJ6vTN8y
-MFcHtFzpp2636TYTBQsLu48nrKs6MqOOyU0R/Ufw9QjiWDLo3Co6pcCTmVf16skO
-odg9BNdEhMXefpiEE1NOL6ZOkSUG5WSY0Q5Il649QcJOYzw2A0Nk3IOxoIexXat4
-siCgSlNfgyRmBn5HNcZo5aEDf9+3gEqFzEFSyH3ClIApC7RePbpPvsCAgpagBOXC
-PgO2w2VW9HfNHkwpF3Yqn7cqw0FQKwKREufVdnSvs9fgFlMZnqA3sMym8o99Fcvq
-WBaTuh54ePfNGmawPt1N8vUZUYXXOasWKmnjfan3S1rsNAf5M2ntLqEJRDwihdSm
-ZSO+B51hDO5jzHoqxHwA71CwUAp4hRO83xR6ziB1KR2834I/7LBzbpZ0EWm9adez
-8V+dwgBhTt0LYEUGLJN22XRi9d4RPhnRJpSLPV/h0Fa/AgMBAAEwDQYJKoZIhvcN
-AQEFBQADggIBAFzYeGiomhKZW//aUM4V4RLMVIf0B4uixSMxZGQIUWVtYckmyG2N
-t8qNBHAQ3gl811NqnqestIQ4DpGkNQRCv/iDa5OwdLJHTOQUxajUE/1xmidHtpzR
-ReBZvW48k0dLEM2gmIrt7qQwqqecjlWjvSQlvJxYWrn6TBAkFL6Quu8gfoPK9/cE
-HG/aRQ0PCywGV20LSZ+J03LN7MlACClgVTB7dJuWIN0dNi7TsqpIupk11ZQ3ybBY
-WPQmLnIiCAijL69kBmBynLvJT5XDy2C4ChyzZ5Y73CXhgJwCqOZJwbO7Doig9PZQ
-yVLtui18W3uVQ7ZlIxCAQUeFzSkZf3/XNlr2FkP+efw4LLGH8kiKMsyKuoLuthO1
-1YrXvI0sjuDOxQwrlNQ2CLVANLBpUMH2U1aiYbA6iICSHr8ORAc84StgG9mFLeyN
-w32/04MGPvZfset8gRCOuvA2sLTjylqh0IpaPWlnT77neqOFtETtzJ+3UuOcdfnN
-t2bxqimHT8WhBB823WajWlLdXcc902e9LLhe9M1/bwOqFIIlKDqtCndjyXpe/qhA
-s0YB8TqJLxJQqvdnmYiBFfGrDTgNBpjt6AKJHRGd4xgsYsmQ3zLJ0Z8mNNQhlLf/
-osGXa2s/ZX7ernfvSDQIOB70gohCLFtBok0unyBJhtHxXmZ7UmpuIanx
------END CERTIFICATE-----
------BEGIN RSA PRIVATE KEY-----
-MIIJKAIBAAKCAgEAoE7KVgSgvU/eruyRMmxEDzJCPp9UdYdLeQ+Dq1xVq9lrBzq1
-HEiINzjX8RaMvuI7BvqKjZRspoP+Dzr3kIURH0W9xbWK6SsL+1RXn8BV3RINy9Ld
-GcZSd0qHzYcI2sytQNT9cfyyYSl+O5eurRg+6IZtjXNItAqKuNhOJqwk4N8aTFJu
-hs1+N0vtoSj7cdcFCrmYYiP0Dg9iZ7IUaQcI3y1G9/iVjUp88mha0MviSY7hIU1n
-gUB6Ef0Anq9M3zIwVwe0XOmnbrfpNhMFCwu7jyesqzoyo47JTRH9R/D1COJYMujc
-KjqlwJOZV/XqyQ6h2D0E10SExd5+mIQTU04vpk6RJQblZJjRDkiXrj1Bwk5jPDYD
-Q2Tcg7Ggh7Fdq3iyIKBKU1+DJGYGfkc1xmjloQN/37eASoXMQVLIfcKUgCkLtF49
-uk++wICClqAE5cI+A7bDZVb0d80eTCkXdiqftyrDQVArApES59V2dK+z1+AWUxme
-oDewzKbyj30Vy+pYFpO6Hnh4980aZrA+3U3y9RlRhdc5qxYqaeN9qfdLWuw0B/kz
-ae0uoQlEPCKF1KZlI74HnWEM7mPMeirEfADvULBQCniFE7zfFHrOIHUpHbzfgj/s
-sHNulnQRab1p17PxX53CAGFO3QtgRQYsk3bZdGL13hE+GdEmlIs9X+HQVr8CAwEA
-AQKCAgEAgDpF8sRE5ukqUHV+Nv0O+7DR+FFuN4x/PFjCk6GKDaodyGyXTgZenv1j
-Db9h2ZYQbSafCVy+A/v0jq42NG2cIo2gnLL4aEY8kU8HwAsTI4A7dNw4a1ONx0ng
-ku/+jzXFJ+S2ziS5cqrEBFryKBcKyugsXUbn0svT5sNuz9RGs3ECEialrkJVQVoE
-vDKR3p+Fsux+DZKAt3Zq2lNBrDkqSYpoCBXZWmlIxIXgjr9nRDt7rS3DK0ot2pGr
-m0LRlH8K17Kb/O4RNaj6bHyOPiWmY33yygwFUXr3XiSTmqYM+oxCzIYjBcxfpUjr
-EcbthOGlZ9h3NNHj+npcfRa4dpxF09c8gW2AVG+nXVhciZpcnLDZ5z/Nd/510axU
-0m0PlCPfh+3L5tiia9k7zlRxjyzER/GofNiJ6v8oo8YZFvhVdbBBQoGs8aadSLH9
-5Kf3fPwm8ZhmmOTVWbFJZul/3o0Ho3yFxMVMq86Qu8Pm+h6Q1Pn7yZsXMg/ECXP/
-/ErBaWA+zuBZkgCSbdZk58cxkN45PGWGkoHHACVUvCbG8IuYQ989JeCy5w01FgFV
-IXm4squNtWgyhLZgvkhl2Hnc4pR+iYJRgh+ouyv7nELQde7hpM6YJLLUpMfjo7r5
-lJyWasZtb9E4iEl4/JrdQYMJCDEyBfDN6sTKr1Ai2txjzQA4uOECggEBAM9LDpJ+
-RR+b1rdYgtS6VL5OR1bWUHSi1W9L8Xz20wSQGbRxfEJfWmSslOU0COXvA01eOxQ9
-OvHcWxISiHdiM3QxpYNtbsgATCQbsSgegMHpbaEgJPadEkUWxdWejbtpA1ypKmGg
-iFB5H5IIcz65wWNFC3g29wrXyBsRevi+K/PTbwOzOlad7AAcbuuHiv73wxi5xo1P
-i6IZfjgQMKzD9AJbACAAqyvg70XT+3vlIo5ABKOw1kLuejbNBaXd1af7OfVXReL7
-BGGJmG6IzI0qP9q7fX3Iq4Gx34Sf0TSomSyW4kxtsDMPXVURMU4ssxeshh0zYFsZ
-GQgsr36mOW5cvbkCggEBAMX5gJTrAW47GgObnQWtYIHRvYO0g7Ge1fN12VzHLiap
-3a3RfhEDTVKkiugO1GxRC1NY0tcDUwrUzS/00ovDZ/8dVqMHITFj6zfA8aX6vnzA
-TnoUWINawPxFBB6FrEuXyGIVbykinuvFyk+z/DzgKzL8X5MaLymYSV+eT+9jjLHO
-pJ37S86evkljq24Ow6KB1rKb8mMsk8GDZB4JalDdGWzlG1qJkHMg7ULkEHx2lDTW
-mcuHwRtMimFPCBGqH0i+p3O1IUkodJPNYbldrEfAkzRdD4lH9B+DNYBgxP4FWhY2
-d9DTHAGCa9ZV0HjnGgPOILRmV69+9yQhNhu5010qNDcCggEABq1VP9S/Z0A+z1MT
-i8SgvCyLUbm/h7JDC723fp34uBnoKg7JwN2PbNS+Sw+9BaMISTKy1nkOcAH4EQH1
-0Vqha6m5uh0JR3ny+erGbxNkdFqPhHQjnKn8j6snHjVoPVQpno94ZQKlwWnVYX/S
-LoAPQaJUtz+V/4xpzq1md6Kwib8SwVzBkU6u7mX8EKwiBwp2B1LcmWqphcQqc6XZ
-24bIUlcaDu3Wlag+LNKiNCByV4CqZZdpn2hNGXzLJMebfTizajqwbppFTtr+xPi1
-Fgr5WZNWfHm9RIU1PPFk7LxNisklau7RkSN6jyXpn6oC7s1I2KHyBZ0uWDwQPxUd
-nndwSQKCAQA/gmrdWwZ6djtCLQmSaKws+TvypFYbBPldwNCaEsubW6Lhv/LRQl3r
-xR1KlHdQyC757eS1VTuundW1LLTeYTFbhe3lHsRnM8ahfCQJOwcgvhBu2VgLy3Fd
-fEZ2BCvhlC+UR4wBhjm1KR5dsz+Xx9IT6SI/7oZysYfYRNEf2q+n2sK0a4lGH2ar
-5G16QQJBf6WAZsa7SfGcgqn7eMnCZytg456CzN6qEEYMz1z6kI+6450yzboFJ+i8
-jr3n7Mtcas0NMW4cKf477AcNkB9UZVLT2YbCY3LNKSpgpKqNUuozdgW51/+D/HLb
-r2vRXVHbJqUXOj2m7vQZgw34lwRXPtLBAoIBAChJgVltpcWKUWqltYXCQsdPPbb4
-DQMb4bb2vV2iON2kl+UlcCdhr0f5yWoAyKjs49lcHBN2Ny4zVR0vIu/IDeX47Fx7
-n0OfcFgcnqiqiFhXkWGcfU2JHq/q5tmk5M04aCgkFM8IyEsG6ZLoi849Km9r8quu
-VfclpJ6SsMGnWo/A2eIVP9GsfqRys9ZWKJ9inZRP5Lmx6pCZa12Mn6ey0h/kxOqh
-ruJQDdV0O4PsvZhTQFhahSVyNmSKnLguq3zsyBwKRsNI9TVXMv/hs0nnwfFgtBK1
-K61c7AL4+9dtAWEnuwqy/1srZEeBr/jgTqyFyr+GQFYUMuE/uXNKCDWlIRI=
------END RSA PRIVATE KEY-----

data/server-wide/haproxy/crt-list

-  
-/etc/haproxy/approved-certs/michielbdejong.com.pem michielbdejong.com
-/etc/haproxy/approved-certs/michielbdejong.com.pem www.michielbdejong.com
-  
-/etc/haproxy/approved-certs/otherdomain.com.pem otherdomain.com
-/etc/haproxy/approved-certs/otherdomain.com.pem www.otherdomain.com

data/server-wide/haproxy/haproxy.cfg

-global
-  maxconn 4096
-  user haproxy
-  group haproxy
-
-defaults
-  mode http
-  option forwardfor
-  option httpclose
-  option httplog
-  option dontlognull
-  retries 3
-  timeout connect 5000
-  timeout client 50000
-  timeout server 50000
-
-frontend https-in
-mode http
-  bind *:443 ssl crt-list /etc/haproxy/crt-list crt /etc/haproxy/approved-certs/default.pem
-  reqadd X-Forwarded-Proto:\ https
-    
-# otherdomain.com:
-  acl https_otherdomain.com hdr(host) -i otherdomain.com
-  acl https_otherdomain.com hdr(host) -i www.otherdomain.com
-  use_backend otherdomain.com if https_otherdomain.com
-    
-# michielbdejong.com:
-  acl https_michielbdejong.com hdr(host) -i michielbdejong.com
-  acl https_michielbdejong.com hdr(host) -i www.michielbdejong.com
-  use_backend michielbdejong.com if https_michielbdejong.com
-  
-frontend http-in
-  bind *:80
-  redirect scheme https code 301
-  
-# michielbdejong.com:
-backend michielbdejong.com
-  cookie SERVERID insert nocache indirect
-  server Server michielbdejong.com-backend:80 cookie Server
-  
-# otherdomain.com:
-backend otherdomain.com
-  cookie SERVERID insert nocache indirect
-  server Server otherdomain.com-backend:80 cookie Server

data/server-wide/haproxy/runme.sh

-docker rm haproxy
-docker run -d --restart=always --name haproxy -v $(pwd):/etc/haproxy -p 80:80 -p 443:443 \
-  --link michielbdejong.com:michielbdejong.com-backend \
-  --link otherdomain.com:otherdomain.com-backend \
-  indiehosters/haproxy

data/server-wide/postfix/destinations

-k2.michiel.indiehosters.net, indiehosters.net, 3pp.io

data/server-wide/postfix/forwards

-/.*@somedomain.com/ person1@gmail.com, person2@hotmail.com
-/michiel@somewhere-else.net/ me@forward.net

data/server-wide/postfix/hostname

-k2.michiel.indiehosters.net

data/server-wide/postfix/runme.sh

-docker rm postfix
-docker run -d --name postfix -v $(pwd):/data -p 25:25 indiehosters/postfix-forwarder

deploy/add-placeholder-site.sh

-#!/bin/sh
-if [ $# -ge 3 ]; then
-  SERVER=$1
-  DOMAIN=$2
-  PEMFILE=$3
-else
-  echo "Usage: sh ./deploy/add-placeholder-site.sh server domain pemfile [user]"
-  exit 1
-fi
-if [ $# -ge 4 ]; then
-  USER=$4
-else
-  USER="core"
-fi
-echo "Adding $DOMAIN to $SERVER with cert from $PEMFILE"
-echo "Remote user is $USER"
-
-ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/static
-scp $PEMFILE $USER@$SERVER:/data/runtime/haproxy/approved-certs/$DOMAIN.pem
-ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN static

deploy/add-site.sh

-#!/bin/sh
-if [ $# -ge 5 ]; then
-  SERVER=$1
-  DOMAIN=$2
-  PEMFILE=$3
-  IMAGE=$4
-  GITREPO=$5
-else
-  echo "Usage: sh ./deploy/add-site.sh server domain pemfile image gitrepo [user]"
-  exit 1
-fi
-if [ $# -ge 6 ]; then
-  USER=$6
-else
-  USER="core"
-fi
-echo "Adding $DOMAIN to $SERVER, running $IMAGE behind $PEMFILE and pulling from $GITREPO"
-echo "Remote user is $USER"
-
-ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/$IMAGE/data
-scp $PEMFILE $USER@$SERVER:/data/runtime/haproxy/approved-certs/$DOMAIN.pem
-ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN $IMAGE $GITREPO

deploy/approve-certs.sh

-#!/bin/sh
-for i in `deploy/list-sites.sh $1`; do
-  echo "Approving combined cert for $i";
-  cp ../orchestration/TLS/combined/$i.pem ../orchestration/TLS/approved-certs/$i.pem;
-  scp ../orchestration/TLS/approved-certs/$i.pem root@$1:/data/runtime/haproxy/approved-certs/
-done

deploy/check-cert.sh

-#!/bin/sh
-if [ $# -eq 2 ]; then
-  CA=$2
-else
-  CA="startssl"
-fi
-echo "CA is $CA"
-
-echo Some information about cert ../orchestration/TLS/cert/$1.cert:
-openssl x509 -text -in ../orchestration/TLS/cert/$1.cert | head -50 | grep -v ^\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 
-
-#echo Some information about chain cert ../orchestration/TLS/chain/$2.pem:
-#openssl x509 -text -in ../orchestration/TLS/chain/$2.pem
-
-#echo Some information about key ../orchestration/TLS/key/$1.key:
-#openssl rsa -text -in ../orchestration/TLS/key/$1.key
-
-cat ../orchestration/TLS/cert/$1.cert ../orchestration/TLS/chain/$CA.pem ../orchestration/TLS/key/$1.key > ../orchestration/TLS/combined/$1.pem
-
-echo Running a test server on port 4433 on this server now \(please use your browser to check\):
-openssl s_server -cert ../orchestration/TLS/combined/$1.pem -www

deploy/default-ca

-startssl

deploy/list-sites.sh

-#!/bin/sh
-cd ../orchestration/per-server/$1/sites
-for i in *; do
-  echo $i
-done

doc/using-just-docker-and-bash.md

-# Using just Docker and bash
-
-## WARNING: Still a work-in-process
-
-Given that CoreOS is not available everywhere, and the Ubuntu 14.10 setup with etcdctl inside a Docker instance still has some problems,
-I'll try if I can run all our services (postfix-forwarder, haproxy, and the various web backend containers) on an off-the-shelf Ubuntu server.
-Note that not all servers support Docker, because of kernel modules etcetera; of the images I tried at Gandi, only the Ubuntu 12.04-64 one
-allowed me to actually run `docker ps`, and even on there, I was not able to run `docker run debian apt-get update` because from the looks
-of it, containers are not allowed to contact the outside world. In the end I got a Debian 7.6 server at Linode.
-Note that about 5GB of disk space will be used, and when running multiple sites, 2Gb of RAM is probably also well-spent.
-Here's what I did to prepare the server:
-
-````bash
-apt-get update && apt-get -y upgrade
-apt-get -y install unattended-upgrades git
-
-dpkg-reconfigure -plow unattended-upgrades
-# set unattended upgrades to 'Yes'
-
-ssh-keygen -t rsa
-# select all the defaults by hitting <enter> repeatedly
-
-# Install Docker:
-curl -sSL https://get.docker.com/ | sh
-
-# Install docker-enter ([recommended](https://github.com/jpetazzo/nsenter#nsenter-in-a-can)):
-docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter
-
-# In this case I also had to:
-/etc/init.d/exim stop
-````
-
-Test your server by running `docker run debian apt-get update` (there should be no 'could not resolve' errors).
-
-Then I added the .ssh/id_rsa.pub to .ssh/authorized_keys at both backup server accounts, and ran:
-
-````
-git clone git@bu25:postfix
-cd postfix; ./runme.sh; cd ..
-git clone git@bu25:haproxy
-cd haproxy; ./runme.sh; cd ..
-````
-
-These 'postfix' and 'haproxy' repos with their `runme.sh` files look like the samples in the `data/` folder of this repo.
-
-And then for each domain I host:
-
-````
-git clone git@bu25:michielbdejong.com
-cd michielbdejong.com; ./runme.sh; cd ..
-````
-
-These domain repos look quite similar to the IndieHosters migration format, except that there is a `runme.sh` file in there,
-as follow:
-
-````
-docker rm michielbdejong.com
-docker run -d --restart=always --name michielbdejong.com -v $(pwd):/data indiehosters/lamp-git
-docker inspect -f {{.NetworkSettings.IPAddress}} michielbdejong.com
-````
-
-And furthermore, the dump.sql file includes the 'mysql' database as well as the 'known' database, so that the local 'root' user
-is created, and the php files of Known 0.6.5 are included (will iron out that difference when switching from generic lamp-git
-image to specific Known image).
-
-Now the only thing you need to be careful with is to start haproxy *after* starting the backends, otherwise the container linking
-doesn't work. This means reboots are not actually safe, and also, at the time of writing, hourly backups are not working yet in the
-lamp-git image, so be aware that this is a work in progress. :)
-
-The idea of this setup would be to eventually support the migration format in a programmatic way.