Commit 2eb2c97a authored by Pierre Ozoux's avatar Pierre Ozoux

Updates documentation

parent 11cf094c
## IndieHosters
This repository contains the confd and bash scripts we use to control our servers.
It can run inside Vagrant (see below; FIXME: check whether these instruction currently work) or
This repository contains the configuration and scripts we use to control our servers.
It can run inside Vagrant or
[deploy to a server](doc/getting-started-as-a-hoster.md) (FIXME: update those instructions to
prescribe less folder structure, explain static https+smtp hosting, and check if they currently
work).
......@@ -10,8 +10,8 @@ work).
- [vagrant](http://www.vagrantup.com/)
- [virtualbox](https://www.virtualbox.org/)
- nfs
- run `apt-get install nfs-kernel-server`, or your OS equivalent
- optional: [vagrant-hostsupdater](https://github.com/cogitatio/vagrant-hostsupdater)
- linux: run `apt-get install nfs-kernel-server`, or your OS equivalent
- [vagrant-hostsupdater](https://github.com/cogitatio/vagrant-hostsupdater)
- run `vagrant plugin install vagrant-hostsupdater` to install
## Get started:
......@@ -20,15 +20,19 @@ work).
vagrant up
```
Wait for the provisioning to finish (~5mins), and go to your browser: https://indiehosters.dev
Wait for the provisioning to finish (~12mins), and go to your browser: https://indiehosters.dev
If the process fails, for instance due to network problems, you can retry by running `vagrant provision`.
### If you want to add another static-git instance apart from indiehosters.dev:
### If you want to add another static instance apart from indiehosters.dev:
```bash
vagrant ssh
sudo sh /data/indiehosters/scripts/activate-user.sh example.dev static-git https://github.com/indiehosters/website.git
sudo mkdir -p /data/import/example.dev/TLS
sudo cp /data/indiehosters/scripts/unsecure-certs/example.dev.pem /data/import/example.dev/TLS
sudo systemctl start static@example.dev
```
Check https://example.dev in your bowser!
### Cleaning up
......@@ -39,3 +43,14 @@ To clean up stuff from previous runs of your VM, you can do:
vagrant destroy
vagrant up
```
## Tests
```bash
vagrant destroy
vagrant up
# Check in your browser https://indiehosters.dev
vagrant ssh
sudo su
/data/indiehosters/scripts/tests/main.sh
```
......@@ -6,38 +6,45 @@ Our architecture revolves around a
The haproxy.* and postfix.* unit files correspond to two server wide processes. They run Docker containers from images in the
[server-wide/ folder of our dockerfiles repo](https://github.com/indiehosters/dockerfiles/tree/master/server-wide).
The haproxy-confd.* unit starts a side-kick service for haproxy, which monitors `etcdctl ls /services` to see if any new backends were created, and updates the haproxy configuration, which lives in `/data/runtime/haproxy/` on the host sytem. It is required by the haproxy.* unit. That means that when you run `systemctl start haproxy`, and then run `docker ps` or `systemctl list-units`, you will see that systemd not only started the haproxy container, but also the haproxy-confd container.
The haproxy-confd service unit starts configuration service for haproxy. It monitors `etcdctl ls /services` to see if any new backends were created, and updates the haproxy configuration files. The files lives in `/data/runtime/haproxy/` on the host sytem. It is required by the haproxy.* unit. That means that when you run `systemctl start haproxy`, and then run `docker ps` or `systemctl list-units`, you will see that systemd not only started the haproxy container, but also the haproxy-confd container.
There is currently no similar side-kick for updating `/data/runtime/postfix/`, so you will have to update the configuration files in that folder manually, and then run `systemctl restart postfix`.
There is currently no similar service for updating `/data/runtime/postfix/`, so you will have to update the configuration files in that folder manually, and then run `systemctl restart postfix`.
The `scripts/setup.sh` takes care of setting up etcd, enabling and starting the haproxy and postfix service (as well as one haproxy backend, to serve the default site), and the haproxy-confd side-kick to listen for changes in the backends configuration in etcd, so that new backends are automatically added to the haproxy config as soon as their IP address is written into etcd.
Etcd is configured in the `cloud-config` file. The `scripts/setup.sh` takes care of enabling and starting the haproxy and postfix service, and the haproxy-confd to listen for changes in the backends configuration in [etcd](https://github.com/coreos/etcd). New backends are automatically added to the haproxy configuration as soon as their private docker IP address is written into etcd.
## HAProxy backends: static, static-git, wordpress
A per user process is a haproxy backend for a specific domain name. At the time of writing we have three applications available: static, static-git and wordpress.
A per user process is a haproxy backend for a specific domain name. For now, we have three applications available: static, static-git and wordpress. But we are working on adding more. Please, if you want, you can create more :)
You will notice there are also some other units in the `unit-files/` folder of this repo, like the gitpuller and mysql ones. Whenever you start a wordpress unit, it requires a mysql service.
Whenever you start a static-git unit, it wants a static-git-puller unit. In all three cases, an -importer unit and a -discovery unit are required.
You will notice there are also some other units in the `unit-files/` folder of this repository, like the gitpuller and mysql ones.
Whenever you start a wordpress unit, it requires a mysql service.
Whenever you start a static-git unit, it wants a static-git-puller unit.
In all three cases, an -importer unit and a -discovery unit are required.
This works through a
[`Requires=` directive](https://github.com/indiehosters/indiehosters/blob/0.1.0/unit-files/nginx@.service#L6-L7) which systemd interprets, so that if you start one service, its dependencies are also started (you can see that in `systemctl list-units`).
## Discovery
The -discovery units check find out the local IP address of the backend, checks if it is up by doing a `curl`, and if so, writes the IP address into etcd. The `haproxy-confd` process notices this, and update the haproxy config.
The `-discovery` units check find out the local IP address of the backend, checks if it is up by doing a `curl`, and if so, writes the IP address into etcd. The `haproxy-confd` service notices this, and update the haproxy config.
## Import
The -import units check if data exists, and if it doesn't, create initial data state, for instance by doing a git clone, untarring the php files of a virgin wordpress installation, or both. So -import is actually a misnomer, probably -init would have been a better name.
The `-import` units check if data exists, and if it doesn't, tries to import from the `/data/import` folder, and create initial data state, for instance by doing a git clone, untarring the content of a vanilla wp-content for wordpress installation.
Note that some initialization is also done by the Docker images themselves - for instance the wordpress image runs a [shell script](https://github.com/pierreozoux/tutum-docker-wordpress-nosql/blob/master/run-wordpress.sh) at container startup, that creates the initial mysql database if it didn't exist yet.
We are working on merging everything inside the docker image.
## Gitpuller
The -gitpuller unit is scheduled to run every 10 minutes by the .timer file, and is configure to only run if the GITURL file exists at the path specified in the .path file. When it runs, it does a git pull to update the website content at one of the haproxy backends from the git repository mentioned in the GITURL file.
The `-gitpuller` unit is scheduled to run every 10 minutes by the .timer file. When it runs, it does a git pull to update the website content at one of the haproxy backends from the git repository mentioned in the GITURL file.
## Scripts
There are two important scripts you can run at your server. You can also run the commands they contain manually, then you just use them as a cheatsheet of how to [set up a new server](https://github.com/indiehosters/indiehosters/tree/master/scripts/setup.sh) or [activate a new user](https://github.com/indiehosters/indiehosters/tree/master/scripts/activate-user.sh), respectively.
There is one important script you can run at your server. You can also run the commands they contain manually, then you just use them as a cheatsheet of how to [set up a new server](https://github.com/indiehosters/indiehosters/tree/master/scripts/setup.sh).
There are also deploy scripts which do the same from a jump box, so you can orchestrate multiple servers from one central vantage points. They are in the
[deploy/](https://github.com/indiehosters/indiehosters/tree/master/deploy)
......
......@@ -37,7 +37,7 @@ Make sure you read [getting started](getting-started-as-a-hoster.md) first.
* Unless you already have a TLS certificate for example.com, get one
(from StartSSL or elsewhere), and concatenate the certificate
and its unencrypted private key into one file.
* Make sure the TLS certificate is valid (use `scripts/check-cert.sh` for this), and scp it to `/data/import/example.com.pem` on k3.
* Make sure the TLS certificate is valid (use `scripts/check-cert.sh` for this), and scp it to `/data/import/example.com/TLS/example.com.pem` on k3.
* Now ssh into k3, and if for instance 'wordpress' is the image you chose, run:
systemctl enable wordpress@example.com
......
#!/bin/bash -eux
if [ $# -ge 2 ]; then
DOMAIN=$1
IMAGE=$2
else
echo "Usage: sh /data/indiehosters/scripts/activate-user.sh domain image [gitrepo]"
exit 1
fi
mkdir -p /data/domains/$DOMAIN/$IMAGE
if [ $# -ge 3 ]; then
GITREPO=$3
echo $GITREPO > /data/domains/$DOMAIN/$IMAGE/GITURL
fi
# Start service for new site (and create the user). This will also enable the git puller.
systemctl enable $IMAGE@$DOMAIN.service
systemctl start $IMAGE@$DOMAIN.service
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment