Skip to content
Hide whitespace changes
Inline Side-by-side
data/server-wide/postfix/forwards deleted 100644 → 0
View file @ 7b39ee0c
/.*@somedomain.com/ person1@gmail.com, person2@hotmail.com
/michiel@somewhere-else.net/ me@forward.net
data/server-wide/postfix/runme.sh deleted 100755 → 0
View file @ 7b39ee0c
docker rm postfix
docker run -d --name postfix -v $(pwd):/data -p 25:25 indiehosters/postfix-forwarder
deploy/add-placeholder-site.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
if [ $# -ge 3 ]; then
SERVER=$1
DOMAIN=$2
PEMFILE=$3
else
echo "Usage: sh ./deploy/add-placeholder-site.sh server domain pemfile [user]"
exit 1
fi
if [ $# -ge 4 ]; then
USER=$4
else
USER="core"
fi
echo "Adding $DOMAIN to $SERVER with cert from $PEMFILE"
echo "Remote user is $USER"
ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/static
scp $PEMFILE $USER@$SERVER:/data/runtime/haproxy/approved-certs/$DOMAIN.pem
ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN static
deploy/add-site.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
if [ $# -ge 5 ]; then
SERVER=$1
DOMAIN=$2
PEMFILE=$3
IMAGE=$4
GITREPO=$5
else
echo "Usage: sh ./deploy/add-site.sh server domain pemfile image gitrepo [user]"
exit 1
fi
if [ $# -ge 6 ]; then
USER=$6
else
USER="core"
fi
echo "Adding $DOMAIN to $SERVER, running $IMAGE behind $PEMFILE and pulling from $GITREPO"
echo "Remote user is $USER"
ssh $USER@$SERVER sudo mkdir -p /data/domains/$DOMAIN/$IMAGE/data
scp $PEMFILE $USER@$SERVER:/data/runtime/haproxy/approved-certs/$DOMAIN.pem
ssh $USER@$SERVER sudo sh /data/indiehosters/scripts/activate-user.sh $DOMAIN $IMAGE $GITREPO
deploy/approve-certs.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
for i in `deploy/list-sites.sh $1`; do
echo "Approving combined cert for $i";
cp ../orchestration/TLS/combined/$i.pem ../orchestration/TLS/approved-certs/$i.pem;
scp ../orchestration/TLS/approved-certs/$i.pem root@$1:/data/runtime/haproxy/approved-certs/
done
deploy/check-cert.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
if [ $# -eq 2 ]; then
CA=$2
else
CA="startssl"
fi
echo "CA is $CA"
echo Some information about cert ../orchestration/TLS/cert/$1.cert:
openssl x509 -text -in ../orchestration/TLS/cert/$1.cert | head -50 | grep -v ^\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
#echo Some information about chain cert ../orchestration/TLS/chain/$2.pem:
#openssl x509 -text -in ../orchestration/TLS/chain/$2.pem
#echo Some information about key ../orchestration/TLS/key/$1.key:
#openssl rsa -text -in ../orchestration/TLS/key/$1.key
cat ../orchestration/TLS/cert/$1.cert ../orchestration/TLS/chain/$CA.pem ../orchestration/TLS/key/$1.key > ../orchestration/TLS/combined/$1.pem
echo Running a test server on port 4433 on this server now \(please use your browser to check\):
openssl s_server -cert ../orchestration/TLS/combined/$1.pem -www
deploy/deploy.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
if [ $# -ge 2 ]; then
SERVER=$1
BACKUP_DEST=$2
else
echo "Usage: sh ./deploy/deploy.sh server backup_dest [branch [user]]]"
exit 1
fi
if [ $# -ge 3 ]; then
BRANCH=$3
else
BRANCH="master"
fi
if [ $# -ge 4 ]; then
USER=$4
else
USER="core"
fi
echo "Server to deploy is $SERVER"
echo "Backups will live under $BACKUP_DEST"
echo "IndieHosters repo branch is $BRANCH"
echo "Remote user is $USER"
scp ./deploy/onServer.sh $USER@$SERVER:
ssh $USER@$SERVER sudo mkdir -p /var/lib/coreos-install/
scp cloud-config $USER@$SERVER:/var/lib/coreos-install/user_data
ssh $USER@$SERVER sudo sh ./onServer.sh $BRANCH $SERVER
# overrides BACKUP_DESTINATION from cloud-config
echo $BACKUP_DEST > ./deploy/tmp.txt
scp ./deploy/tmp.txt $USER@$SERVER:/data/BACKUP_DESTINATION
rm ./deploy/tmp.txt
deploy/list-sites.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
cd ../orchestration/per-server/$1/sites
for i in *; do
echo $i
done
deploy/onServer.sh deleted 100755 → 0
View file @ 7b39ee0c
#!/bin/sh
#Usage from deploy/deploy.sh:
#ssh $USER@$SERVER sudo sh ./onServer.sh $BRANCH $SERVER
echo Starting etcd:
/usr/bin/coreos-cloudinit --from-file=/var/lib/coreos-install/user_data
echo Cloning the indiehosters repo into /data/indiehosters:
mkdir -p /data
cd /data
git clone https://github.com/indiehosters/indiehosters.git
cd indiehosters
echo Checking out $1 branch of the IndieHosters indiehosters:
git checkout $1
git pull
echo Running the server setup script:
sh scripts/setup.sh $2
doc/adding-an-application.md deleted 100644 → 0
View file @ 7b39ee0c
# Adding an application
There are two types of application: server-wide, and per-user. Right now, server-wide applications are the postfix-forwarder,
and the haproxy. Available per-user applications right now are wordpress, static, and static-git.
# Adding a server-wide application
To add a server-wide application, first make sure it can run on one IPv4 address for multiple domains. If it can, then:
* Package it into one Dockerfile per process, and add these to the [dockerfiles](https://github.com/indiehosters/dockerfiles) repo.
* Add `docker pull`, `systemctl enable` and `systemctl start` commands to `scripts/setup.sh`.
* Make it take user data from /data/PROCESS/, and data that is needed at runtime but should not be backed up can
go into /data/runtime/PROCESS
* Create a systemd unit file for each process in the `unit-files/` folder
* Either add functionality to `scripts/add-site.sh` to configure domains on this new service, or describe the manual steps for this
in [deploying-a-server.md](deploying-a-server.md).
* Double-check all documentation to see if it is all still correct.
# Adding a per-user application
To add a per-user application, first make sure it can run behind an SNI offloader. For instance, for WordPress to work behind an SNI
offloader, we had to activate the "https" plugin. If it can, then:
* First study how for instance the WordPress application works in this repo, the easiest approach is to copy it as a starting point.
* Package it into one Dockerfile per process, and add these to the [dockerfiles](https://github.com/indiehosters/dockerfiles) repo.
* Add `docker pull` commands to `scripts/setup.sh`.
* Make it take user data from /data/domains/DOMAIN.COM/PROCESS/, and data that is needed at runtime but should not be backed up can
go into /data/runtime/domains/DOMAIN.COM/PROCESS.
* Create a systemd unit file for each process in the `unit-files/` folder, ending in "@.service", so they get symlinked for each domain.
* If there are cronjobs to be added, then this is done with "@.timer" files.
* Link the processes together, using docker linking and systemd dependencies.
* Add import logic, and if there is any pre-backup action necessary to make sure all relevant data is under /data/domain, then add this as
a pre-step to the backup unit.
* Either add functionality to `scripts/add-site.sh` to configure domains on this new service, or describe the manual steps for this
in [deploying-a-server.md](deploying-a-server.md).
* Double-check all documentation to see if it is all still correct.
doc/architecture-overview.md deleted 100644 → 0
View file @ 7b39ee0c
# Architecture based on systemd, docker, haproxy, and some bash scripts
Our architecture revolves around a
[set of systemd unit files](https://github.com/indiehosters/indiehosters/tree/master/unit-files). They come in various types:
## Server-wide processes
The haproxy.* and postfix.* unit files correspond to two server wide processes. They run Docker containers from images in the
[server-wide/ folder of our dockerfiles repo](https://github.com/indiehosters/dockerfiles/tree/master/server-wide).
The haproxy-confd service unit starts configuration service for haproxy. It monitors `etcdctl ls /services` to see if any new backends were created, and updates the haproxy configuration files. The files lives in `/data/runtime/haproxy/` on the host sytem. It is required by the haproxy.* unit. That means that when you run `systemctl start haproxy`, and then run `docker ps` or `systemctl list-units`, you will see that systemd not only started the haproxy container, but also the haproxy-confd container.
There is currently no similar service for updating `/data/runtime/postfix/`, so you will have to update the configuration files in that folder manually, and then run `systemctl restart postfix`.
Etcd is configured in the `cloud-config` file. The `scripts/setup.sh` takes care of enabling and starting the haproxy and postfix service, and the haproxy-confd to listen for changes in the backends configuration in [etcd](https://github.com/coreos/etcd). New backends are automatically added to the haproxy configuration as soon as their private docker IP address is written into etcd.
## HAProxy backends: static, static-git, wordpress
A per user process is a haproxy backend for a specific domain name. For now, we have three applications available: static, static-git and wordpress. But we are working on adding more. Please, if you want, you can create more :)
You will notice there are also some other units in the `unit-files/` folder of this repository, like the gitpuller and mysql ones.
Whenever you start a wordpress unit, it requires a mysql service.
Whenever you start a static-git unit, it wants a static-git-puller unit.
In all three cases, an -importer unit and a -discovery unit are required.
This works through a
[`Requires=` directive](https://github.com/indiehosters/indiehosters/blob/0.1.0/unit-files/nginx@.service#L6-L7) which systemd interprets, so that if you start one service, its dependencies are also started (you can see that in `systemctl list-units`).
## Discovery
The `-discovery` units check find out the local IP address of the backend, checks if it is up by doing a `curl`, and if so, writes the IP address into etcd. The `haproxy-confd` service notices this, and update the haproxy config.
## Import
The `-import` units check if data exists, and if it doesn't, tries to import from the `/data/import` folder, and create initial data state, for instance by doing a git clone, untarring the content of a vanilla wp-content for wordpress installation.
Note that some initialization is also done by the Docker images themselves - for instance the wordpress image runs a [shell script](https://github.com/pierreozoux/tutum-docker-wordpress-nosql/blob/master/run-wordpress.sh) at container startup, that creates the initial mysql database if it didn't exist yet.
We are working on merging everything inside the docker image.
## Gitpuller
The `-gitpuller` unit is scheduled to run every 10 minutes by the .timer file. When it runs, it does a git pull to update the website content at one of the haproxy backends from the git repository mentioned in the GITURL file.
## Scripts
There is one important script you can run at your server. You can also run the commands they contain manually, then you just use them as a cheatsheet of how to [set up a new server](https://github.com/indiehosters/indiehosters/tree/master/scripts/setup.sh).
There are also deploy scripts which do the same from a jump box, so you can orchestrate multiple servers from one central vantage points. They are in the
[deploy/](https://github.com/indiehosters/indiehosters/tree/master/deploy)
folder of this repo, and they are the scripts referred to in the 'how to deploy a server' document. They basically run the scripts from the scripts/ folder over ssh.
doc/deploying-a-server.md deleted 100644 → 0
View file @ 7b39ee0c
# Deploying a server
## Before you start
Make sure you read [getting started](getting-started-as-a-hoster.md) first.
### Prepare your servers
#### with CoreOS
* Get 2 CoreOS server, for instance from [RackSpace](rackspace.com), [Vultr](vultr.com), or [Hetzner](http://serverboerse.de/).
* let's call them k1 and k2
* they will be backup of each other
* Modify the cloud-config according to your needs
* make sure the backup user get the ssh public key of the root of the other server
#### other linuxes
* * If you prefer another operating system, you can also run our Docker images [using just Docker and bash](using-just-docker-and-bash.md).
* If you didn't add your public ssh key during the order process (e.g. through your IaaS control panel or a cloud-config file),
scp your laptop's public ssh key (probably in `~/.ssh/id_rsa.pub`) to `.ssh/authorized_keys` for the remote user
you will be ssh-ing and scp-ing as (the default remote user of our deploy scripts is 'core').
* Give the new server a name (in this example, we call the server 'k3')
* Add k3 to your /etc/hosts with the right IP address
* If you have used this name before, run `ssh-keygen -R k3`
* Ssh into your server, and run `ssh-keygen -t rsa` (use all the default settings, empty passphrase)
* Set up a backups server at an independent location (at least a different data center, but preferably also a different IaaS provider, the bu25 plan of https://securedragon.net/ is a good option at 3 dollars per month).
* Set up a git server by following http://www.git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server (no need to set up any repos like 'project.git' yet). Let's call the backup server 'bu25' (add this to /etc/hosts on k3).
* Add the ssh key from k3:.ssh/id_rsa.pub to the authorized_keys for the git user (not the root user) on bu25.
* Check that you can `ssh git@bu25` from k3.
* Exit from the double ssh back to your laptop, and from the root folder of this repository, run `sh ./deploy/deploy.sh k3 git@bu25 master root`
* The rest should be automatic! (ignore the warning about backup.dev, and note that haproxy will not start as long as there are no websites on your server).
### Adding an existing website
* The IndieHosters architecture is migration-oriented, so it aims to make moving a domain from one server to another very easy.
* If you already have a domain in backups, say example.com, then it's easy to add it to this new server.
* Say domain example.com runs the 'static' image. Then ssh into k3, and run:
````bash
systemctl enable static@example.com
systemctl start static@example.com
````
* This will automatically do the following things:
* Clone the backup repo from bu25
* Set up an hourly backup job for the user data (which will live in `/data/domains/example.com` on k3)
* Start an nginx container
* Note its IP address in etcd
* Rewrite the haproxy configuration
* (Re)start haproxy
* If the domain runs wordpress (and similarly if it runs Known), then:
* the git clone from backup will fail because of [issue 46](https://github.com/indiehosters/indiehosters/issues/46), so make sure to follow the workaround steps from there.
* initial wordpress setup will be impossible with the redirect because of [Dockerfiles issue 39](https://github.com/indiehosters/dockerfiles/issues/39), so make sure to follow the workaround steps from there.
* Indiewebifying the WordPress instance is still a manual process, so make sure you know the user's preferred username (you can use this for the SiteName as well), and preferred language.
* Configure an arbitrarily long password, and save it in wordpress/login.txt in /data/domains/domain.com/wordpress/login.txt. The user will be logging in with IndieAuth, but it's good to make a not of which password you set.
* Activate the IndieWeb plugins/themes/widget as per [Dockerfiles issue 40](https://github.com/indiehosters/dockerfiles/issues/40).
* Make sure to edit the files in /data/runtime/postfix/ to set up email forwarding. Also, ask another indiehoster to add the same forwarding and act as a fallback MX.
### Adding a new website to your server
* For each site you want to deploy on the server, e.g. example.com, do the following:
* Does example.com already exist as a domain name?
* If yes, then find out to what extent it's currently in use (and needs to be migrated with care). There are a few options:
* Transfer the domain into your DNR account
* Set up DNS hosting for it and ask the owner to set authoritative DNS to the DNS servers you control
* Ask the user to keep DNR and DNS control where it is currently, but to switch DNS when it's ready at the new server, and every time
you add or remove an IP address (not a good idea, unless the user insists that they prefer this option)
* In any case, you will probably need access to the hostmaster@example.com email address, for the StartSSL process *before*
the final DNS switch. You could also ask them to tell you the verification code that arrives there, but that has to be done
in real time, immediately when you click 'verify' in the StartSSL UI. If they forward the email the next day, then the token
will already have expired.
* If no, register it (at Namecheap or elsewhere).
* Decide which image to run as the user's main website software (in version 0.2.2, 'static', 'static-git', 'wordpress', and 'known' are supported)
* For the 'wordpress' image, all you need is the TLS certificate. Use the 'static-git' image if you already have some static
content that should go on there, and which you can put in a public git repository somewhere.
* Unless you already have a TLS certificate for example.com, get one
(from StartSSL if you own the domain in question, from namecheap or elsewhere if you do not), and concatenate the certificate
and its unencrypted private key into one file.
* Make sure the TLS certificate is valid (use `scripts/check-cert.sh` for this), and scp it to `/data/import/example.com/TLS/example.com.pem` on k3.
* Now ssh into k3, and if for instance 'wordpress' is the image you chose, run:
systemctl enable wordpress@example.com
systemctl start wordpress@example.com
* In case you're going for the 'static-git' repo, store the git url with the content in `/data/domains/example.com/static-git/GITURL`.
* In case you're going for the 'static' repo, store the html content under `/data/domains/example.com/static/www-content`.
* Test the site using your /etc/hosts. You should see the data from the git repo, or the static content, or a wordpress start page
on both http and https.
* If all looks well, switch DNS and monitoring.
* If not, check what happened by looking at what's in `/data/domains/example.com`, `data/runtime/domains/example.com`, and `/data/runtime/haproxy` on k3. Note that this part of our scripts is currently a bit complex, we will clean this up in a next version. There are six different scripts that try to initialize the contents of `/data/domains/example.com`:
* The git clone from the back up service (will try to initialize an empty git repository)
* The local data import process (will try to move the data from `/data/import/example.com` into place
* The wordpress image (which we used from the wordpress-stackable Dockerfile published by Tutum)
* The mysql image (which we used from the mysql Dockerfile published by Tutum)
* The wordpress importer (a one-time systemd task)
* The mysql importer (a one-time systemd task)
* It might help to remove /data/domains/example.com, /data/runtime/domains/example.com, and /etc/systemd/system/*/*example.com*, and remove the git@bu25:example.com repo from the backup server, make sure /data/import/example.com/TLS/example.com.pem exists, make sure /data/BACKUP_DESTINATION contains git@bu25 (and not core@backup.dev), shutdown -r now, and retry.
* If you're setting up a fresh wordpress site, you have to access the admin panel over http first (e.g. run it `-p 80:80`), and activate the [https plugin](https://wordpress.org/plugins/wordpress-https/) before it will work behind the haproxy SSL-offloader.
doc/developing-dockerfiles-and-infrastructure.md deleted 100644 → 0
View file @ 7b39ee0c
# Developing Dockerfiles and infrastructure
## Developing Dockerfiles
To develop Dockerfiles, you can use a server that's not serving any live domains, use `docker` locally on your laptop, or use the `vagrant up` instructions to run the infrastructure inside vagrant.
## Developing infrastructure
To develop the infrastructure, create a branch on this repo and specify that branch at the end of the deploy command, for instance:
```bash
sh ./deploy/deploy.sh k4 dev
```
That will deploy a server at whatever IP address "k4" points to in your /etc/hosts, using the "dev" branch of https://github.com/indiehosters/indiehosters.
## Testing new Dockerfiles in the infrastructure
To test the infrastructure with a changed Dockerfile, you need to take several steps:
* Develop the new Dockerfiles as described above at "Developing Dockerfiles"
* When you're happy with the result, publish this new Dockerfile onto the docker hub registry under your own username (e.g. michielbdejong/haproxy-with-http-2.0)
* Now create a branch on the infrastructure repo (e.g. "dev-http-2.0")
* In this branch, grep for the Dockerfile you are updating, and replace its name with the experimental one everywhere:
* the `docker pull` statement in scripts/setup.sh
* the `docker run` statement in the appropriate systemd service file inside unit-files/
* Push the branch to the https://github.com/indiehosters/indiehosters repo (if you don't have access to that, you will have to edit
`deploy/onServer.sh` to use a different repo, to which you do have access).
* Now deploy a server from your experimental infrastructure branch (which references your experimental Docker image), as described above at "Developing infrastructure"
doc/getting-started-as-a-hoster.md deleted 100644 → 0
View file @ 7b39ee0c
Getting started as an IndieHosters hoster
===========
# Prerequisites
Each IndieHoster is an entirely autonomous operator, without any infrastructural ties to other IndieHosters.
These scripts and docs will help you run and manage servers and services as an IndieHoster, whether you're
certified as a branch of the IndieHosters franchise or not.
# Hoster data
If you're used to working with git as a versioning tool, then it's a good idea to make `hoster-data` and
`billing` into (private!) git repos where you keep track of what you're doing, including e.g. TLS certificates, so
that you can track changes over time, and search the history to resolve mysteries when they occur. You may also use a different
versioning system, or just take weekly and daily backups (but then it's probably a good idea to retain the weeklies for a couple
of years, and even then it will not be as complete as a history in a versioning system).
The hoster-data is about what each one of your servers *should* be doing at this moment.
This is fed into CoreOS (systemd -> etcd -> confd -> docker) to make sure the server actually starts and keeps doing these things,
and also into monitoring, to make sure you get alerted when a server misbehaves.
You probably also want to keep track of Domain Name Registration, Transport
Layer Security, Monitoring, and Domain Name System services which you are probably getting from
third-party service providers, alongside the services which
you run on your own servers.
Note that although it's probably inevitable that you resell DNR and TLS services from some third party, and your monitoring would ideally
also run on a system that's decoupled from your actual servers, you may not be reselling DNS
hosting. If you host DNS for your customer on server-wide bind services that directly read data from files on the per domain data folders,
then DNS data will be considered user-data for you.
# User data
User data is data owned by one of your users. Which human owns which site is something you can administer
by hand somehow in the `billing` folder.
All user data is *untrusted* from your point of view, it is not owned by you as a hoster,
and users may change it at any time (and then probably contact you for a backup whenever they mess up!). It makes sense to give users
only read-only access to this data by default, and have a big "Are you sure? Warranty will be void!" warning before they can activate
write-access to their own data (and then probably trigger an extra backup run just before allowing them to edit their own raw data).
This is how some operating systems on user devices also deal with this.
But in the end, the user, and not you, owns this data, and they can do with it what they want, at their own risk.
Just like a mailman is not supposed to open and read letters, you also should treat each user's data as a closed envelope
which you never open up, unless in the following cases:
* There may be things you need to import from specific files on there (like a user-supplied TLS certificate or DNS zone)
* When running backups, you sometimes can't avoid seeing some of the modified filenames flying by (depending on the backup software)
* After explicit permission of the user, when this is useful for tech support (e.g. fix a corrupt mysql database for them)
In version 0.1 no user data exists because the TLS cert is part of the hoster-data, and so are the secondary email address to forward
to and the git repository to pull the website content from. We don't need to back up users' websites, because they are already versioned
and backed up in the git repository from which we're pulling it in.
# Backups
Your user-data, hoster-data, and billing folders together contain all the critical data
of your operations as an IndieHoster, from start to finish, so make sure you don't
ever lose it, no matter what calamity may strike. Once a month, put a copy of it on a USB stick, and put that in a physically safe place.
You may give a trusted person an emergency key to your infrastructure, in case you walk under a bus. Think about the risk of data loss and
establish an emergency recovery plan for when, for instance, the hard disk of your laptop or of one of your servers die.
Make sure you often rsync the live data from each of your servers to somewhere else, and store snapshots of it
regularly. Users *will* contact you sooner or later asking for "the backup from last Tuesday"
and they will expect you to have one.
# Basic digital hygiene
At the same time, be careful who may obtain access to your critical data. Is your laptop really safe? Does the NSA have access to the servers you run?
Someone may plant a Trojan on a computer in an internet cafe from where you access your Facebook account, access your gmail account
for which you use the same password, reset your RackSpace password and restore a backup from your Cloud Files to somewhere else.
Make a diagram of how your laptop talks to your USB sticks and your servers. Then make a diagram of the services you use and to which
email addresses they send password reset emails. Draw a perimeter of trust in both diagrams, and start taking some basic measures to
keep your daily operations secure.
Don't mix accounts and email addresses which you may
use from other computers, and keep your IndieHosters passwords and accounts separate from your other passwords and accounts, and reset
them every few months. It might even
make sense to dual-boot your laptop or boot from a live disk which resets on boot to make sure everything you do with IndieHosters data
is done in a sterile environment. Ubuntu has a 'guest account' option on the login screen which maybe also be handy for this.
Also: lock your screen when walking away from your laptop, and think about what someone could do with it if they were to steal your bag,
or your smartphone.
# Do I have to use this?
As an IndieHoster you can of course use any infrastructure and scripts you want, as long as it doesn't change the format of each user-data folder, so that
your customers can still migrate at will between you and other IndieHosters. However, you might find some of the scripts in this repo
helpful at some point, and we hope you contribute any features you add to it back upstream to this repo.
Thanks for taking the time to read through these general considerations - the next topic is [deploying a server](deploying-a-server.md)! :)
doc/migration-format.md deleted 100644 → 0
View file @ 7b39ee0c
# IndieHosters migration format
## Deprecated
Deprecated by the [web app migration procedure](../proc/webapp.md)
## Version 0.2.2 (deprecated)
When a user exports their data for domain.com, they get a zip or tar file that contains different files, depending on which application is
running on their domain:
### If using the 'static' application
* TLS/domain.com.pem - Concatenation of the unencrypted private and public key of the TLS certificate, and intermediate CA cert if applicable.
* static/www-content - static content to be placed in the web root
### If using the 'static-git' application
* TLS/domain.com.pem - Concatenation of the unencrypted private and public key of the TLS certificate, and intermediate CA cert if applicable.
* static-git/GITURL - git url to pull the static website content from
### If using the 'WordPress' application
* TLS/domain.com.pem - Concatenation of the unencrypted private and public key of the TLS certificate, and intermediate CA cert if applicable.
* mysql/dump.sql - the dump of all their MySQL databases
* mysql/.env - contains the MySQL password
* wordpress/.env - contains the MySQL password
* wordpress/login.txt - username and password for the WordPress admin panel
* wordpress/.htaccess - htaccess file for WordPress
* wordpress/wp-content - php files to be placed in the web root
### If using the 'Known' application
* TLS/domain.com.pem - Concatenation of the unencrypted private and public key of the TLS certificate, and intermediate CA cert if applicable.
* mysql/dump.sql - the dump of all their MySQL databases
* mysql/.env - contains the MySQL password
* known/ - php files to be placed in the web root
* known/.env - contains the MySQL password
* known/login.txt - email address and password for the Known admin panel
doc/migration-procedure.md deleted 100644 → 0
View file @ 7b39ee0c
# Migration procedure version 0.3
The IndieHosters network aims to allow users to easily migrate their hosted services from one hosting provider to another.
To this goal, we describe a number of services which a hosting provider may offer, and a migration procedure for each of these.
In this document, we will say the "user" migrates "services" from the "old hoster" to the "new hoster".
We distinguish two types of migration: full migrations, and partial migrations. A full migration includes the migration of
the domain name registration, DNS hosting, and all other hosted applications from the old hoster to the new hoster, and the user
will no longer be a customer of old hoster once the full migration is complete.
In a partial migration, only some services are migrated, and others are not. For instance, the IndieHosters product "Known hosting 0.3"
consists of:
* a domain name registration at a well-known registrar
* DNS hosting
* email forwarding
* a redirect from http to https on port 80
* a TLS certificate on port 443
* version 0.6.5-mysql of the Known application running behind that
If the old hoster offers this product, but the new hoster does not offer email forwarding, then only a partial migration is
possible. The user will then have to accept that their email forwarding will stop working. Presumably, the user is OK with
that, since they picked the new hoster themselves. But it's worth mentioning that this is then only a partial migration.
Migration procedures:
* [Domain name registration](../proc/dnr.md)
* [DNS hosting](../proc/dns.md)
* [Email forwarding](../proc/email.md)
* [TLS certificate](../proc/tls.md)
* [Web application](../proc/webapp.md)
doc/using-just-docker-and-bash.md deleted 100644 → 0
View file @ 7b39ee0c
# Using just Docker and bash
## WARNING: Still a work-in-process
Given that CoreOS is not available everywhere, and the Ubuntu 14.10 setup with etcdctl inside a Docker instance still has some problems,
I'll try if I can run all our services (postfix-forwarder, haproxy, and the various web backend containers) on an off-the-shelf Ubuntu server.
Note that not all servers support Docker, because of kernel modules etcetera; of the images I tried at Gandi, only the Ubuntu 12.04-64 one
allowed me to actually run `docker ps`, and even on there, I was not able to run `docker run debian apt-get update` because from the looks
of it, containers are not allowed to contact the outside world. In the end I got a Debian 7.6 server at Linode.
Note that about 5GB of disk space will be used, and when running multiple sites, 2Gb of RAM is probably also well-spent.
Here's what I did to prepare the server:
````bash
apt-get update && apt-get -y upgrade
apt-get -y install unattended-upgrades git
dpkg-reconfigure -plow unattended-upgrades
# set unattended upgrades to 'Yes'
ssh-keygen -t rsa
# select all the defaults by hitting <enter> repeatedly
# Install Docker:
curl -sSL https://get.docker.com/ | sh
# Install docker-enter ([recommended](https://github.com/jpetazzo/nsenter#nsenter-in-a-can)):
docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter
# In this case I also had to:
/etc/init.d/exim stop
````
Test your server by running `docker run debian apt-get update` (there should be no 'could not resolve' errors).
Then I added the .ssh/id_rsa.pub to .ssh/authorized_keys at both backup server accounts, and ran:
````
git clone git@bu25:postfix
cd postfix; ./runme.sh; cd ..
git clone git@bu25:haproxy
cd haproxy; ./runme.sh; cd ..
````
These 'postfix' and 'haproxy' repos with their `runme.sh` files look like the samples in the `data/` folder of this repo.
And then for each domain I host:
````
git clone git@bu25:michielbdejong.com
cd michielbdejong.com; ./runme.sh; cd ..
````
These domain repos look quite similar to the IndieHosters migration format, except that there is a `runme.sh` file in there,
as follow:
````
docker rm michielbdejong.com
docker run -d --restart=always --name michielbdejong.com -v $(pwd):/data indiehosters/lamp-git
docker inspect -f {{.NetworkSettings.IPAddress}} michielbdejong.com
````
And furthermore, the dump.sql file includes the 'mysql' database as well as the 'known' database, so that the local 'root' user
is created, and the php files of Known 0.6.5 are included (will iron out that difference when switching from generic lamp-git
image to specific Known image).
Now the only thing you need to be careful with is to start haproxy *after* starting the backends, otherwise the container linking
doesn't work. This means reboots are not actually safe, and also, at the time of writing, hourly backups are not working yet in the
lamp-git image, so be aware that this is a work in progress. :)
The idea of this setup would be to eventually support the migration format in a programmatic way.
dockerfiles/email/postfix/Dockerfile deleted 100644 → 0
View file @ 7b39ee0c
FROM debian:jessie
MAINTAINER Michiel de Jong <michiel@indiehosters.net>
# Borrows from https://docs.docker.com/articles/dockerfile_best-practices/
# Borrows from https://registry.hub.docker.com/u/previousnext/postfix
# Borrows from https://registry.hub.docker.com/u/catatnight/postfix
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update \
&& echo "postfix postfix/main_mailer_type string 'Internet site'" | debconf-set-selections \
&& echo "postfix postfix/mailname string 'HOSTNAME.EXAMPLE.COM'" | debconf-set-selections \
&& echo "postfix postfix/root_address string 'ROOTMAIL@EXAMPLE.COM'" | debconf-set-selections \
&& apt-get install -q -y \
postfix \
rsyslog \
supervisor \
&& rm -rf /var/lib/apt/lists/*
COPY assets/install.sh /opt/install.sh
VOLUME ["/etc/postfix", "/var/spool/mail", "/var/log"]
EXPOSE 25
CMD /opt/install.sh;/usr/bin/supervisord -c /etc/supervisor/supervisord.conf