Skip to content
Commits on Source (122)
# 1.2.0
* Add app admin email as argument in provision #189
* Handle git branches when provisioning #174
* REPO mode to retrieve application recipe #187
# 0.3.0
* adds automation script for user provisionning
......
# Instructions to install libre.sh on linux with Systemd
## Recommendation
- Systemd distro (ubuntu server 18.04.3 or debian 9 )
# Installation
Where basicly reproduce what the user_data do for us.
as root
# configure sshd (Optional)
Don't forget to create the user core and adding your ssh key before
You could also remove AllowUsers core or/and change the username.
```
cat > /etc/ssh/sshd_config <<EOF
UsePrivilegeSeparation sandbox
Subsystem sftp internal-sftp
PermitRootLogin no
AllowUsers core
PasswordAuthentication no
ChallengeResponseAuthentication no
EOF
chmod 600 /etc/ssh/sshd_config
systemctl restart sshd
```
# add kernel parameter (optional but recommended )
```
cat > /etc/sysctl.d/libresh.conf <<EOF
fs.aio-max-nr=1048576
vm.max_map_count=262144
vm.overcommit_memory=1
EOF
chmod 644 /etc/sysctl.d/libresh.conf
sysctl -p /etc/sysctl.d/libresh.conf
echo never > /sys/kernel/mm/transparent_hugepage/enabled
```
# define Localhost (should not be needed but... )
```
cat > /etc/hosts <<EOF
127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost
EOF
```
# define envrionment
```
cat > /etc/environment <<EOF
NAMECHEAP_URL="namecheap.com"
NAMECHEAP_API_USER="pierreo"
NAMECHEAP_API_KEY=
IP="curl -s http://icanhazip.com/"
FirstName="Pierre"
LastName="Ozoux"
Address=""
PostalCode=""
Country="Portugal"
Phone="+351.967184553"
EmailAddress="pierre@ozoux.net"
City="Lisbon"
CountryCode="PT"
BACKUP_DESTINATION=root@xxxxx:port
MAIL_USER=
MAIL_PASS=
MAIL_HOST=mail.indie.host
MAIL_PORT=587
EOF
```
# install docker
*Current tested version : 19.03.5 see https://docs.docker.com/install/linux/docker-ce/ubuntu/ .*
# install docker-compose
*Remark I did a variante to find the last version of DockerCompose and download it*
```
mkdir -p /opt/bin &&\
dockerComposeVersion=$(curl -s https://api.github.com/repos/docker/compose/releases/latest|grep tag_name|cut -d'"' -f4) &&\
curl -L https://github.com/docker/compose/releases/download/$dockerComposeVersion/docker-compose-`uname -s`-`uname -m` > /opt/bin/docker-compose &&\
chmod +x /opt/bin/docker-compose
```
# install Libre.sh
```
git clone https://lab.libreho.st/libre.sh/compose.libre.sh /libre.sh &&\
mkdir -p /{data,system} &&\
mkdir -p /data/trash &&\
cp /libre.sh/unit-files/* /etc/systemd/system && systemctl daemon-reload &&\
systemctl enable web-net.service &&\
systemctl start web-net.service &&\
mkdir -p /opt/bin &&\
cp /libre.sh/utils/* /opt/bin/
```
# add /opt/bin path
```
cat > /etc/profile.d/libre.sh <<EOF
export PATH=$PATH:/opt/bin
EOF
chmod 644 /etc/profile.d/libre.sh
```
# libre.sh
# libre.sh Version 1.2
[![Backers on Open Collective](https://opencollective.com/libresh/backers/badge.svg)](#backers)
[![Sponsors on Open Collective](https://opencollective.com/libresh/sponsors/badge.svg)](#sponsors)
## Introduction
A PaaS that is aimed at hosting free software \o/
An ecosystem to ease free software hosting \o/
To install it, follow instructions in `INSTALL.md`.
We are working on bootstrapping an ecosystem of tools to facilitate the hosting of free software.
Think of it as
- [ISPconfig](https://www.ispconfig.org/)
- FLOSS [cpanel](https://www.cpanel.net/products/)
- [cloudron](https://cloudron.io/) with email
* Libre.sh V1 (Stable) is using docker-compose
* Libre.sh V2 (Alpha) is using [kubernetes](https://kubernetes.io/).
This ecosystem can be deployed on [Raspberries](https://kubecloud.io/setting-up-a-kubernetes-1-11-raspberry-pi-cluster-using-kubeadm-952bbda329c8) or on popular cloud providers and scale globally or anything in between.
We can affirm that V2 scales globally because it is based on kubernetes, a tool developped from the experience of Google hosting containers at scale.
## Installation
To install it, follow the instructions in `INSTALL_LINUX.md` : https://lab.libreho.st/libre.sh/compose.libre.sh/blob/master/INSTALL_LINUX.md
Or run our installer script
https://lab.libreho.st/libre.sh/compose.libre.sh/raw/master/install.linux.sh
### What is libre.sh
......@@ -30,7 +54,7 @@ Amazing, right?
The PaaS is really modular, that's why it contains the strict necessary, then you'll probably want to add `system` modules or `applications`.
It contains 2 [unit-files](https://github.com/indiehosters/LibrePaaS/tree/master/unit-files) to manage system modules and applications, start them at boot, and load the appropriate environment.
It contains 2 [unit-files](https://lab.libreho.st/libre.sh/compose.libre.sh/tree/master/unit-files) to manage system modules and applications, start them at boot, and load the appropriate environment.
### Support
......@@ -40,19 +64,18 @@ You can use the following channels to request community support:
For paid support, just send an inquiry to support@libre.sh.
You can also watch the Fosdem Video : [Video Fosdem](https://fosdem.org/2017/schedule/event/libre_sh/)
All of this is hosted by libre.sh :)
## System modules
Here is a list of modules supported:
- https proxy:
- [HAProxy](https://github.com/indiehosters/haproxy)
- [Nginx](https://github.com/indiehosters/nginx)
- [logs](https://github.com/indiehosters/logs)
- [monitoring](https://github.com/indiehosters/monitoring)
- [git-puller](https://github.com/indiehosters/git-puller)
- [backups](https://github.com/indiehosters/backups)
- [sshd](https://github.com/indiehosters/sshd)
- [HAProxy](https://lab.libreho.st/libre.sh/compose/haproxy)
- [Nginx](https://lab.libreho.st/libre.sh/compose/nginx)
- [monitoring](https://lab.libreho.st/libre.sh/compose/monitoring)
- [git-puller](https://lab.libreho.st/libre.sh/compose/git-puller)
Go to their respective page for more details.
......@@ -60,7 +83,7 @@ Go to their respective page for more details.
```
cd /system/
git clone module
git clone https://lab.libreho.st/libre.sh/compose/[module]
cd module
libre enable
libre start
......@@ -68,15 +91,30 @@ libre start
## Applications
### List of supported applications
| Application | Latest Version | Comments |
|--------------|---------------------------|------------|
| wordpress | 5.9 | Includes the support of SMTP email though libresh variables |
| dolibarr | 15.0.3 | need manual deletion of the install.lock to upgrade |
### Installation
To install application `wordpress` on `example.org`, first make point example.org to your server IP, and then, just run:
```
libre provision -a github.com/indiehosters/wordpress -u example.org -s
libre provision -a wordpress -u example.org -s
```
Run `libre provision` for more details on the capabilities of the script.
- -u [arg] URL to process. Required.
- -a [arg] Application to install. (wordpress in REPO_MODE)
- -t [arg] Checkout a specific tag or branch from the application repo. default to master
- -e [arg] Specify the email of the application admin
- -s Start the application right away.
- -b Buys the associated domain name.
- -i Configure OpenDKIM.
- -c Configures DNS if possible.
## To debug a module or an application:
......@@ -100,3 +138,45 @@ You can help us by:
- writing documentation/blog on how you got up and running in 5mins
- writing more documentation
- sending us cake :) We loove cake!
## Contributors
This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)].
<a href="https://github.com/indiehosters/libre.sh/graphs/contributors"><img src="https://opencollective.com/libresh/contributors.svg?width=890&button=false" /></a>
## Backers
Thank you to all our backers! 🙏 [[Become a backer](https://opencollective.com/libresh#backer)]
<a href="https://opencollective.com/libresh#backers" target="_blank"><img src="https://opencollective.com/libresh/backers.svg?width=890"></a>
## Sponsors
Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/libresh#sponsor)]
<a href="https://opencollective.com/libresh/sponsor/0/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/0/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/1/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/1/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/2/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/2/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/3/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/3/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/4/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/4/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/5/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/5/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/6/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/6/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/7/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/7/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/8/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/8/avatar.svg"></a>
<a href="https://opencollective.com/libresh/sponsor/9/website" target="_blank"><img src="https://opencollective.com/libresh/sponsor/9/avatar.svg"></a>
## Other projects
Simplifying web application hosting has always been a goal for a lot of other projects, here is some project that share goals with libre.sh
- Yunohost https://yunohost.org
- Sandstorm https://sandstorm.io/
- Cloudron https://git.cloudron.io/cloudron/box
......@@ -12,7 +12,7 @@ The aim of this document is to write the big lines of the future of libre.sh.
# Version 1
The current version, let's call it 1, is a nice opiniated framework on how to run a single host with docker-compose.
It provides a list of packages and mofule compatible with this framework.
It provides a list of packages and module compatible with this framework.
The best features of this framework are:
- https only
- some integration between the tools (auto provisioning of emails for new applications)
......@@ -48,7 +48,8 @@ There are already some work done on it like the [ceph-docker](https://github.com
Another big challenge is network. k8s is strongly opiniated on what should be the network configuration.
Ideally, we would use some IPsec to secure the links between machine in a context we can't trust the network (like at hetzner).
There are 2 options:
There are 3 options:
- zerotier
- tinc vpn
- flannel that might implement IPsec in a near future
......
#!/bin/bash
#This script is tested on Debian 12
#Current version of libre.sh to be installed
LIBRE_VERSION=1.2
# System env vars : can be overrided by a values.env file next to this install file
### CONFIG : Specify you template repo ROOT without training slash (Optional) or comment if you want to supply full url for apps
APP_REPO_URL="lab.libreho.st/libre.sh/compose"
## domain handling
### CONFIG : change to your domain vendor ( namecheap, ovh , scaleway, )
DOMAIN_SERVER=namecheap
### Namecheap specific
NAMECHEAP_URL="namecheap.com"
NAMECHEAP_API_USER="pierreo"
NAMECHEAP_API_KEY=
### ovh specific (WIP)
OVH_URL="eu.api.ovh.com"
OVH_API_USER=""
OVH_API_KEY=
### Scaleway specific (WIP)
SCALEWAY_URL=""
SCALEWAY_API_USER=""
SCALEWAY_API_KEY=
### TODO : change your settings
IP="curl -s http://icanhazip.com/"
FirstName="Pierre"
LastName="Ozoux"
Address=""
PostalCode=""
Country="Portugal"
Phone="+351.967184553"
EmailAddress="pierre@ozoux.net"
City="Lisbon"
CountryCode="PT"
## Backup
BACKUP_DESTINATION=root@xxxxx:port
### CONFIG : Change your mail settings.
## SMTP
MAIL_USER=
MAIL_PASS=
MAIL_HOST=mail.indie.host
MAIL_PORT=587
MAIL_SECURITY=
# Default admin emails for apps
ADMIN_EMAIL=support@ekimia.fr
### TODO : source a setting file is present to override defaults
echo "-------- Welcome to libre.sh $LIBRE_VERSION installer"
echo "---- sourcing local values.env file if present"
source values.env
# STEP add kernel parameter
# STEP Define environnement
echo "-------- setting up system variables"
echo "APP_REPO_URL=${APP_REPO_URL}" >> /etc/environment
echo "LIBRE_VERSION=${LIBRE_VERSION}" >> /etc/environment
echo "MAIL_USER=${MAIL_USER}" >> /etc/environment
echo "MAIL_PASS=${MAIL_PASS}" >> /etc/environment
echo "MAIL_HOST=${MAIL_HOST}" >> /etc/environment
echo "MAIL_PORT=${MAIL_PORT}" >> /etc/environment
echo "MAIL_SECURITY=${MAIL_SECURITY}" >> /etc/environment
echo "ADMIN_EMAIL=${ADMIN_EMAIL}" >> /etc/environment
# STEP Install Docker
name="docker.io"
# TODO : Fix a version for docker ?
dpkg -s $name &> /dev/null
if [ $? -ne 0 ]
then
echo "$name not installed"
apt-get update
# curl -fsSL https://get.docker.com -o get-docker.sh
# sh get-docker.sh
apt install -y $name
echo "-------- Native docker installed "
else
echo "$name already installed"
fi
# STEP "install docker-compose"
echo "-------- Install native docker-compose "
# TODO : Fix a version for docker compose ?
#mkdir -p /opt/bin &&\
#dockerComposeVersion=$(curl -s https://api.github.com/repos/docker/compose/releases/latest|grep tag_name|cut -d'"' -f4) &&\
#curl -L https://github.com/docker/compose/releases/download/$dockerComposeVersion/docker-compose-`uname -s`-`uname -m` > /opt/bin/#docker-compose &&\
#chmod +x /opt/bin/docker-compose
apt install -y docker-compose
# STEP "install git"
echo "-------- Install git"
distro=$( ( lsb_release -ds || cat /etc/*release || uname -om ) 2>/dev/null | head -n1 | cut -d " " -f1)
if [[ "$distro" == "Ubuntu" || "$distro" == "Debian" ]]; then
apt-get install -y git
elif [[ "$distro" == "CentOS" || "$distro" == "AlmaLinux" || "$distro" == "Rocky" || "$distro" == "Fedora" ]]; then
yum install -y git
elif [[ "$distro" == "openSUSE" ]]; then
zypper install git
elif [[ "$distro" == "Arch" ]]; then
pacman -S git
elif [[ "$distro" == "Mageia" ]]; then
urpmi git
fi
# STEP install Libre.sh
echo " ---Removing previous install --- "
rm -rf /libre.sh
echo "-------- installing libre.sh"
git clone https://lab.libreho.st/libre.sh/compose.libre.sh.git /libre.sh
mkdir -p /{data,system}
mkdir -p /data/trash
mkdir -p /data/domains
cp /libre.sh/unit-files/* /etc/systemd/system && systemctl daemon-reload
systemctl enable web-net.service
systemctl start web-net.service
mkdir -p /opt/bin
cp /libre.sh/utils/* /opt/bin/
# STEP add /opt/bin path
echo "-------- updating PATH"
cat > /etc/profile.d/libre.sh <<EOF
export PATH=$PATH:/opt/bin
EOF
chmod 644 /etc/profile.d/libre.sh
bash /etc/profile.d/libre.sh
#TODO : reload profile to use libre right away
#!/bin/bash -eux
# Verify they are all in sync with git, if not, print the domain name.
for oo in `ls -d ./oo-*`;do
cd $oo
if ! git diff --exit-code --quiet; then
echo $oo
fi
cd ..
done
# Update all oo
for oo in `ls -d ./oo-*`;do
cd $oo
libre update
cd ..
done
[Service]
Type=oneshot
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
ExecStart=/opt/bin/dump_all.sh
[Unit]
Description=Run dump all dayly
[Timer]
OnCalendar=*-*-* 00:15:30
[Install]
WantedBy=timers.target
[Service]
Type=oneshot
ExecStart=/libre.sh/utils/mail-mon.sh
[Unit]
Description=Run mail mon hourly and on boot
[Timer]
OnBootSec=15min
OnUnitActiveSec=1h
[Install]
WantedBy=timers.target
......@@ -15,9 +15,9 @@ TimeoutStopSec=15
EnvironmentFile=-/system/%i/env
Environment=HOSTNAME=%H
WorkingDirectory=/system/%i/
ExecStartPre=-/opt/bin/docker-compose rm -f
ExecStart=/bin/bash -euxc "/opt/bin/docker-compose up"
ExecStop=/opt/bin/docker-compose stop
ExecStartPre=-docker-compose rm -f
ExecStart=/bin/bash -euxc "docker-compose up"
ExecStop=docker-compose stop
[Install]
WantedBy=multi-user.target
[Unit]
Description=Turn on swap
[Service]
Type=oneshot
RemainAfterExit=true
ExecStartPre=-/bin/bash -euxc ' \
fallocate -l 8192m /swap &&\
chmod 600 /swap &&\
mkswap /swap'
ExecStart=/sbin/swapon /swap
ExecStop=/sbin/swapoff /swap
[Install]
WantedBy=local.target
......@@ -15,9 +15,9 @@ TimeoutStopSec=15
EnvironmentFile=-/data/domains/%i/env
Environment=HOSTNAME=%H
WorkingDirectory=/data/domains/%i/
ExecStartPre=-/opt/bin/docker-compose rm -f
ExecStart=/bin/bash -euxc "VIRTUAL_HOST=%i /opt/bin/docker-compose up"
ExecStop=/opt/bin/docker-compose stop
ExecStartPre=-docker-compose rm -f
ExecStart=/bin/bash -euxc "LETSENCRYPT_HOST=%i VIRTUAL_HOST=%i,www.%i docker-compose up"
ExecStop=docker-compose stop
[Install]
WantedBy=multi-user.target
#cloud-config
ssh_authorized_keys:
- "PUT YOUR SSH KEY PUBLIC HERE"
write_files:
- path: /etc/ssh/sshd_config
permissions: 0600
owner: root:root
content: |
# Use most defaults for sshd configuration.
UsePrivilegeSeparation sandbox
Subsystem sftp internal-sftp
PermitRootLogin no
AllowUsers core
PasswordAuthentication no
ChallengeResponseAuthentication no
- path: /etc/sysctl.d/libresh.conf
permissions: 0644
owner: root
content: |
fs.aio-max-nr=1048576
vm.max_map_count=262144
vm.overcommit_memory=1
vm.nr_hugepages=0
- path: /etc/hosts
permissions: 0644
owner: root
......@@ -15,7 +31,7 @@ write_files:
255.255.255.255 broadcasthost
::1 localhost
- path: /etc/environment
permission: 0644
permissions: 0644
owner: root
content: |
NAMECHEAP_URL="namecheap.com"
......
......@@ -6,6 +6,6 @@ user=$1
email=$2
quota=$3
curl -X POST --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users -d userid="$1" -d password="`tr -dc A-Za-z0-9_ < /dev/urandom | head -c 10 | xargs`"
curl -X PUT --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${user} -d key="email" -d value="${email}"
curl -X PUT --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${user} -d key="quota" -d value="${quota}"
curl -X POST -H "OCS-APIRequest:true" --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users -d userid="$1" -d password="`tr -dc A-Za-z0-9_ < /dev/urandom | head -c 10 | xargs`"
curl -X PUT -H "OCS-APIRequest:true" --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${user} -d key="email" -d value="${email}"
curl -X PUT -H "OCS-APIRequest:true" --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${user} -d key="quota" -d value="${quota}"
......@@ -5,4 +5,4 @@ source /etc/environment
user=$1
group=$2
curl -X POST --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${1}/groups -d groupid="${2}"
curl -X POST -H "OCS-APIRequest:true" --user ${cloud_admin}:${cloud_pass} https://${cloud_hostname}/ocs/v1.php/cloud/users/${1}/groups -d groupid="${2}"
#!/bin/bash -eux
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin
echo BEGIN > /tmp/dump-logs
date > /tmp/dump-logs
for domain in `ls /data/domains`; do
echo $domain >> /tmp/dump-logs
cd /data/domains/${domain}/
if [[ -f ./scripts/pre-backup ]]; then
cat ./scripts/pre-backup >> /tmp/dump-logs
./scripts/pre-backup >> /tmp/dump-logs
fi
done
date > /tmp/dump
echo END >> /tmp/dump-logs
date >> /tmp/dump-logs
......@@ -29,6 +29,21 @@ function systemctl_param {
fi
}
function show_usage {
echo "Usage:"
echo " - provision -a <app_repo_url> -u <domainname> -s : install and start a libre.sh service."
echo " - start|status|enable|disable|restart|stop: command sent to systemctl."
echo " - ps|exec|logs: command sent to docker compose."
echo " - update: to update the current folder."
echo " - stats: show docker stats with names."
echo " - delete <domainname>: remove a libre.sh service."
echo " - getsize <domainname>: give you the size of the installed application"
exit 1
}
if [ $# -eq 0 ]; then
show_usage
fi
case "$1" in
start|status|enable|disable|restart|stop)
......@@ -61,10 +76,38 @@ case "$1" in
provision ${@:2};;
stats)
docker stats $(docker ps|grep -v "NAMES"|awk '{ print $NF }'|tr "\n" " ");;
delete)
if [ $# -ne 2 ]; then
echo "delete requires a domainname argument."
exit 1
fi
read -p "Are you sure you want to delete ${2}? (yN)" -n 1 -r
echo # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
cd /data/domains/${2}
libre stop
libre disable
cd /data/domains
tar cvzf ${2}.tgz ./${2}
if [ -f ./${2}.tgz ]; then
if [ ! -d /data/trash/ ]; then
mkdir /data/trash/
fi
mv ${2}.tgz /data/trash/
rm -rf /data/domains/${2}
rm -rf /system/haproxy/certs/${2}
rm /system/haproxy/haproxy/certs/${2}.pem
fi
fi;;
getsize)
# check the current size
if [ $# -ne 2 ]; then
echo "getsize requires a domainname argument."
exit 1
fi
echo $(du -hs /data/domains/${2}) |cut -d ' ' -f 1;;
*)
echo "Usage:"
echo " - start|status|enable|disable|restart|stop: command sent to systemctl."
echo " - ps|exec|logs: command sent to docker compose."
echo " - update: to update the current folder."
echo " - stats: show docker stats with names."
show_usage
esac
#!/bin/bash -eux
/usr/bin/journalctl --since '1 hour ago' | grep postfix/cleanup | grep "from=" | sed -n 's/.*from=<\(.*\)> to=<\(.*\)> pro.*/\1/p' | sort |uniq -c | sort | awk 'BEGIN {FS=" ";} {printf "mail_sent{domain_name=\"%s\"} %s\n", $2, $1}' > /system/metrics-collection/textfiles/mail.prom