Backups of the live site (moved)

This is a page about improving or modifying OpenHatch.

We call that "Hacking OpenHatch," and there is a whole category of pages about that.

Overview

We have a free, donated account from rsync.net that lets us store 50GB of data there.

We use duplicity (as per the rsync.net official document). We do full backups weekly and incrementals daily.

We encrypt these backups.

Details

We use this script to run backups. It runs via root's crontab, and emails the results to Asheesh daily. (FIXME: The script should email monitoring-private AT lists.openhatch.org instead.)

#!/bin/bash

set -e
set -x

MODE="incr"
DAY_OF_WEEK_AS_NUMBER="$(date +%u)"
if [ "$DAY_OF_WEEK_AS_NUMBER" -eq 6 ] ; then
    MODE="full"
fi

### Prepare mysql snapshots
for db in $(find /var/lib/mysql/ -mindepth 1 -maxdepth 1 -type d | sed 's,/var/lib/mysql/,,')
do
    mysqldump -uroot --single-transaction -p"$(cat /root/passwords/mysql-root-password)" "$db" | gzip | sponge /var/backups/mysql/"$db".sql.gz
done

function do_backup() {
    TARGET="backups/linode.openhatch.org/$1"
    LOCAL_PATH="$2"
    ssh rsync.net mkdir -p "$TARGET"
    duplicity $MODE --exclude /tmp --exclude /var/lib/mysql --exclude /usr/share/locale --exclude /usr/share/doc --exclude /var/tmp --exclude /var/cache --exclude-other-filesystems --encrypt-key="A5CC321E" "$LOCAL_PATH" scp://rsync.net/"$TARGET"
}

do_backup "all" "/"

Restoring

Untested as yet. http://openhatch.org/bugs/issue530 describes that

duplicity has a built-in "verify" feature, which checksums the data, but that doesn't help us ensure that our backup was complete.

More info about encryption

This backup is encrypted with a GPG key that has been emailed to hello @openhatch.org on Thu, Jan 26.