OpenHatch wiki

Backups of the live site (moved): Difference between revisions

Page Discussion

Backups of the live site (moved) (view source)

Revision as of 00:57, 18 August 2012

1,601 bytes removed ,  11 years ago
m
Mdaniel moved page Backups of the live site to Backups of the live site (moved)
imported>Paulproteus
imported>Mdaniel
 
(3 intermediate revisions by 2 users not shown)
Line 1:
{{Hacking OpenHatch}}
 
We are updating our documentation system. This page is now included in our project package, and is automatically generated by sphinx at openhatch.readthedocs.org [http://openhatch.readthedocs.org/en/latest/internals/backup.html Backups of the live site]
== Overview ==
 
We have a free, donated account from rsync.net that lets us store 50GB of data there.
 
We use duplicity (as per the rsync.net [http://www.rsync.net/resources/howto/duplicity.html official document]). We do full backups weekly and incrementals daily.
 
We encrypt these backups.
 
The only server essential to continued operation of the site is linode.openhatch.org. The other servers do unimportant things that do not keep state. It would be convenient to have backups for them, but it is not essential, so for now I suggest we simply skip it.
 
== Details ==
 
We use this script to run backups. It runs via root's crontab, and emails the results to Asheesh daily.
 
<pre>
#!/bin/bash
 
set -e
set -x
 
MODE="incr"
DAY_OF_WEEK_AS_NUMBER="$(date +%u)"
if [ "$DAY_OF_WEEK_AS_NUMBER" -eq 6 ] ; then
MODE="full"
fi
 
### Prepare mysql snapshots
for db in $(find /var/lib/mysql/ -mindepth 1 -maxdepth 1 -type d | sed 's,/var/lib/mysql/,,')
do
mysqldump -uroot --single-transaction -p"$(cat /root/passwords/mysql-root-password)" "$db" | gzip | sponge /var/backups/mysql/"$db".sql.gz
done
 
function do_backup() {
TARGET="backups/linode.openhatch.org/$1"
LOCAL_PATH="$2"
ssh rsync.net mkdir -p "$TARGET"
duplicity $MODE --exclude /tmp --exclude /var/lib/mysql --exclude /usr/share/locale --exclude /usr/share/doc --exclude /var/tmp --exclude /var/cache --exclude-other-filesystems --encrypt-key="A5CC321E" "$LOCAL_PATH" scp://rsync.net/"$TARGET"
}
 
do_backup "all" "/"
</pre>
 
== Restoring ==
 
Untested as yet. http://openhatch.org/bugs/issue530 describes that
 
duplicity has a built-in "verify" feature, which checksums the data, but that doesn't help us ensure that our backup was complete.
 
== More info about encryption ==
 
This backup is encrypted with a GPG key that has been emailed to hello @openhatch.org on Thu, Jan 26.
Anonymous user
Retrieved from "https://wiki.openhatch.org/wiki/Special:MobileDiff/8852...8856"