Posted by & filed under AWS/Linux/Unix/Devops stuff.

If you are interested in reproducing a backup scheme similar to Apple’s Time Machine on non OS X Linux/Unix machine, it is actually pretty easy to do so using a combination of rsync and cpio. You obviously won’t get the pretty space themed GUI (or any sort of GUI), but you will get as many complete snapshots as you like where hard links are used to account for files that haven’t changed in order to save a very significant amount of disk space. This is the same scheme used by Time Machine, although with this approach you can also have your source and destination machines be totally separate machines, even located in different physical locations so long as both are accessible via SSH.

If you are interested in reproducing a backup scheme similar to Apple’s Time Machine on non OS X Linux/Unix machine, it is actually pretty easy to do so using a combination of rsync and cpio. You obviously won’t get the pretty space themed GUI (or any sort of GUI), but you will get as many complete snapshots as you like where hard links are used to account for files that haven’t changed in order to save a very significant amount of disk space. This is the same scheme used by Time Machine, although with this approach you can also have your source and destination machines be totally separate machines, even located in different physical locations so long as both are accessible via SSH.

To do so, you just need to concoct a simple shell script such as the following:

SNAPSHOTS=12
let STARTSNAP=$SNAPSHOTS+1

cd /backup/serverbackup
for i in *; do

    for ((m=$STARTSNAP;m>=1;m--))
    do
        if [ $m == $STARTSNAP -a -d snapshots/$i.$m ]
        then
            #echo "deleting snapshots/$i.$m"
            rm -rf snapshots/$i.$m
        elif [ -d snapshots/$i.$m ]
        then
            let next=`expr $m+1`
            #echo "moving snapshots/$i.$m -> snapshots/$i.$next"
            mv snapshots/$i.$m snapshots/$i.$next
        fi
    done

    if [ $i != "snapshots" ]
    then
        #echo "creating snapshot $i.1"
        cd $i && find . -print | cpio --quiet -dpl ../snapshots/$i.1
        cd ..
        #echo "rsyncing $i"
        rsync -aHz --delete yourserver:/datadir/$i/ /backup/serverbackup/$i/
    fi
done

simply make the appropriate substitutions:

  • SNAPSHOTS – the number of snapshots to keep
  • /backup/serverbackup – the directory containing the backup on the local machine
  • any alterations necessary to the rsync command, the source and/or destination directories can be on the same or different machines
  • you can uncomment the lines starting with “echo” if you’d like to see what the script is crunching away at.

You will also have to create a “snapshots” directory in the directory you are backing up your data to (in this example /backup/serverbackup), the individual snapshots will be created in this directory. To restore from a snapshot simply copy the data out of one of the snapshot directories. To schedule this to run at regular intervals simply invoke this script via a cron job (or anacron job). If you do so you’ll want to leave the echo statements commented so that the user who owns this cron job does not get spammed with output messages in the form of email :)