13 November 2010

Preface

As far as I can tell, I have finished my backup-solution @ home. It is working fine for a couple of weeks and I am happy with it. I call it "Backup Robot".

Main idea

The main aim of this idea is that I have something what backups all my devices and targets I have defined automatically. It determines the availability of a device and if available, it starts the agent for that particular device, what is backing up whatever's necessary.

For my needs it is only necessary that a device is backed up once a day. If I try to run the backup again, it considers that and runs only those agent of devices whose backup is older than a day.

I have implemented the main script as a Perl Script, it was originally a Shell script, but that got too complicated. The agents are at the moment Shell scripts - they are effectively just a little bit more than a simple rsync.

An agent

I start with an explanation of the agent first. They are pretty dumb. When executed, they just do what they are defined to backup, ie. copy it to a central directory. The exitcode of any action what could error is logged and the final exitcode is determined in a sensible way. If there was no error, a date- and timestamp is set.

If the agent is just backing up some files, an rdiff-backup is ran at the end. So we have one directory containing all the files, and another one for rdiff-backup. So the rdiff-backup is just copying the actual filesystem directory onto the rdiff-backup-directory.

An agent needs to support two parameters: "check" and "age".

  • "check" is testing whether the actual device/target is available.
  • "age" is returning the number of days when the last successful backup was.

The most prominent example is an agent that rsync's or scp-r some files from a device onto our backup server. But there are also other things what it could do, like:

  • Retrieving backups of embedded devices (like a wireless router) with CURL.
  • Logging into a website and perform some actions.
  • Download electronic billing of various suppliers.

backuprobot.pl

This is pretty simple. It contains a definition which devices/targets need to be backed up. For each target/device it checks first when the last backup was run by calling the agent with the parameter "age". If it is too old, then the availability is checked by "check". And if that is fine, the agent is finally executed.

Everything is nicely logged in a file and in addition a very short email is compiled, stating what has been backed up and what was not backed up, like this:

SERVER has been backed up.
LAPTOP has been backed up with errors.

MOBILE is 3 days old.
(... output of the error following ...)

And here comes the big deal now. This backup is running at 3am every night. My wireless router, a Linksys WRT54GL with OpenWRT on it, is using WOL to wake up the backup server. backuprobot.pl is executed by an init.d script and backuprobot.pl itself  realizes it is running at 3am and as nobody is logged in, the system is  shut down, when it is finished.

Should the system already be running, backuprobot.pl is called with "manual" mode, what is effectively doing the same. The only difference is this can also be used during the day, and it only shuts down the system when nobody is logged in.

Another start parameter is "shutdown" at the moment, it is implemented, but I have never used it. Guess what it is doing.

I have a shortcut on my desktop for backuprobot.pl. This is starting it with parameter "gnome", that means, meaningful desktop notifications are shown, when the script is walking the list with the devices/targets to be backed up.



blog comments powered by Disqus