Archive

Archive for the ‘Scripting’ Category

Downloading “Astronomy Picture of the Day” with subtitles

February 3rd, 2013 Comments off
Image Credit: NASA (www.nasa.gov), ESA (www.spacetelescope.org), and The Hubble Heritage Team (heritage.stsci.edu)

Image Credit: NASA (www.nasa.gov), ESA (www.spacetelescope.org), and The Hubble Heritage Team (heritage.stsci.edu)

Some time back I came across Astronomy Picture of the Day – some picture are really stunning. I wanted to have this as a wallpaper, but subtitled with the description (I still want to know what I am looking at!). So I wrote a Bash-script doing that for me.

It requires the following programs

  • wget
  • html2text
  • convert from “imagemagick”

be accessible within $PATH.

All directory references are relative to the directory it is in. It needs two directories “resized” and “subtitled”.

The script doesn’t download already downloaded pictures. When called with no parameter it downloads the latest, or you can also alternatively browse their website and pass the URL to it.

You can download the script here.

By the way, APOD also offers a calendar.

Categories: Scripting Tags:

Automatically learn Spam/Ham on your own IMAP server

January 30th, 2013 Comments off

There are these little things on your server, you set them up once – then a long time passes – and then you realize, it is really helping you a lots and (most importantly) it is still working.

One of these things for me is how I handle Spam/Ham on my IMAP server. I had initially the idea to automate this as much as possible and let the computer/machine do the work (and I don’t have to worry about it anymore).

I have a number of IMAP folders, obviously “INBOX”, “LearnAsHam”, “LearnAsSpam” and “Spam”. If I have an email in my “INBOX” what is spam I’ll move it into “LearnAsSpam”. It will be picked up by a process I’ll explain later and moved into “Spam”. If there is a false positive in “Spam” I move it to “LearnAsHam” and after a process picks it up it ends up in “INBOX” again.

And this “process” is basically fetchmail feeding it through sa-learn with “–spam” or “–ham” as its parameter.

The relevant shell commands I think I got from here. A bit of a biest, but they’ll work.

Categories: Scripting Tags:

kill -9 $$

June 26th, 2012 Comments off

I hate it when I keyed in a password on a shell prompt and pressed enter already.

This is short, easy to remember and doesn’t do nasty things like editing your history file…

PS: Works definitely with bash, dont know about others.

Categories: Scripting Tags:

The perfect backup

November 13th, 2010 11 comments

As far as I can tell, I have finished my backup-solution @ home. It is working fine for a couple of weeks and I am happy with it. I call it “Backup Robot”.

Main idea

The main aim of this idea is that I have something what backups all my devices and targets I have defined automatically. It determines the availability of a device and if available, it starts the agent for that particular device, what is backing up whatever’s necessary.

For my needs it is only necessary that a device is backed up once a day. If I try to run the backup again, it considers that and runs only those agent of devices whose backup is older than a day.

I have implemented the main script as a Perl Script, it was originally a Shell script, but that got too complicated. The agents are at the moment Shell scripts – they are effectively just a little bit more than a simple rsync.

An agent

I start with an explanation of the agent first. They are pretty dumb. When executed, they just do what they are defined to backup, ie. copy it to a central directory. The exitcode of any action what could error is logged and the final exitcode is determined in a sensible way. If there was no error, a date- and timestamp is set.

If the agent is just backing up some files, an rdiff-backup is ran at the end. So we have one directory containing all the files, and another one for rdiff-backup. So the rdiff-backup is just copying the actual filesystem directory onto the rdiff-backup-directory.

An agent needs to support two parameters: “check” and “age”.

  • “check” is testing whether the actual device/target is available.
  • “age” is returning the number of days when the last successful backup was.

The most prominent example is an agent that rsync’s or scp-r some files from a device onto our backup server. But there are also other things what it could do, like:

  • Retrieving backups of embedded devices (like a wireless router) with CURL.
  • Logging into a website and perform some actions.
  • Download electronic billing of various suppliers.
backuprobot.pl

This is pretty simple. It contains a definition which devices/targets need to be backed up. For each target/device it checks first when the last backup was run by calling the agent with the parameter “age”. If it is too old, then the availability is checked by “check”. And if that is fine, the agent is finally executed.

Everything is nicely logged in a file and in addition a very short email is compiled, stating what has been backed up and what was not backed up, like this:

SERVER has been backed up.
LAPTOP has been backed up with errors.

MOBILE is 3 days old.
(... output of the error following ...)

And here comes the big deal now. This backup is running at 3am every night. My wireless router, a Linksys WRT54GL with OpenWRT on it, is using WOL to wake up the backup server. backuprobot.pl is executed by an init.d script and backuprobot.pl itself  realizes it is running at 3am and as nobody is logged in, the system is  shut down, when it is finished.

Should the system already be running, backuprobot.pl is called with “manual” mode, what is effectively doing the same. The only difference is this can also be used during the day, and it only shuts down the system when nobody is logged in.

Another start parameter is “shutdown” at the moment, it is implemented, but I have never used it. Guess what it is doing.

I have a shortcut on my desktop for backuprobot.pl. This is starting it with parameter “gnome”, that means, meaningful desktop notifications are shown, when the script is walking the list with the devices/targets to be backed up.

Categories: Scripting Tags: