[Novalug] A Backup Script

James Ewing Cottrell, III jecottrell3@comcast.net
Thu Jun 11 17:41:36 EDT 2015


Thanks for your Comments, but if you are seriously asking "happens if this month there is no 31?" then you didn't read the script.

As for Restore Verification, it's an rsync of the original; same Data, same Metadata.
As for Errors, well, you get Errors, the same kind you would normally get.

Fortunately, rsync handles not only Spaces, but Newlines in filenames. Haven't tested it with \N{pile of poo} yet.

Yes, cron as root, and rsync. As for logs, perhaps in addition to Mailing the output (which contains the command run) of the run to the Admins, it is saved as $TODAY.log.
And perhaps a df -h and a Success Message would be good at the end. Perhaps pruning the older backups would be good at the beginning.
Or maybe it can be self-pruning...if the rsync gets an error, prune, and then "exec $0 $*", but maybe only a few times.

This is run as a Pull Job from one of the Infrastructure Servers. It has the Same Permissions that the Original files have.
It could be run as Any User, but the files it could read would be limited, and that user would Own all the files.

JIM

----- Original Message -----
From: "Mark Smith" <mark@winksmith.com>
To: "James Ewing Cottrell, III" <jecottrell3@comcast.net>
Cc: "NoVA LUG" <novalug@firemountain.net>
Sent: Thursday, June 11, 2015 5:13:53 PM
Subject: Re: [Novalug] A Backup Script

if you're gonna use your own homebrew, one of the things you need to
verify is restoration.  pay attention to what happens around the edges
too, turn of the month?  what happens if this month there is no 31?
missing a backup that day? (it happens), unexpected characters in
filenames, error handling (e.g. disk out of space comes to mind), and
so on. sometimes it's useful to store meta data along with your files
(e.g. what command was run? when was it run?  who ran it?)  ah, and
authorization too... cron as root?  as jecottrell3?  is it protected
from spying by guests (wanted or unwanted) on your system(s)?

On Thu, Jun 11, 2015 at 02:19:39AM +0000, James Ewing Cottrell, III via Novalug wrote:
> I've been talking about this for awhile, and I finally got around to writing it.
> 
> usage: /back/up [ HOST ]
> 
> Backs up selected filesystems from HOST (or localhost if not specified) for the
> past month. If a file doesn't change from day to day, it is hard-linked instead
> of being copied. Files are kept in /back/$HOST/$DAY where DAY is {01..31}. Uses
> a symlink named "last" to point to the previous backup.
> 
> TODO: specify different FS by HOST
> TODO: optionally remove some days if space needed.
> TODO: hardlink between different hosts
> TODO: Suggestions?
> 
> I just ran this on a local host and a remote host,
> so I thought I'd post it while it still works.
> 
> JIM
> 
> #! /bin/sh -x
> 
> HOST=${1:-$(hostname)}
> SIZE=${2:-99G}
> TOP=/back/$HOST
> TODAY=$(date +%d)
> 
> if test -d $TOP
> then    LAST=$(readlink $TOP/last)
> else    for LAST in {01..31}; do mkdir -p $TOP/$LAST; done
> fi
> 
> cd $TOP
> for FS in / /dev /home/jcottrell /var
> do
>         mkdir -p $TODAY$FS
>         rsync -WHvaxz --link-dest=$PWD/$LAST$FS/ $HOST:$FS/ $PWD/$TODAY$FS/ --delete --max-size=$SIZE
> done
> 
> rm -f last
> ln -s $TODAY last

-- 
Hei konā mai
Mark Smith
mark@winksmith.com




More information about the Novalug mailing list