Linode mentioned on Slashdot
caker
Linode Staff
-Chris
10 Replies
@Cool_blade:
I said my bit that linode is cool!
Me too … I hope this Ask Slashdot generates some nice business for Linode.com!
How long does it take to get 'validated'?
@mcowger:
It just did…my previous provider, JVDS.com, just had a disk failure, and hadn't been keeping backups like they said they were! So, I demanded a refund of the last 6 months, got it, checked out the Slashdot article, and just signed up here.
Wow, that is just terrible. I can't believe that they would say they were making backups and yet not do it.
Of course, Linode.com doesn't offer backups but at least they don't claim to do them and then not do them. It's pretty easy to do rsync backups to a home system anyway, if you need an example script for doing this let me know, mine has been working great for the past 6 months (I keep two weeks worth of incremental backups and the nightly backups take less than 5 minutes).
@mcowger:
How long does it take to get 'validated'?
Shouldn't take very long, especially if you find Chris on the IRC channel. Welcome to Linode.com, and best wishes!
http://www.cis.upenn.edu/~bcpierce/unison/
I have yet to set it up . . . I know a few people that have and read the doc's, and look just as good as any others, one plus for the windows users it does a windows build for those that want to back it up and a windows box. (others might I just don't know)
Welcome to Linode!
@mcowger:
Im all setup now….would love to see that rsync script…
Happy to oblige. A few caveats:
This isn't "productized" in any way. You have to know what you are doing and you may need to modify this to work the way you want.
This script is run from the system which is to be backed up to and it contacts the system which is to be backed up.
In order to run this script unattended, you will need to set up keys so that the system which is doing the backup can ssh into the system which is being backed up, and as root no less. This could be a security concern because it means that having root on the backed up to machine is as good as having root on the backed up machine.
I run this script on both systems, so that they back each other up. It is important to "exclude" the backups directory of one system from being backed up on the other, otherwise you will have "recursive" backups that will grow exponentially.
Usage is as follows:
rsyncbackup.sh [remoteroot] [localroot] [excludesfile]
[remote_root] is the directory from the remote system to be backed up. I use '/' to back up everything.
[local_root] is the local directory under which the backups will be written. They are rotated so that if you are keeping 14 backups, after 14 days you will have backup.01, backup.02, …, backup.14, with backup.01 being the oldest.
[excludes_file] is a file on the local system which lists, with one file or directory per line, the files and directories from the remote system which should not be backed up.
You can set the following environment variable also:
NUM_BACKUPS is the number of backups to keep (I back up once per day and use 14 to have two weeks' worth of backups at all times).
This script uses hard linking to eliminate redundancy of files, so each incremental backup only actually uses disk space for those files which have changed since the last backup, although each backup.XX directory will look like a complete backup.
Here is what my crontab entry for doing the backups looks like:
# At 5:13 every morning, back up mitya.ischo.com
13 5 * * * /data/backup/rsync_backup.sh mitya.ischo.com / /data/backup /data/backup/excludes.txt
And here is what the /data/backup/excludes.txt file looks like:
/proc
/data/backup/eva.ischo.com
/data/rsync/modules
/data/share
/data/tmp
Have fun!
NOTE: I'm having some problems with the way that this forums system formats the code lines; it's wrapping some even though it doesn't show them as wrapped in the "preview". It looks like it's only some of the comment lines though that have alot of dashes in them. Be careful when/if you copy the script text into a file to fix that before trying to run it.
`#!/bin/bash
echo "rsync_backup.sh started at " `date` "."
# ------------------------------- Constants ----------------------------------
NUM_BACKUPS=3
# -------------------------------- Commands ----------------------------------
MKDIR=/bin/mkdir
MV=/bin/mv
RM=/bin/rm
RSYNC=/usr/bin/rsync
SEQ=/usr/bin/seq
# --------------------------------- Paths ------------------------------------
BACKUP_ROOT="$3"
BACKUP_SYSTEM=$1
EXCLUDES_FILE="$4"
EXCLUDES_ARG=""
REMOTE_ROOT="$2"
TMP_OUT=$BACKUP_ROOT/rsync.out
TMP_TOUCH=$BACKUP_ROOT/rsync.touch
# --------------------------- Check Requirements -----------------------------
if [ -z "$BACKUP_ROOT" ]; then
echo "Usage: rsync_backup.sh [remote_system] [remote_root] [local_root] [excludes file]"
exit -1;
fi
if [ -z "$BACKUP_SYSTEM" ]; then
echo "Usage: rsync_backup.sh [remote_system] [remote_root] [local_root] [excludes file]"
exit -2;
fi
if [ -z "$REMOTE_ROOT" ]; then
echo "Usage: rsync_backup.sh [remote_system] [remote_root] [local_root] [excludes file]"
exit -3;
fi
BACKUP_DIR="$BACKUP_ROOT/$BACKUP_SYSTEM"
echo "Back up local root: $BACKUP_DIR"
if [ \! -d "$BACKUP_DIR" ]; then
echo "Making directory: $BACKUP_DIR"
$MKDIR -p "$BACKUP_DIR"
fi
# Figure out which is the newest backup
NEWEST_EXISTING=0
for i in `$SEQ 1 $NUM_BACKUPS`; do
if [ $i -lt 10 ]; then
if [ -d "$BACKUP_DIR/backup.0$i" ]; then
NEWEST_EXISTING=$i
fi
else
if [ -d "$BACKUP_DIR/backup.$i" ]; then
NEWEST_EXISTING=$i
fi
fi
done
if [ $NEWEST_EXISTING -gt 0 ]; then
if [ $NEWEST_EXISTING -lt 10 ]; then
BACKUP_PREV="$BACKUP_DIR/backup.0$NEWEST_EXISTING"
else
BACKUP_PREV="$BACKUP_DIR/backup.$NEWEST_EXISTING"
fi
echo "Newest backup: $BACKUP_PREV"
else
echo "No previous backups"
fi
NEW_NUM=$[NEWEST_EXISTING + 1]
if [ $NEW_NUM -lt 10 ]; then
BACKUP_DEST="$BACKUP_DIR/backup.0$NEW_NUM"
else
BACKUP_DEST="$BACKUP_DIR/backup.$NEW_NUM"
fi
echo "New backup dir: $BACKUP_DEST"
# ------------ Compose rsync args ------------
# Copy all files recursively including all file attributes, verbosely,
# and use compression over the wire, also delete any deleted files
RSYNC_ARGS="-avz --delete"
# Use ssh to the remote system
RSYNC_ARGS="$RSYNC_ARGS -e ssh"
# Exclude file, if present
if [ -n "$EXCLUDES_FILE" ]; then
RSYNC_ARGS="$RSYNC_ARGS --exclude-from=$EXCLUDES_FILE"
fi
# Link dest, if we already have a previous rsync
if [ $NEWEST_EXISTING -gt 0 ]; then
RSYNC_ARGS="$RSYNC_ARGS --link-dest=$BACKUP_PREV/"
fi
# Source location
RSYNC_ARGS="$RSYNC_ARGS root@$BACKUP_SYSTEM:$REMOTE_ROOT/"
# Destination
RSYNC_ARGS="$RSYNC_ARGS $BACKUP_DEST/"
# Do the rsync
$RM -f $TMP_OUT
echo "rsync command: $RSYNC $RSYNC_ARGS"
$RSYNC $RSYNC_ARGS > $TMP_OUT
# Touch the rsync directory to set the backup time
touch "$BACKUP_DEST"
tail -2 $TMP_OUT
# --------------------------------- Rotate -----------------------------------
# This function is unfortunately necessary because Linux is messing with the
# modification times of the directories on mv
function correct_mv() {
$RM -f $TMP_TOUCH;
touch -r $1 $TMP_TOUCH;
$MV $1 $2;
touch -r $TMP_TOUCH $2;
$RM -f $TMP_TOUCH;
}
if [ $NEW_NUM -gt $NUM_BACKUPS ]; then
# Remove number 1
$RM -rf "$BACKUP_DIR/backup.01"
# Renumber the others
for i in `seq 2 $NEW_NUM`; do
if [ $i -lt 10 ]; then
FROM=0$i
else
FROM=$i
fi
j=$[i]
if [ $j -lt 10 ]; then
TO=0$j
else
TO=$j
fi
correct_mv "$BACKUP_DIR/backup.$FROM" "$BACKUP_DIR/backup.$TO"
done
fi
echo "rsync_backup.sh finished at " `date` "."` [/i]
rsync_backup.sh started at Thu Feb 12 05:13:00 EST 2004 .
Back up local root: /data/backup/mitya.ischo.com
Newest backup: /data/backup/mitya.ischo.com/backup.03
New backup dir: /data/backup/mitya.ischo.com/backup.04
rsync command: /usr/bin/rsync -avz --delete -e ssh
--exclude-from=/data/backup/excludes.txt
--link-dest=/data/backup/mitya.ischo.com/backup.03/ root@mitya.ischo.com://
/data/backup/mitya.ischo.com/backup.04/
wrote 207352 bytes read 1092446 bytes 18178.99 bytes/sec
total size is 841411632 speedup is 647.34
rsync_backup.sh finished at Thu Feb 12 05:14:24 EST 2004 .