Kin, I thought I was paranoid about data security but your system is outsta=
ndingly clever. I am very impressed!
As you suggest, I do the incremental Time Machine backup continuously and a=
fully bootable image using SuperDuper once a month. This way I at least ha=
ve all my tools and entire work setup handy (on the bootable backup) should=
the laptop give up. I can just plug the external hard drive into a second =
Macbook Pro and I am able to work. I can then restore individual files from=
the Time Machine backup as needed while I sort out the problems with the b=
roken down laptop. Once the broken laptop is repaired I can use the increme=
ntal Time Machine backup to restore everything to its former glory. That is=
, if the Time Machine backup disk does not break down in the meantime!
However, I only store the finished versions of sound recordings, albums, ph=
otos and videos on my Macbook. This takes up significantly less space on th=
e internal drive and makes the backup much easier/faster while protecting m=
e from running out of internal HDD space too quickly. However, it also mean=
s that only these final versions get the royal treatment of the double back=
up. The raw and intermediate versions are stored on the RAID system that I =
mentioned earlier. I only need them very rarely but still want to retain th=
em as I have had some instances where I just needed a particular recording =
that I knew I hadn't used because it was not relevant in the past but had n=
ow become the perfect sound effect, for example.
This RAID server is indeed set up as a RAID 5, however, I only do this beca=
use it features 4 HDD so I can be reasonably sure that there are enough HDD=
left to re-sync after a failure. Plus, I am not sure that I could just tak=
e one of the drives out and plug it into an external case should the RAID s=
erver itself give up the ghost. With a RAID 1 I can do that since all HDD i=
n the array are an exact clone of the others. So for that reason I would us=
e a RAID 1 if I was to use two standard external HDD for a (Software) RAID =
system.
And I might do that soon as my current RAID server solution comes with some=
more restrictions. Apart from the slow connection it takes about 5 minutes=
to boot. Plus, in the case of a sudden power outage it needs to re-sync th=
e HDD for 2 days! Since in Darwin we have power outages at least once a mon=
th and I don't have time to wait for 2 days after each outage for my data t=
o sync (while praying that everything works as expected) I have given in an=
d purchased a UPS so at least the RAID server is protected from the power o=
utage for up to 30 minutes and I can shut it down properly. However, all of=
this is a bit too cumbersome so for my upgrade of this system I really lik=
e your solution. I might just compare Retrospect and SoftRAID and see what =
the pro's and con's are. As for HDD's I thought about Thunderbolt, however,=
USB 3 provides 5GB/s which is plenty enough for my and the extra 5BG/s do =
not justify the significantly higher costs of these drives for me. All that=
said, the pain is not big enough yet with my current solution so that I wo=
uld shell out another couple hundred dollars for a new solution. I am getti=
ng a bit more into videos though lately, so this may change the situation v=
ery quickly.
Anyway, thanks for sharing your solution, it is very good indeed.
Lastly to answer Peter's "how do you know for sure that it's really working=
as a set? And how will you know when one drive has died?" - The RAID serve=
r takes care of that. If everything is sweet and dandy it displays 4 green =
lights (one for each HDD) and it says something along the lines of "all dri=
ves are synchronised". Should there be a problem, I get a warning. The RAID=
server even notifies me if a HDD is showing some oddities like producing d=
efect sectors. The server then alerts me that this is happening and that th=
ere is a risk that the HDD might fail in the future so I can already purcha=
se a replacement before this happens. I've never had to do this though (kno=
ck on wood).
Eric
On 03/04/2013, at 11:29 PM, Kin Corning <> wrote:
> Hi Eric,
>
> With apologies to others on the list if I'm responding with too much deta=
il on this specific topic...
>
> Yes, one external drive acts as primary storage for all media files (phot=
ographs, sound recordings, etc.) and gets backed up to each of 2 other iden=
tical external drives. My current drives are 6TB each, which is more than e=
nough for all the data I have. To be a bit more detailed, one of those back=
ups is a straight clone and the second is an "incremental" backup, both mai=
ntained using Retrospect (more on this in a moment). The advantage of this =
approach is that the clone is a "warm" copy identical to the primary (excep=
t for any intervening work since the last backup, which as I mentioned is n=
ever more than 1 day's work), and thus ready to use immediately should the =
primary drive fail. The clone, however, only has the current version of eac=
h file on disk. The 2nd "incremental" backup retains time history of any ch=
anged files, so for example, if a file is accidentally deleted or saved in =
an unintended edited state, or is somehow corrupted, it is possible to go b=
ack and restore an earlier version of the file. This "incremental" backup i=
s, however, in a particular format specific to the backup software and ther=
efore needs to be restored to a clean disk before the data can be used. Hav=
ing dual backups in this fashion gives the best of both worlds. The 3rd off=
site backup is also incremental, and is swapped back and forth with its ons=
ite twin as often as possible.
>
> (A small aside is that in this setup it is probably desirable to swap the=
primary drive and the "warm" clone from time to time, so that the run-time=
is evened out between the two drives and the primary is therefore less lik=
ely to fail. I have done this once over the course of 2 years or so.)
>
> (Another small aside is that whenever I download new digital photographs =
or sound recordings, I perform the two backups immediately notwithstanding =
my 1-or-2-times-per-day guideline before I do anything else to the files, a=
nd before I re-format the memory cards from the recorder/camera.)
>
> As mentioned I am using Retrospect for both types of backup. At the time =
I chose it some years ago, it was considered the enterprise-class standard =
for backup up Macintosh computers. Other than reputation, I picked it becau=
se it did some things I specifically needed which other software did not: n=
etwork backups, backups of mixed Mac and PC machines, and it drives my Sony=
AIT drive which I use to back up selected data to tape. Retrospect has gon=
e through several changes in corporate ownership, and I believe is now oper=
ating as an independent company. There was one period where it seemed to be=
receiving minimal support, though as best I can tell that is no longer the=
case.
>
> There are probably simpler solutions if you don't need any of the "corpor=
ate" type features. If you are on a Mac, Time Machine would do a perfectly =
fine job of the "incremental" backups and any one of a number of products (=
Carbon Copy Cloner for example) could manage the "warm" clone.
>
> Incremental backups impose a storage overhead (to keep the earlier versio=
ns of the file), so if your working practices lead to large files changing =
regularly (for example if regular metadata changes are saved within a media=
file), you may need to think about how this interplays with your backup st=
rategy. The backup software which performs the incremental backups will all=
ow you to specify policies about how many past stages are retained and at w=
hat intervals.
>
> There is nothing wrong with using a RAID configuration as the "primary" i=
n this strategy -- in fact it would be an improvement over my approach in p=
roviding some level, albeit imperfect, of intra-day protection. I just didn=
't go down this route again to keep the cost and complexity down. But I kno=
w people who are using Drobos, and others who are using G-Technology RAID b=
oxes, and are very happy with them. You just need to recognise they are not=
a complete solution in themselves, and can be costly if they fail, as per =
my first note.
>
> My units connect to my Mac tower by eSATA, but in today's world USB3 or T=
hunderbolt would be better and more future-proof. As you say, any of these =
will be faster than Ethernet to a network storage box.
>
> I am aware that you can construct RAID arrays in the way you suggest with=
software, either with external devices or indeed with internal drives with=
in your Mac/PC, but I do not have any direct experience with this approach.
>
> I'm repeating what you already said to some degree, but there are indeed =
different RAID strategies. As you said, RAID Level 1 is a mirror, and is of=
ten combined with RAID Level 0 ("striping"). RAID 0 often confuses people b=
ecause it has nothing to do with redundancy -- it simply spreads data acros=
s 2 or more drives for speed in access. Many of the larger capacity drives =
you can buy today (including the ones I have) in fact have 2 drives interna=
lly in a RAID 0 configuration for speed, but look like a single large drive=
to the computer, and the RAID provides no backup protection. RAID Level 1 =
(a mirror) is the most secure but least space-efficient of the redundant RA=
ID configurations. RAID Level 5 is possibly a better choice for redundancy =
-- in practice is is very secure and is much more space efficient than RAID=
1.
>
> Kin
>
> --- In Eric Fassbender <> wrote=
:
> >
> > Hi Kin,
> >
> > I agree that there is still the element of the RAID server that can fai=
l and in your case it has happened, so apparently it is not unlikely. So I =
really like your solution/approach. Am I assuming correctly that you use on=
e drive that always stays the primary and that then gets backed up to the t=
wo other external drives? If so, what software are you using to perform thi=
s feat or do you do it manually? I would be very interested to hear your fe=
edback as although I am still confident in my RAID, it has the limitation t=
hat it is slowed down by its only I/O connection - a 1GBit Ethernet connect=
ion. So I was eyeing a USB3 RAID solution using 2 or 3 standard external HD=
D and SoftRAID 4.5 (Mac), however, I am wondering whether your solution off=
ers any benefits over a software RAID?
> >
> > For anyone who is not that familiar with RAID systems, there are differ=
ent types of RAID. What we need for data protection/safety is RAID 1, a 'Mi=
rrored Set'. This can be 2 or 3 even 4 drives and they all hold the same mi=
rrored data. So always check the specifications because there are many RAID=
solutions out there that offer RAID 0, which makes data transfer super fas=
t but does not offer protection against data loss. So if you are in the mar=
ket, look for RAID 1.
> >
> > Cheers,
> >
> > Eric
> >
> >
> > On 03/04/2013, at 7:49 PM, Kin Corning <> wrote:
> >
> > > I would offer some comments about using desktop RAID systems, which h=
ave been mentioned in this thread.
> > >
> > > When RAID arrays are used in Storage Area Networks or other such conf=
igurations in professional datacenters, *everything* is redundant -- the se=
rvers, the disk storage, power supplies, RAID controllers, and the intercon=
nections between server and disk. In a desktop RAID unit, typically it is o=
nly the disk drives themselves which are protected. And, while a disk is th=
e part most likely to fail, other failures can occur as well.
> > >
> > > I was reminded of these limitations the hard way a couple years ago w=
hen I had a desktop RAID unit fail, apparently (I'll never know for sure) d=
ue to a failed controller board in the box. The device was (just) out of wa=
rranty, the manufacturer (a major well-known firm) would not offer any serv=
icing, and their only suggestion was to buy another identical unit, transfe=
r the drives, and see if the data was intact. I did a fair bit of explorati=
on and could not find any third-party who had the expertise to service the =
unit, which became an expensive scrap job.
> > >
> > > I was backed up separately, so my data was protected, but out of curi=
osity I contacted one of the best known firms in the UK for a quote on data=
restoration, and the answer was =A33000 to give it a try with no guarantee=
of success. This may sound crazy but is understandable given the complexit=
y of how data is distributed across drives in the various RAID strategies, =
and the fact that the data may or may not have been corrupted.
> > >
> > > These desktop RAID units may still be attractive to some in offering =
a degree of real-time protection against a drive failure, but in my own set=
up (used primarily for photography) I have concluded this benefit isn't wor=
th the expense and complexity. I have moved to a simpler system with 4 iden=
tical large-capacity drives, 3 of which are onsite and 1 of which is always=
offsite. Every day (twice a day if I am doing a lot of work) the primary d=
rive gets backed up to each of the 2 other onsite drives, and as often as i=
s practical one is swapped with the offsite device, to keep the data protec=
ted from fire or burglary as up-to-date as possible. In this setup, I may l=
ose a few hours of work if a drive fails, but on the other hand the system =
is instantly restorable and the cost of replacing the failed unit is much l=
ower.
> > >
> > > Not to state what may be obvious, but if you choose to use one of the=
se units notwithstanding, it is important to remember it is not a backup de=
vice. RAID offers real-time protection against a drive failure, but an acci=
dentally-deleted file will be deleted across the RAID array, and any softwa=
re-generated file corruption likewise may be duplicated in each 'copy' of t=
he file. So even if you employ one of these units to protect your work intr=
a-day, you still need to back up to separate devices regularly.
> > >
> > > Kin
> > >
> > > --- In "sounds.images" <sounds.imag=
es@> wrote:
> > > >
> > > >
> > > > Many thanks everyone for you words of support and replies, a new la=
ptop has been ordered and I do have a caddy already, so will put it in ther=
and have a go at transferring it of it is possible to recover them..
> > > >
> > > > Seen in PC world.co.uk is a USB 3.0 1tb external drive for =A359.00=
so may grab one..
> > > >
> > > > Will update as soon as I can.
> > > >
> > > > Simon
> > > >
> > > >
> > > >
> > > >
> > > > --- In "sounds.images" <sounds.im=
ages@> wrote:
> > > > >
> > > > > Last night the other half shut down the laptop as normal before b=
ed. This morning it was dead. It would power on and fans would spin but not=
hing. Took it to my parents and tried their hard drive in my machine.... No=
thing..
> > > > >
> > > > > Tried my hd on their machine and it tried to boot but went straig=
ht to blue screen of death twice...
> > > > >
> > > > > So never put off your back ups. I have lost every recording I mad=
e since September. It was on the cards to do this little job this weekend, =
but it is too late.. Will try again when I get a replacement machine. This =
was completely unforeseen..
> > > > >
> > > > > Oh well... Lesson learned. Never put off the most important job o=
f all.
> > > > >
> > > > > Simon.
> > > > >
> > > >
> > >
> > >
> >
> >
> >
> >
>
>
|