naturerecordists
[Top] [All Lists]

Re: File Archiving [was: tape digitizing]

Subject: Re: File Archiving [was: tape digitizing]
From: "Rob Danielson" danielson_audio
Date: Thu Oct 1, 2009 11:59 am ((PDT))
Hi Phil--
That's a good story about the advantages of TC. As you suggest, TC is
automatically doing things one should do if one understands the
liabilities.

Sounds like you have outstanding habits. Some folks, understandably,
never love computing enough to acknowledge the vulnerabilities. Its
exciting to see someone turn this attitude around and become very
proficient-- usually by assuming a determined DIY mentality.

The are many practices I could make more efficient with scripts,
batch processing, automatic back-ups and automatic maintenance.
Sometimes, a system that is asked to do less is better. The near,
true disasters I've encountered seem to have one thing in common, the
computer was in the process of performing one or more tasks in the
background when encountering another problem. The instructions
appeared to come into conflict. Intensive applications like Final Cut
Pro, Logic Audio, Spotlight, Dreamweaver and others tend to perform
system demanding and intensive routines like modifying drive
directories. They literally try to monopolize certain volumes on a
drive. I feel its crucial to be able to detect problems associated
with application generated "necessities" like these quickly enough
for a "save," "re-launch," or a reboot to eliminate the roots of
something serious.

Corrupted drive directories are by far the worst. I've encountered
maybe 20 times when restart drive maintenance didn't fix the roots of
a drive directory problem.  I produced most of these serious
situations by ignoring clues for too long. Luckily, Disk Warrior
(knock on,... something) was able to rebuild the drive directories in
all of these cases, including around eight dying drives. But, I've
seen many users face complete disasters as a result of plodding
along, ignoring peculiarities and assuming the computer "knew what it
was doing." The clues are usually amazingly obvious when you run
through the narrative.

Precautions for preventing data loss and data preservation are joined
at birth but it might also help the cause to see the steps as
divinely divorced. Maybe that's why we feel more comfortable relying
on less "animated" optical discs? ;-) Not the best timing to invest
in the gold ones I suppose,.. Rob D.

At 8:20 AM +0000 10/1/09, macmang4125 wrote:
>  I can understand where you are coming from Rob regards making
>backups. What I find so great about the TimeCapsule approach is it
>just does it without any input from you. I still carry out a clone
>of my hard drive with SuperDuper and I also burn DVD's of files. All
>as an insurance, but the TimeCapsule is always there and the files
>are so easy to locate.
>The other day I was checking a file I had worked on, probably too
>late at night and decided I have over cooked it. Open TimeCapsule
>and I quickly located an unadulterated file and was able to recover
>my position very quickly. I had the file on some DVD somewhere but
>the TC makes it very easy to travel through the file history.
>Horses for courses I suppose, I used to just rely on DVD's and CDR's
>to back data up on, as well as separate hard drives, and I still do
>as you have to protect against the 'what if' scenario. But with
>something like TC it will make recovery a lot quicker and simpler.
>But if the worst case scenario happened, as mentioned, then I can
>piece it together from the other copies.
>
>Phil
>--- In
><naturerecordists%40yahoogroups.com>=
m,
>Rob Danielson <> wrote:
>>
>>  Hi Phil-
>>  There are versions of CCC that work back to OS X 10.2 too. I use CCC
>>  every few weeks to back-up my entire system install but its very
>>  simple to turn on an external drive every day for a few minutes and
>>  manually back-up the new contents of a few, well-organized folders.
>>  Its easy for my mind to separate OS backup from media file back-up
>>  because the files are on a separate drive partitions. The risks of
>>  loss are also isolated. If my OS creates a corrupt file (and its
>  > making thousands a day) my media file directories are not affected.
>>
>>  Maybe its because the absolute need to back-up data became very
>>  evident to me in 1995, but I'd rather not have any application
>>  performing these crucial tasks "automatically" for me. The
>>  self-sufficiency of knowing exactly what needs to be done and doing
>>  it oneself probably lessens the chances of error more than relying on
>>  automation-- perhaps more so for a person starting to learn good,
>>  back-up habits.
>>
>>  The CCC and Time Machine back-up programs are probably very good, but
>>  backing-up recordings isn't a precaution for me, it part of the
>>  process of recording. My maxim is, "A digital file doesn't really
>>  exist until there are two copies on different media and I know
>>  exactly where both are." The simple, "manual approach" promotes the
>>  cause because when I am very familiar with the structure of my file
>>  archive, the changes I need to make are more cohesive and hopefully
>>  more understandable down the road. Rob D.
>>
>>  =3D =3D =3D =3D =3D =3D =3D
>>
>>  At 9:46 AM +0000 9/30/09, Philip Tyler wrote:
>>  >Hi
>>  >
>>  >Any one running an Apple Mac with Leopard as their operating system
>>  >should have a look at Time Machine and a Time Capsule. It has
>>  >revoloutionised backing up for me, coupled with something like
>>  >SuperDuper or Carbon Copy Cloner and an external drive makes a great
>>  >backup system.
>>  >
>>  >Phil
>>  >
>>  >On 30 Sep 2009, at 03:43, Rob Danielson
>>  ><<type%40uwm.edu>> wrote:
>>  >
>>  >Hi Jeremiah--
>>  >In "archival" time, I think it is wise to assume that the products
>>  >that best meet necessary criteria will change pretty frequently. We
>>  >now add "instantly accessible" to "indestructible" and "cheap." I
>>  >agree that a huge argument for _redundant_ hard drives is the
>>  >probable ability to quickly convert one's data from one storage
>>  >system to another (at least compared to current optical discs).
>>  >
>>  >For folks with a few hundred GB's to store, its not a huge chore to
>>  >use both drives and optical disks. For larger archives, arrays of
>>  >more than 8 drives become quite pricey and 1 TB per drive does seems
>>  >to be the recommended maximum capacity today. For me, its worth the
>>  >extra time to burn everything, including the less critical, original
>>  >long takes to DVD-R discs for more confidence and perhaps the ability
>>  >to "skip-over" a few less than-perfect storage solutions. If I follow
>>  >recommended storage conditions, I'll feel safer waiting for the ideal
>>  >storage medium with a bunch of optical discs 30 years from now than
>>  >holding onto 20 hard drives.
>>  >
>>  >What I want to avoid is not making recordings because it will take
>>  >too much time or money to care for them. It will be fun to see where
>>  >all of this ends-up :-). Rob D.
>>  >
>>  >=3D =3D =3D =3D =3D
>>  >
>>  >At 2:55 PM -0700 9/29/09, Jeremiah Moore wrote:
>>  >> I think the idea is to see archive maintenance as an ongoing process=
. To
>>  >>maintain a digital archive, I see it as essential to re-copy the
>>data every
>>  >>few years to new media. In addition to allowing verification, this wi=
ll
>>  >>allow bridging the various gaps presented by constantly shifting stor=
age
>>  >>technologies. i.e. for a while, it was common to have SCSI and firewi=
re
>>  >>interfaces on workstations; that would've been a good time to migrate=
 to
>>  >>Firewire drives.
>>  >>
>>  >>Factors in my decision to use hard drive mechanisms as primary archiv=
e
>>  >>media:
>>  >>
>>  >>- easy to archive large amounts of data without handling many individ=
ual
>>  >>pieces of media. This saves time and labor downstream as each piece n=
eeds
>>  >>to be cataloged and stored.
>>  >>
>>  >>- easy to copy a significant chunk of archived material in one
>>move, saving
>>  > >time and labor when the archive is migrated to the next medium.
>>  >>
>>  >>- costs per MB are reasonable. 1TB HDD is around $100. 1TB of DVD-R i=
s
>>  >>roughly 240 discs, at $0.30 ea is $72.00 not including sleeves or cas=
es.
>>  >>
>>  >>- de-facto, I was always way behind on archiving via DVD-R becuase of=
 the
>>  >>work involved. Typical doc film mix would involve backing up to eight=
 or
>>  >>ten DVD-Rs, plus their redundant copies. It would take much of a day =
to
>  > >>archive a single project, time I could not afford to spend.
>>  >>
>>  >>Significant downside is that the medium is writeable, meaning it's
>>  >>susceptible to filesystem damage or file corruption. Corruption could=
 be
>>  >>carried from the "A" copy to the "B" copy during synchronizing.
>>  >>
>>  >>A better system would involve checksumming, perhaps zipping or
>>using a unix
>>  >>tool like tar. I access files from my archive semi-frequently, so it'=
s
>>  >>helpful to have it all easily mountable and file-accessible on
>>my system, so
>>  >>I make this tradeoff knowingly. If others have suggestions, I'd love =
to
>>  >>hear them.
>>  >>
>>  >>-jeremiah
>>  >>
>>  >>Rob Danielson wrote --
>>  >>-snip-
>>  >>
>>  >>> Its interesting to read that folks are using redundant drives as a
>>  >>> primary storage medium. Maybe drive hardware, stored under the righ=
t
>>  >>> conditions will work fine in 30-50 years. It might be faster to
>>  >>> convert data from a drive to the improved media that come along tha=
n
>>  >>> from optical disks. However, it could also become a headache to mou=
nt
>>  >>> current drives or a disc reader on a computers made even 15 years
>>  >>> from now. Certainly, both drives and optical discs will look ancien=
t
>>  >>> in 100 years. Consider the challenge of mounting a SCSI drive on a
>>  >>> computer produced in 2009.
>  > >>>


--









<Prev in Thread] Current Thread [Next in Thread>
Admin

The University of NSW School of Computer and Engineering takes no responsibility for the contents of this archive. It is purely a compilation of material sent by many people to the naturerecordists mailing list. It has not been checked for accuracy nor its content verified in any way. If you wish to get material removed from the archive or have other queries about the archive e-mail Andrew Taylor at this address: andrewt@cse.unsw.EDU.AU