naturerecordists
[Top] [All Lists]

Re: File Archiving [was: tape digitizing]

Subject: Re: File Archiving [was: tape digitizing]
From: "Paul Jacobson" thebrunswicktwitcher
Date: Wed Sep 30, 2009 6:13 pm ((PDT))
Most of the solutions put forward are great for protecting against  
media failure, they do nothing to prevent complete loss in the event  
of a disaster.  Consider what you will be left with if your property  
is burgled and your computer and recording is gear stolen, or if a  
fire, flood or other natural disaster destroys the property and  
contents. If your original and backup are stored at the same location  
and there is an event of this kind you will suffer total loss,  
regardless of what kind of storage media you are using.

If you are seriously concerned about ensuring that your archived  
recordings are protected you need to have at least one complete copy  
of the backup stored in a physically separate location and these  
backups need to be regularly checked for condition and accessibility -  
preferably every 12 months.

On-line backup satisfies the need to have a copy stored off-site, and  
can be very attractive if your ISP doesn't bill for outgoing data.  It  
is worth considering what methods you have available to retrieve the  
backed up data if you do suffer data loss.  Most ISP's either charge  
for excess inbound data or cap transfer speed once your data quota is  
reached.  If you try to restore 500Gb of data via the Internet it will  
either be hugely expensive if excess is charged or incredibly slow if  
the connection is speed limited.   My suggestion is to look for Online  
back-up service that has a local presence and offers  service to  
courier a disk pack with your backed-up data in the event of data loss.

cheers
Paul


On 01/10/2009, at 8:45 AM, Jeremiah Moore wrote:

> On my primary work machine, I have CCC wake up every tuesday night  
> and clone
> the system drive to a second internal mechanism (warm failback boot  
> volume.)
> This protects me against lost productivity if (when) my boot drive  
> dies.
>
> Project folders get pushed to the working-backup drive at multiple  
> point
> throughout the day by manually triggered unix script (rsync).  It  
> compares
> mod dates and checksums and only copies files which are changed.
>
> User folders on laptop and work machines backed up to one another  
> using
> Crashplan.  (this is becoming a slight problem however, since the  
> crashplan
> backup is taking SO much space on the target machine, my laptop  
> drive is
> nearly full.)
>
> TimeMachine looks great but I'm still os 10.4 on laptop.
>
> I log when drives are purchased - in a spreadsheet, and a sticker on  
> the
> drive - and replace them before they're five years old.
>
> Drobo looks good for libraries and other large data you want to have  
> mounted
> but doesn't need to be high performance.
>
> -jeremiah
>
>
>
> On Wed, Sep 30, 2009 at 9:00 AM, Rob Danielson <> wrote:
>
>>
>>
>> Hi Phil-
>> There are versions of CCC that work back to OS X 10.2 too. I use CCC
>> every few weeks to back-up my entire system install but its very
>> simple to turn on an external drive every day for a few minutes and
>> manually back-up the new contents of a few, well-organized folders.
>> Its easy for my mind to separate OS backup from media file back-up
>> because the files are on a separate drive partitions. The risks of
>> loss are also isolated. If my OS creates a corrupt file (and its
>> making thousands a day) my media file directories are not affected.
>>
>> Maybe its because the absolute need to back-up data became very
>> evident to me in 1995, but I'd rather not have any application
>> performing these crucial tasks "automatically" for me. The
>> self-sufficiency of knowing exactly what needs to be done and doing
>> it oneself probably lessens the chances of error more than relying on
>> automation-- perhaps more so for a person starting to learn good,
>> back-up habits.
>>
>> The CCC and Time Machine back-up programs are probably very good, but
>> backing-up recordings isn't a precaution for me, it part of the
>> process of recording. My maxim is, "A digital file doesn't really
>> exist until there are two copies on different media and I know
>> exactly where both are." The simple, "manual approach" promotes the
>> cause because when I am very familiar with the structure of my file
>> archive, the changes I need to make are more cohesive and hopefully
>> more understandable down the road. Rob D.
>>
>> = = = = = = =
>>
>>
>> At 9:46 AM +0000 9/30/09, Philip Tyler wrote:
>>> Hi
>>>
>>> Any one running an Apple Mac with Leopard as their operating system
>>> should have a look at Time Machine and a Time Capsule. It has
>>> revoloutionised backing up for me, coupled with something like
>>> SuperDuper or Carbon Copy Cloner and an external drive makes a great
>>> backup system.
>>>
>>> Phil
>>>
>>> On 30 Sep 2009, at 03:43, Rob Danielson
>>> <<type%40uwm.edu <type%2540uwm.edu>> <type 
>>> %40uwm.edu>>
>> wrote:
>>>
>>> Hi Jeremiah--
>>> In "archival" time, I think it is wise to assume that the products
>>> that best meet necessary criteria will change pretty frequently. We
>>> now add "instantly accessible" to "indestructible" and "cheap." I
>>> agree that a huge argument for _redundant_ hard drives is the
>>> probable ability to quickly convert one's data from one storage
>>> system to another (at least compared to current optical discs).
>>>
>>> For folks with a few hundred GB's to store, its not a huge chore to
>>> use both drives and optical disks. For larger archives, arrays of
>>> more than 8 drives become quite pricey and 1 TB per drive does seems
>>> to be the recommended maximum capacity today. For me, its worth the
>>> extra time to burn everything, including the less critical, original
>>> long takes to DVD-R discs for more confidence and perhaps the  
>>> ability
>>> to "skip-over" a few less than-perfect storage solutions. If I  
>>> follow
>>> recommended storage conditions, I'll feel safer waiting for the  
>>> ideal
>>> storage medium with a bunch of optical discs 30 years from now than
>>> holding onto 20 hard drives.
>>>
>>> What I want to avoid is not making recordings because it will take
>>> too much time or money to care for them. It will be fun to see where
>>> all of this ends-up :-). Rob D.
>>>
>>> = = = = =
>>>
>>> At 2:55 PM -0700 9/29/09, Jeremiah Moore wrote:
>>>> I think the idea is to see archive maintenance as an ongoing  
>>>> process. To
>>>> maintain a digital archive, I see it as essential to re-copy the  
>>>> data
>> every
>>>> few years to new media. In addition to allowing verification,  
>>>> this will
>>>> allow bridging the various gaps presented by constantly shifting  
>>>> storage
>>>> technologies. i.e. for a while, it was common to have SCSI and  
>>>> firewire
>>>> interfaces on workstations; that would've been a good time to  
>>>> migrate to
>>>> Firewire drives.
>>>>
>>>> Factors in my decision to use hard drive mechanisms as primary  
>>>> archive
>>>> media:
>>>>
>>>> - easy to archive large amounts of data without handling many  
>>>> individual
>>>> pieces of media. This saves time and labor downstream as each  
>>>> piece needs
>>>> to be cataloged and stored.
>>>>
>>>> - easy to copy a significant chunk of archived material in one  
>>>> move,
>> saving
>>>> time and labor when the archive is migrated to the next medium.
>>>>
>>>> - costs per MB are reasonable. 1TB HDD is around $100. 1TB of DVD- 
>>>> R is
>>>> roughly 240 discs, at $0.30 ea is $72.00 not including sleeves or  
>>>> cases.
>>>>
>>>> - de-facto, I was always way behind on archiving via DVD-R  
>>>> becuase of the
>>>> work involved. Typical doc film mix would involve backing up to  
>>>> eight or
>>>> ten DVD-Rs, plus their redundant copies. It would take much of a  
>>>> day to
>>>> archive a single project, time I could not afford to spend.
>>>>
>>>> Significant downside is that the medium is writeable, meaning it's
>>>> susceptible to filesystem damage or file corruption. Corruption  
>>>> could be
>>>> carried from the "A" copy to the "B" copy during synchronizing.
>>>>
>>>> A better system would involve checksumming, perhaps zipping or  
>>>> using a
>> unix
>>>> tool like tar. I access files from my archive semi-frequently, so  
>>>> it's
>>>> helpful to have it all easily mountable and file-accessible on my  
>>>> system,
>> so
>>>> I make this tradeoff knowingly. If others have suggestions, I'd  
>>>> love to
>>>> hear them.
>>>>
>>>> -jeremiah
>>>>
>>>> Rob Danielson wrote --
>>>> -snip-
>>>>
>>>>> Its interesting to read that folks are using redundant drives as a
>>>>> primary storage medium. Maybe drive hardware, stored under the  
>>>>> right
>>>>> conditions will work fine in 30-50 years. It might be faster to
>>>>> convert data from a drive to the improved media that come along  
>>>>> than
>>>>> from optical disks. However, it could also become a headache to  
>>>>> mount
>>>>> current drives or a disc reader on a computers made even 15 years
>>>>> from now. Certainly, both drives and optical discs will look  
>>>>> ancient
>>>>> in 100 years. Consider the challenge of mounting a SCSI drive on a
>>>>> computer produced in 2009.
>>>>>
>>>>>
>>>>
>>>> -snip-
>>>>
>>>> --
>>>> ----------------------------------------------------------
>>>> jeremiah moore | SOUND |
>>>> <jmoore%40northstation.net <jmoore 
>>>> %2540northstation.net>><mailto:
>> jmoore%40northstation.net <jmoore%2540northstation.net>>
>>  <jmoore%40northstation.net>
>>>> <<http://www.jeremiahmoore.com/>http://www.jeremiahmoore.com/><
>> http://www.jeremiahmoore.com/>http://www.jeremiahmoore.com/
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>>
>>
>> --
>>
>>
>>
>>
>
>
>
> -- 
> -----------------------------------------------------------
> jeremiah moore | SOUND | 
> http://www.jeremiahmoore.com/
>
>
>
>
>
> ------------------------------------
>
> "While a picture is worth a thousand words, a
> sound is worth a thousand pictures." R. Murray Schafer via Bernie  
> Krause
> Yahoo! Groups Links
>
>
>
>







<Prev in Thread] Current Thread [Next in Thread>
Admin

The University of NSW School of Computer and Engineering takes no responsibility for the contents of this archive. It is purely a compilation of material sent by many people to the naturerecordists mailing list. It has not been checked for accuracy nor its content verified in any way. If you wish to get material removed from the archive or have other queries about the archive e-mail Andrew Taylor at this address: andrewt@cse.unsw.EDU.AU