Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Managing tape drives and libraries with the Unix/Linux CLI

blogs.intellique.com

Post reply on HN
Print view View original

31–40 of 45 posts

Re: Managing tape drives and libraries with the Unix/Linux CLI

#31

Post by Hamuko »

waynesonfire wrote: post #20
funstuff007 wrote: post #11

LTO does seem to be the last bastion of closed source vendor lock-in. That being said, even if prices dropped by 75%, I'm not sure our shop would look to do LTO instead of a series of hard drives. 20 TB hard drives are now $400.

Hdds are soo fragile. They constantly break. A proper 3-2-1 backup solution is incomplete without tape IMHO.

At least HDDs are fast, cheap, don't have massive up-front investments. You can take a couple of HDDs, RAID them together for reliability and you can still write and read data at humane speeds.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#32

Post by justsomehnguy »

zozbot234 wrote: post #25
wazoox wrote: post #24

Earlier quoted context omitted.

There's the very interesting new feature of LTO7M and 8M, which adds 50% capacity to your LTO7/8 tapes when formatting them with a - respectively - LTO8/9 drive. A 30$, 9TB LTO7 tape is the cheapest storage you can buy.

It's a one-way conversion, and then you're stuck with that one LTO generation; a LTO7M tape won't work anymore in a LTO7 drive.

It's a moot point.

If you are buying the whole system new then you buy M-capable drives.

If you are stuck with G7 drives - then you don't format your tapes to M, even if you have 1 G7M drive.

And if you value your data, ie you want to able to recover it someday then you need N+1 drives of needed generation and format anyway.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#33

Post by justsomehnguy »

matheusmoreira wrote: post #21
Karunamon wrote: post #5

I'm surprised still that so many people sleep on tape. It is a very mature and stable technology with the best dollars/gigabyte ratio of any storage technology out there. Start up costs are a bit unpleasant but purchasing secondhand is reasonable and economical, and unlike the cloud the costs are mostly one time instead of forever. The tapes are durable and last for decades, so that initial outlay will last you a ver…

It's not just the expense. Backwards compatibility is only guaranteed up to the last two generations of tape media. If you hold onto these things for decades, you might find the drives required to read them no longer exist.

> If you hold onto these things for decades

It's always amusing when people bash tapes for the lack of backwards compatibility, but...

Hey, try to find a working system with PATA drives support? Try to connect SATA150 drive to SATA-III port? How about SCSI-320?

People like to talk, but I always said what if your data has a value then it's your responsibility to have means to backup and to restore it. If your data "is extremely valuable" but all your C-level are approving is a single drive decade old library - you can write your backups straight to /dev/null, because your data is worthless for your C-level.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#34

Post by wazoox »

Hamuko wrote: post #31
waynesonfire wrote: post #20

Earlier quoted context omitted.

Hdds are soo fragile. They constantly break. A proper 3-2-1 backup solution is incomplete without tape IMHO.

At least HDDs are fast, cheap, don't have massive up-front investments. You can take a couple of HDDs, RAID them together for reliability and you can still write and read data at humane speeds.

A RAID array isn't a backup. Your data is still live on it, an "rm -rf" away from destruction. HDDs fare poorly when shelved for a long time, RAID are even worse in that regard. OTOH a tape on a shelf is perfectly safe for 5 to 10 years or more (I recently restored 1998/99 sensitive data from LTO2 tapes written in 2003).

Re: Managing tape drives and libraries with the Unix/Linux CLI

#35

Post by Hamuko »

wazoox wrote: post #34
Hamuko wrote: post #31

Earlier quoted context omitted.

At least HDDs are fast, cheap, don't have massive up-front investments. You can take a couple of HDDs, RAID them together for reliability and you can still write and read data at humane speeds.

A RAID array isn't a backup. Your data is still live on it, an "rm -rf" away from destruction. HDDs fare poorly when shelved for a long time, RAID are even worse in that regard. OTOH a tape on a shelf is perfectly safe for 5 to 10 years or more (I recently restored 1998/99 sensitive data from LTO2 tapes written in 2003).

>A RAID array isn't a backup.

Who said it was?

>Your data is still live on it, an "rm -rf" away from destruction.

How is accidental deletion not a problem with tapes?

Re: Managing tape drives and libraries with the Unix/Linux CLI

#36

Post by dabiged »

Can anyone comment on the brands of tape you can purchase? Given it is an open format what sets the brands apart? Are there records of failure rates like backblaze has done with HDDs?

Currently for LTO8 tapes you can buy the following brands:

* Fujifilm

* Quantum

* IBM

* HPE

* Oracle

* Dell EMC

Re: Managing tape drives and libraries with the Unix/Linux CLI

#37

Post by matheusmoreira »

justsomehnguy wrote: post #33
matheusmoreira wrote: post #21

Earlier quoted context omitted.

It's not just the expense. Backwards compatibility is only guaranteed up to the last two generations of tape media. If you hold onto these things for decades, you might find the drives required to read them no longer exist.

> If you hold onto these things for decades It's always amusing when people bash tapes for the lack of backwards compatibility, but... Hey, try to find a working system with PATA drives support? Try to connect SATA150 drive to SATA-III port? How about SCSI-320? People like to talk, but I always said what if your data has a value then it's your responsibility to have means to backup and to restore it. If your data "is…

I'm not "bashing" tapes, I'm pointing out the fact they have limited backwards compatibility guarantees. Given the massive expense of tape drives and the fact you need to keep replacing them and the tapes just to keep up with new LTO standards puts this almost perfect solution out of reach of mere mortals like me. I'm not an enterprise, there is no "C-level" approving anything.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#38

Post by matheusmoreira »

Hamuko wrote: post #31
waynesonfire wrote: post #20

Earlier quoted context omitted.

Hdds are soo fragile. They constantly break. A proper 3-2-1 backup solution is incomplete without tape IMHO.

At least HDDs are fast, cheap, don't have massive up-front investments. You can take a couple of HDDs, RAID them together for reliability and you can still write and read data at humane speeds.

With current software and hardware RAID solutions, we also have massive up-front cost. We can't buy drives as needed and add them to the system one at a time.

I wish it was easier to incrementally expand capacity of RAID setups. ZFS promises RAID-Z expansion but it still hasn't been released. Btrfs has a flexible block allocator that supports incremental expansion as well as heterogeneous drive pools but unfortunately the parity support cannot be relied upon to this day.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#39

Post by justsomehnguy »

matheusmoreira wrote: post #37
justsomehnguy wrote: post #33

Earlier quoted context omitted.

> If you hold onto these things for decades It's always amusing when people bash tapes for the lack of backwards compatibility, but... Hey, try to find a working system with PATA drives support? Try to connect SATA150 drive to SATA-III port? How about SCSI-320? People like to talk, but I always said what if your data has a value then it's your responsibility to have means to backup and to restore it. If your data "is…

I'm not "bashing" tapes, I'm pointing out the fact they have limited backwards compatibility guarantees. Given the massive expense of tape drives and the fact you need to keep replacing them and the tapes just to keep up with new LTO standards puts this almost perfect solution out of reach of mere mortals like me. I'm not an enterprise, there is no "C-level" approving anything.

> I'm not "bashing" tapes

Oh, this wasn't pointed at you at all, sorry for the misunderstanding.

> Given the massive expense of tape drives and the fact you need to keep replacing them and the tapes just to keep up with new LTO standards

Well, you don't need to keep replacing them. The rolling upgrade (gradually replacing the tapes and drives for a new generation) makes sense only if you have tens to hundreds of drives and thousands and thousands of tapes - ie for those insane enterprise libraries. For everything else, be it a small enterprise or a "mere mortals" it would be always cheaper to just buy a whole new library/autoloader.

And if the cost of initial buyout is too much then of course tapes aren't for you... along with the compatibility woes.

Re: Managing tape drives and libraries with the Unix/Linux CLI

#40

Post by wazoox »

Hamuko wrote: post #35
wazoox wrote: post #34

Earlier quoted context omitted.

A RAID array isn't a backup. Your data is still live on it, an "rm -rf" away from destruction. HDDs fare poorly when shelved for a long time, RAID are even worse in that regard. OTOH a tape on a shelf is perfectly safe for 5 to 10 years or more (I recently restored 1998/99 sensitive data from LTO2 tapes written in 2003).

> A RAID array isn't a backup. Who said it was? > Your data is still live on it, an "rm -rf" away from destruction. How is accidental deletion not a problem with tapes?

Unless you store your tapes right next to your cache of neodymium magnets, that should be pretty easy. Writing to a tape is very deliberate generally. Much more than writing to disks.
Post reply on HN