Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes • Hacker News Board

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#271

Post by adrian_b » Tue, Dec 29, 2020, 9:18 AM UTC

Synaesthesia wrote: post #224

samstave wrote: post #206

Earlier quoted context omitted.

"Let me just surveillance out for a second" Guess where this will have the biggest impact in the next decade - the long term storage of billions of personally invading surveillance data points forever... Seriously, this tech will be abused. Against everyone. Facial recognition much? Zoom data? TikTok Data in a very compact physical form-factor? Utah and China will LOVE this!

Data storage is already so cheap thanks to HDDs that a poor African country can afford to spy on its population. This will hardly change anything. If we want to do something about it we have to revolt in the streets.

HDDs are fine for short term storage, but they are too unreliable when you want to keep the data for many years, possibly for a lifetime.

Unfortunately, currently there is no other commercially available method of archival storage, except magnetic tapes. Optical storage has a too low density to be able to compete with magnetic tapes.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#272

Post by lostlogin » Tue, Dec 29, 2020, 9:20 AM UTC

bronco21016 wrote: post #223

gruez wrote: post #209

Earlier quoted context omitted.

>Otherwise that sounds like a nightmare that would require an insanely high paycheck. If the courier in question is 6'2" that might be the case, but I suspect if the person was much smaller and is a heavy sleeper, that it wouldn't be too bad.

As a person of 6’0” I found the flight from LAX to SYD, and return, miserable in coach. I slept a lot and watched some movies but any way you cut it that is a long time to be sitting in a very uncomfortable seat.

In the before times I flew Auckland to Doha over 20ish hours. I can’t believe anyone likes flying, every bit of it is awful.

I’m 1.97m or so and it’s really not designed for me. I suspect it’s optimised for those at 1.6-1.7m.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#273

Post by adrian_b » Tue, Dec 29, 2020, 9:26 AM UTC

daRealDodo wrote: post #135

fernly wrote: post #99

Aw, c'mon chaps! Can we just admire the tech for a second? > Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch... > When tape is being read it is streamed over the head at a speed of about 15 km/h and with our new servo technologies we are still able to position the tape head with an accuracy that is about 1.5 times the width of a DNA molecule.

> Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch Me wondering about those micro SD cards that can store 1Tb.

The density of flash memory is competitive with magnetic tapes, but the retention time is too low, making flash memory completely unusable for archival storage, even if it would have been as cheap as magnetic tape.

In theory, write-once memory cards, using some kind of antifuses, could be designed to have a lifetime good enough for archival storage, but nobody has attempted to develop such a technology, because it is not clear if there would be a market for them.

Most people do not think far ahead in the future, so they do not care much about archival data storage, until it is too late and the information had already been lost.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#274

Post by sgt » Tue, Dec 29, 2020, 9:27 AM UTC

aidenn0 wrote: post #26

jjulius wrote: post #21

Earlier quoted context omitted.

$100 is definitely not "basically zero". Not only that, but a $5k tape drive and tapes are designed to last a significantly longer period of time, essentially bringing the cost of long-term storage closer to $0/year than your option of spending $100/year.

It's 50 years before $100/year catches up with a one-time cost of $5k. On top of that, the total capacity of the mediumgoes up steadily over time, while you will need to drop another $5k every 10 years or so or span your backups across multiple tapes.

In reality I am sure he could get away with buying a new drive every 5 years.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#275

Post by dagurp » Tue, Dec 29, 2020, 9:31 AM UTC

jdofaz wrote: post #61

agumonkey wrote: post #25

Earlier quoted context omitted.

Let's have someone invents a ultra high def head to repurpose all the old VCRs and VHS tapes :)

D-VHS was a thing, tape capacity started at 25GB

Such a shame it didn't take off

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#276

Post by itismetheidiot » Tue, Dec 29, 2020, 9:32 AM UTC

diveanon wrote: post #257

closeparen wrote: post #250

Earlier quoted context omitted.

Is the internet in Australia really less reliable than airline on-time performance?

Time is one factor, it's also surprisingly easy to corrupt vfx master files with just a few missing bits. Checksums will identify something went wrong, and then you need to redownload the file to a quarantine network and scan it. Takes time. Much easier to go from trusted source to trusted source and verify the files on drive prior to shipping. Amazon just released a device (forget the name) for exactly the same use…

FYR Amazon has the AWS Snow Family with the largest being a 45ft TEU, while Microsoft does Azure Data Boxes that are somewhat smaller.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#277

Post by paulmd » Tue, Dec 29, 2020, 9:40 AM UTC

ddingus wrote: post #231

fernly wrote: post #99

Aw, c'mon chaps! Can we just admire the tech for a second? > Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch... > When tape is being read it is streamed over the head at a speed of about 15 km/h and with our new servo technologies we are still able to position the tape head with an accuracy that is about 1.5 times the width of a DNA molecule.

Absolutely. I had no idea tape was being pushed to these levels! Crazy good tech accomplishment right there. In a way, I am somehow comforted. Tape remaining relevant kind of dropped off my radar. Good to see. 317Gb / square inch AND 15km/h transport speed? That is nuts fast. Wonder how robust it really is? Of course at those densities and potential transfer rates there is ample room for error recovery.

It has never really entered the public consciousness since it's not a consumer-facing technology but tape development has continued apace. LTO-9 should be hitting the market very soon and supports up to 45TB per tape (compressed capacity, 18TB raw).

Not quite sure where IBM's numbers here come from, their previous numbers don't match up to the progression of the LTO tape series' capacity. Maybe they are citing "research numbers" that they can do in a lab but aren't production-ready yet. I would certainly assume they are citing "compressed" data figured there.

But certainly tape has continued to progress much faster than most people would have imagined. Big tape libraries are still a thing in certain environments and they work very well, there is no better solution for bulk cold storage.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#278

Post by dannyw » Tue, Dec 29, 2020, 9:43 AM UTC

wojciii wrote: post #244

dannyw wrote: post #240

Earlier quoted context omitted.

Ask HN: How can I create the most reliable and durable NAS today? I have a lot of very sentimental, very-important files, such as family photos and videos. And I simply like to hoard data. I currently have 8TB of data stored on a Synology DS218+ with RAID1, and monthly data scrubs (verifying checksums). It is backed up remotely to Google Drive (in encrypted form), and I also maintain an infrequently-updated, once-per…

I just built my Truenas box using Ryzen. Let me know if you want a part list for inspiration or any tips.

I would love your part list! Did you go with ECC memory? Did you have any way of verifying the ECC is working and actually detecting/correcting bitflips?

The other thing I am interested in is minimizing idle power consumption. Just to be more environmentally friendly.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#279

Post by paulmd » Tue, Dec 29, 2020, 9:45 AM UTC

ajnin wrote: post #152

fernly wrote: post #99

Aw, c'mon chaps! Can we just admire the tech for a second? > Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch... > When tape is being read it is streamed over the head at a speed of about 15 km/h and with our new servo technologies we are still able to position the tape head with an accuracy that is about 1.5 times the width of a DNA molecule.

> 15 km/h The tape stores 580TB for a length of 1255m, does that mean the read speed is 580TB/1255m*15km/h = 1.9TB/s ? Seems too high.

Yeah I've been wondering about the numbers too.

The numbers they cite for "previous generations" don't match up to the progression of the LTO tape series' capacity. Maybe they are citing "research numbers" that they can do in a lab but aren't production-ready yet. I would certainly assume they are citing "compressed" data figured there.

Also bear in mind that tapes typically store data striped across the tape in multiple tracks and multiple bands. There are four bands per tape and 12-52 wraps per band, so reading the whole tape requires up to 208 passes across the tape.

https://en.wikipedia.org/wiki/Linear_Tape-Open#Physical_stru...

But yes, to agree with another parallel comment, tape data rates are quite high sequentially (abysmal in random of course, but that's not how they're used). LTO-8 does 750 megabytes per second compressed / 360 megabytes per second raw.

Re: Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes

#280

Post by paulmd » Tue, Dec 29, 2020, 9:48 AM UTC

TedDoesntTalk wrote: post #199

texse wrote: post #179

Earlier quoted context omitted.

Most recent tape standards have multiple "bands" and "wraps" placed in parallel. The head reads only one wrap at a time, so it takes many passes to read the whole tape. For example an LTO-8 tape has 52 wraps each within 4 bands, requiring 208 passes to read completely.

Why not just have multiple heads then?

It would be like downloading a file using GetRight back in the day, where you'd have one thread downloading the 0-25% chunk, one downloading the 25-50% chunk, and so on.

You could hypothetically do that, for sure, but the software basically is derived from the tape era where you'd just have one logical stream coming out of the tape.