241–250 of 335 posts
Last I looked into this, if you had a lot of data to back up (Say 1 Gbyte per second, continuous, with a retention time of 1 year), it was still far cheaper to simply use hard drives. One employee can keep up with all drive replacements, hardware setup, etc with time to spare. Drives aren't super power hungry, so any old office building is suitable. Encrypt the drive contents on another site so you don't need 24/7 se…
Earlier quoted context omitted.
Given the pace at which storage capacity increases, what is the rationale for not copying over your data every 5-10 years onto the next cheapest consumer mass storage of the moment? You get all your data in one place and you don’t have to deal with standards disappearing (I read 2020’s game consoles can’t read CDs anymore, people should rip their CD collection right now).
The bookkeeping it requires for one, since you don't usually buy all your backup media at once, you acquire them over time, that gets unnecessarily complicated. It's riskier to copy the media periodically as you might increase the chances of data corruption due to the fault in the copying process (faulty RAM, faulty software, not concentrating good enough etc). You periodically introduce possibility of user/hardware/…
For book keeping, I think my point is that storage media are becoming so big that you always consolidate into a single device every time you carry the data over (you may still want to duplicate for reliability). Like you can buy a 18TB hard drive today. A consumer isn't going to require more than one or perhaps two of those for anything to be preserved long term. And in 5y-10y, you will likely have 25-30TB hard drives.
The equipment problem is precisely what this addresses. You are always using the latest hardware, and the previous hardware you are using is still supported if you stick to a 5-10y cycle. For instance you would have moved away from IDE drives while you could still find motherboards with both IDE and SATA ports. But if your data is stored on an IDE drive, good luck connecting it on a computer in 2030 (if we haven't moved full Apple's "you can't customize your hardware and we deprecate everything very frequently").
Skills (and I would say mostly dedication) is still a problem. But we are talking about copy-pasting files between two media, it's not rocket science even if you don't script it.
Earlier quoted context omitted.
HDDs can't require powering all that periodically. I just found my hard drive from college, and plugging it into my computer just worked. Great finding all my old documents and music that I'd forgotten about.
If your drive uses ZFS you apparently need to power it on more regularly than that to avoid bit rot.
I'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings m…
Ask HN: How can I create the most reliable and durable NAS today? I have a lot of very sentimental, very-important files, such as family photos and videos. And I simply like to hoard data. I currently have 8TB of data stored on a Synology DS218+ with RAID1, and monthly data scrubs (verifying checksums). It is backed up remotely to Google Drive (in encrypted form), and I also maintain an infrequently-updated, once-per…
I'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings m…
Ask HN: How can I create the most reliable and durable NAS today? I have a lot of very sentimental, very-important files, such as family photos and videos. And I simply like to hoard data. I currently have 8TB of data stored on a Synology DS218+ with RAID1, and monthly data scrubs (verifying checksums). It is backed up remotely to Google Drive (in encrypted form), and I also maintain an infrequently-updated, once-per…
I'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings m…
Ask HN: How can I create the most reliable and durable NAS today? I have a lot of very sentimental, very-important files, such as family photos and videos. And I simply like to hoard data. I currently have 8TB of data stored on a Synology DS218+ with RAID1, and monthly data scrubs (verifying checksums). It is backed up remotely to Google Drive (in encrypted form), and I also maintain an infrequently-updated, once-per…
Go with option 1, and Raid-Z2+. With RaidZ2, you’ll be able to not only detect but correct a flipped bit - Even if that flip happens when writing out the data.
Pay attention to the counters. Your ZFS scrubs will report how many resilvers they have. You’re likely to encounter 1 in a scrub. You’re unlikely to encounter more than 2. If you see that, that’s when you check for memory errors. A single bad sector is likely the hard drive. Even a single flipped bit is likely a transient error; It could be your memory, or your disk, or anything in between. It happens at scale, and 8TB, read repeatedly, is a lot of bits.
Look into rasdaemon and memtest86 - They’re the tools you use to debug what happens when
The other advice I can give you: Don’t be paranoid. Your photos are likely to acquire bit rot. You will have dozens, even hundreds of bit flips that will happen in your lifetime. Of the many thousands of photos you will take, the chances that you will ever notice the discoloration or bad line that a bit flip will have in a photo are pretty small. Bit rot happens. Your photos are important to you, and you should treasure them, but treasure them for what they are: Things that you protect and are under your care, not things that must be twenty nines of correct. You can realistically achieve 10, even 12 nines of correct reads on your data. You don’t need more.
I'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings m…
Ask HN: How can I create the most reliable and durable NAS today? I have a lot of very sentimental, very-important files, such as family photos and videos. And I simply like to hoard data. I currently have 8TB of data stored on a Synology DS218+ with RAID1, and monthly data scrubs (verifying checksums). It is backed up remotely to Google Drive (in encrypted form), and I also maintain an infrequently-updated, once-per…
"Next, you read a copy of the same block – this copy might be a redundant copy, or it might be reconstructed from parity, depending on your topology. The redundant copy is easy to visualize – you literally stored another copy of the block on another disk. Now, if your evil RAM leaves this block alone, ZFS will see that the second copy matches its checksum, and so it will overwrite the first block with the same data it had originally – no data was lost here, just a few wasted disk cycles. OK. But what if your evil RAM flips a bit in the second copy? Since it doesn’t match the checksum either, ZFS doesn’t overwrite anything. It logs an unrecoverable data error for that block, and leaves both copies untouched on disk. No data has been corrupted. A later scrub will attempt to read all copies of that block and validate them just as though the error had never happened, and if this time either copy passes, the error will be cleared and the block will be marked valid again (with any copies that don’t pass validation being overwritten from the one that did)."
(dont just read the quote, read the link)
I am using ZFS since 2007/2008 and I have never had any issues (except with those damn Seagate 3tb DeathStar hdds, where I was barely able to replace them fast enough - in 3 months 3 were gone - i will never buy seagate again)
Co-creator of ZFS: https://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=...
I am having a asus microatx board with 16gb of non ecc ram, additional SAS HBA and 3x3Tb Toshiba into zraid and additional 10TB HGST He3 disks + Ultrium 3000 (LTO-5, they are quite cheap today and are designed for 24/7 operation which is barely what I will ever encounter while tapes can be restored on latest LTOs if needed, while you can get cartridges on some sale for peanuts) for backups. There is no way in hell to go for my important data (like images) to disk only and tape is nice. You take the cartridge and store it at parents/gf/worspace drawer/...
Anyway if I remembered correctly, google has lost 150k of user accounts in 201x and restored them from tapes. So even for cloud opinionated people, it still makes sense to shovel important data to tapes if you dont use the data in everyday processing (and just an info - even shelved disks die)
Earlier quoted context omitted.
If you're flying back and forth on that route that often, you're going to get upgraded quickly, I would imagine. I've done 6 flights a month domestic and have been upgraded to first class for free fairly often.
Depends on the rewards program I suppose. A US airline I’m familiar with does not upgrade frequent fliers on international flights.
I'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings m…
There really isn't much market for it. You can pay Google or Apple or one of the large cloud providers a very reasonable and decreasing rate for a literal guarantee that your data is accessible. The only risk is the company goes under, which is extraordinarily unlikely for someone like Google / Apple and the shutdown would have advance notice. I realize for the hacker news audience there are multitudes of reasons the…
A single bad bug or security issue can make data inaccessible or corrupt. Tapes on the other hand does not have that issue. IMHO trusting all your important data to a single vendor or technology is a recipe for disaster.
Earlier quoted context omitted.
Unattended for 14+ hours you mean. A baggage handler on the departing side could be bribed to steal it and you would have half a day of unfettered access before the courier on the plane even knew it was missing. Of course you don’t check it if it’s that critical.
Transport security wasn't as big a concern as consistent delivery. Network volitility is a huge problem when transferring huge files.