Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

Post reply on HN
Print view View original

231–240 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#231

Post by throwaway0a5e »

imglorp wrote: post #194
jaclaz wrote: post #173

Earlier quoted context omitted.

Or that actually happened, in the known Ford Pinto Memo "affair": https://en.wikipedia.org/wiki/Ford_Pinto#Cost–benefit_analys...

AND in the Suzuki Samuri rollover deaths (I knew one victim): where they knew there was an issue very early and had a "crisis plan". https://en.wikipedia.org/wiki/Suzuki_Motor_Corp._v._Consumer...

And meanwhile something like 20% of Bronco IIs got rolled and nobody cared. But as soon as someone puts an Explorer on it's roof or a Samurai barrel rolls through an intersection and there's a problem.

Pretty much that whole generation of midsize SUVs rolled easily. That's just one of the engineering trade-offs all those vehicles made and anyone can tell you that. If Consumer Reports was picking my SUV as the one to target I'd assemble a "get these people off our back" team too.

The media has made more than a few scandals out of pure normalcy over the years. The Frontline(?) gas cap thing comes to mind as a good example. Reality doesn't seem to matter nearly as much as the marketing team and lawyers you use to shape the public's perception is what matters. It's no wonder that corporations reaction to reported problems seems to always be to try to steamroll whomever is complaining and/or keep things on the down low.

Re: Canon's cloud platform has lost users' files and can't restore them

#232

Post by lukevp »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

Replication is about having data level redundancy to protect from drive failure. Backups are about having point in time snapshots of the system state, and about having them tiered from a location perspective. The 3-2-1[1] principle says to have 3 total copies, 2 of which are local but on different devices, and 1 of which is offsite. This gives you tiers of recoverability.

It’s important from a backup perspective that it’s point in time as well, otherwise as soon as you get ransomware that encrypts your file you now have replicated those changes everywhere.

[1] https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

Re: Canon's cloud platform has lost users' files and can't restore them

#233

Post by [deleted] »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

[deleted]

Re: Canon's cloud platform has lost users' files and can't restore them

#234

Post by chrismorgan »

zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

The original story spoke of the rumours of malware involvement, but in the update which forms the first half of the linked article, Canon says there was no malware involved.

(“Replication isn’t backup” still applies, of course.)

Re: Canon's cloud platform has lost users' files and can't restore them

#235

Post by geerlingguy »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

Classically, backups are taken then stored offline (e.g. on archive tape, on optical media, or on drives that are disconnected and put in storage between backups). Otherwise, if they are not 'cold' backups, they are still disconnected so if the main storage blows up, it won't impact the data in the backups.

Replication usually involves storage that is powered up and connected somehow to the same systems as the main storage; and any data that's corrupted on the main storage would propagate to the replicated storage.

Re: Canon's cloud platform has lost users' files and can't restore them

#236

Post by kgermino »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

I think the biggest factor is timing (although backups should also be on a second system).

In short, if anything that happens to the files is immediately copied to the "backup" then you don't actually have a chance to recover from any software problems. Whereas, if you make a copy of the data every night and keep the last 30 copies you can find an issue like this and go back in time to retrieve the files from before it started.

Re: Canon's cloud platform has lost users' files and can't restore them

#237

Post by ResidentSleeper »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

Backups protect against a wider class of problems. For example, a software vulnerability/bug or a human error could result in the deletion of an object in all replicas (because deletions are replicated too), but you'd usually need another incident to occur for you to lose the backup as well.

Re: Canon's cloud platform has lost users' files and can't restore them

#238

Post by Xylakant »

droopyEyelids wrote: post #220
throwaway0a5e wrote: post #180

Earlier quoted context omitted.

>Intent tends to matter with the law. Ford knew there was a problem, knew they could fix it, and elected not to because "it's a small number of deaths". You can replace things, not people. Per the link posted by the GP the Pinto did not have any problems relative to other cars of the same type and era. Ford's "intent" (if you can call it that since they didn't know the future at the time) was to say "err, we don't re…

Seems like you're failing to address the 'knew they were going to kill people' part. Everyone can always do better, but everyone doesn't know exactly which of their decisions will end up killing another person. If you DO know that, then you're obligated to change things.

Everyone entering a car today knows that given the wrong combination of events, they may kill someone, maybe even a toddler chasing a ball. Yet, millions of people still drive daily.

Even cyclists are not free from that risk - when I leave later and take my cargo bike to pick up my kid, I might be distracted in the wrong instant and run over someone else’s child. I try hard not to, but the safest way to reduce that risk would be to walk. Still, I don’t.

We all balance risks, even deadly ones, to us and others, all of the time.

Re: Canon's cloud platform has lost users' files and can't restore them

#239

Post by amiga_500 »

MaxBarraclough wrote: post #203
DaiPlusPlus wrote: post #54

Earlier quoted context omitted.

Google, as a company, is not reliable - I refuse to host anything critical on Google Drive (even G Suite) because there's zero recourse or accountability if a bug in Google's software or if Google arbitrarily decides to ban you for life from their services (e.g. due to misidentifying you from a dodgy YouTube user or repeated Google Play Store policy violations) then they will delete all of your personal and business…

What do you make of comparable services from other providers, say, Amazon Glacier?

I've used glacier. For stuff you don't need to restore in a rush, it's cheap.

I did however have a recent health scare and it made me wonder how my non tech wife could possibly have restored the files as the interfaces are all heavy.

Not a factor I'd previously considered when assessing my backup/restore.

Re: Canon's cloud platform has lost users' files and can't restore them

#240

Post by throwaway9d0291 »

ibeckermayer wrote: post #226
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

Replication is intended to create a 1:1 copy of the data. If data is removed from the primary system, replication will ensure it's removed from the replica.

A backup can be a 1:1 copy but it should be set up such that something going wrong with the primary (e.g. cryptolocker malware), can't affect it (since "something went wrong with the primary" is what the backup is intended to resolve).

To do that you could take the backup offline or use features like filesystem snapshots to ensure that changes can be rolled back.

Post reply on HN