Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

Post reply on HN
Print view View original

191–200 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#191

Post by canofbars »

mixedbit wrote: post #46
klodolph wrote: post #2

Lesson for those designing cold-storage solutions--design your system, as much as reasonable, to not support deletion operations. The reason why files get lost is usually due to software bugs, bad configurations, and operator errors. Design your system to protect against these things.

Both AWS S3 and Google cloud storage buckets have an option that makes it impossible to delete stored objects for some period of time. The option was added for some legal compliance reasons, but I find it useful as an extra safeguard that important service data is not accidentally or maliciously deleted.

I noticed that gitlab added a change so that once you delete a repo, it holds it for over a week and puts a big "Pending Deletion" banner on the repo.

Re: Canon's cloud platform has lost users' files and can't restore them

#192

Post by jefftk »

virgilp wrote: post #94
nix23 wrote: post #69

Earlier quoted context omitted.

>budgets to engineer redundancies You dont need that, but a backup ;)

They obviously do have _some_ backups (since were able to restore all videos and all photos - but not all photos at the original resolution). In the end it's all a tradeoff... ignoring cost, we all know what to do to get pretty safe storage. But how much are you really willing to pay, per month, for that safety?

> since were able to restore all videos and all photos - but not all photos at the original resolution

"Canon has said that there is no technical measure to restore lost video files, but that photo files can be restored – albeit not at their original resolution."

Re: Canon's cloud platform has lost users' files and can't restore them

#194

Post by imglorp »

jaclaz wrote: post #173
wodenokoto wrote: post #156

Earlier quoted context omitted.

No, that comment sounds like the insurance job described in Fight Club[1] > Narrator: A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A…

Or that actually happened, in the known Ford Pinto Memo "affair": https://en.wikipedia.org/wiki/Ford_Pinto#Cost–benefit_analys...

AND in the Suzuki Samuri rollover deaths (I knew one victim): where they knew there was an issue very early and had a "crisis plan".

https://en.wikipedia.org/wiki/Suzuki_Motor_Corp._v._Consumer...

Re: Canon's cloud platform has lost users' files and can't restore them

#195

Post by agumonkey »

vidarh wrote: post #186
agumonkey wrote: post #141

Earlier quoted context omitted.

I wonder if they discuss that and actually write down calculations

I've designed a storage system for an image storage app for a client, and I had an extensive spreadsheet with the cost and risks of various combinations of erasure coding and data centres, as well as other mitigations. So yes. But this data loss highlights that all too often it ends there, with calculating the risk of loss due to hardware or data centre failures. You need to also do the same in terms assessing how yo…

But was this framed as easy business from gambling with users artefacts ? or just the usual "shit happens" preliminary thinking ?

Re: Canon's cloud platform has lost users' files and can't restore them

#196

Post by Bobbcatt »

kohtatsu wrote: post #145
wruza wrote: post #84

Earlier quoted context omitted.

Or what? They could and they did.

Or they could like, not. I think there's a world where corporations aren't given a carte blanche pass on human decency. I believe that world is nicer and where possible I work to give it room to exist; I boycott Disney because I believe what they did to the copyright system is wrong. I don't use Google or Microsoft products because I don't think they treat users with respect. (Minus youtube which I use a mirror, and…

>But then again maybe someone was just getting into photography after having a baby, and their precious first photos were lost! Or photos from a special date, or first (god forbid last) moments with a pet; it's sad to think about,

Things die and change. Holding onto the past with photographs is not healthy.

Re: Canon's cloud platform has lost users' files and can't restore them

#197

Post by bluedino »

I think people would be surprised at the amount of backup or file storage services that keep everything in one data center. Your files might be redundant across machines but it's not like they have a second data center somewhere with another copy of all your data, and they certainly don't backup everything up to tape or something.

Re: Canon's cloud platform has lost users' files and can't restore them

#198

Post by vsareto »

jarym wrote: post #158

So today I learnt: 1. Canon don’t have any backup procedures in place for their cloud platform. Any hacker now will be salivating at the idea of pulling a ransomware hack I imagine. 2. Canon developers follow the ‘test in production’ methodology of continuous integration.

>Canon developers follow the ‘test in production’ methodology of continuous integration.

Assuming they are making those calls

Re: Canon's cloud platform has lost users' files and can't restore them

#199

Post by DaiPlusPlus »

rightbyte wrote: post #170
DaiPlusPlus wrote: post #54

Earlier quoted context omitted.

Google, as a company, is not reliable - I refuse to host anything critical on Google Drive (even G Suite) because there's zero recourse or accountability if a bug in Google's software or if Google arbitrarily decides to ban you for life from their services (e.g. due to misidentifying you from a dodgy YouTube user or repeated Google Play Store policy violations) then they will delete all of your personal and business…

I have bad experiences encrypting backup drives. If they corrupt a little the data might be gone. You also have the problem of keeping track of the key for years.

Protip: write the passphrase in sharpie on the slither of whitespace left available on the drive’s spec sticker

Re: Canon's cloud platform has lost users' files and can't restore them

#200

Post by qntmfred »

hvidgaard wrote: post #73
hellofunk wrote: post #66

There is one somewhat reassuring lesson here -- No matter the company, no matter the brand, no matter their worth: software development is wrought with challenges, and major mistakes for seemingly obvious things are made by everyone, all the time, at every level.

This is not one of them. Ensuring proper data loss protection is something both Amazon and Google are very good at. A company of Canons size could easily get the proper guidance to avoid it. But it would not have been as cheap as storing the files on your own servers. So this happened because Canon did not want to spend the money on doing it right with offline backups or append only storage in a cloud with enough nin…

Companies of Canon's size can be constrained for resources just as much as any other company. Canon is largely a legacy company at this point, with significantly declining revenue the last several years. https://i.redd.it/lrlfen2x7sm31.jpg
Post reply on HN