Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

371–380 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#371

Earlier quoted context omitted.

If you retrieve the data slowly it's cheap. It's expensive if say you are a retailer who needs their database restored asap. I use a Linux perl client!

I suppose the biggest risk is failure-to-pay, as with all cloud backups/storage. If I allow my payment card to expire, Amazon aren't obligated to continue to store my data. If I drop off the grid for an extended holiday, that could be a real risk. To my knowledge, Amazon offer no means of prepaying.

> If I allow my payment card to expire, Amazon aren't obligated to continue to store my data.

Business idea: cloud archive storage where you pay when you upload data and optionally pay a modest monthly fee for real-time access to stored data, but they'll guarantee to keep your data for you if you stop paying: you'll just need to pay to retrieve that data.

As the long-term archival data wouldn't need to be stored in a data-center: just a commodity tape-library box in a basement in a farm somewhere near a freeway I imagine it would be kinda cheap to run as a business. You could set-up a Foundation or other entity to ensure long-term continuity of operations and have it self-sufficient through an endowment. E.g. a $1m endowment would easily pay for something like this into perpetuity.

Re: Canon's cloud platform has lost users' files and can't restore them

#372

Earlier quoted context omitted.

Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…

I haven't looked at dropbox in a while, but they at least used to have an option for keeping revisions. It did come with an extra cost.

I'm completely soured on dropbox since they dialed the greed up with the arbitrary device count limit and nagging suggestions to upgrade that can't be turned off if you're near the max. I don't care enough to stop using it but I will never give them money after this.

Re: Canon's cloud platform has lost users' files and can't restore them

#373

Earlier quoted context omitted.

> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…

> Even open source programs, like RawTherapee and Darktable, use entirely different and incompatible approaches, algorithms, and instructions. `git checkout v3.0.2` `./build.sh` No idea what you're talking about. As long as you have your RAWs and your sidecar file you can trivially reproduce the picture, at least that's how it works in darktable. Hell, the old code that is deprecated and "hidden" in the newest versio…

I mean that RawTherapee and Darktable don't use the same code as each other at all. The instructions (usually sidecar files) the tell each program what to do to generate the output are completely incompatible.

> the old code that is deprecated and "hidden" in the newest versions is actually all still there and if you import your stuff in the newest version

I'd expect this to be the case with any good photo developer, but it doesn't pay to take chances. There have been significant bugs in the past with DNG files, for example, and different bugs could be introduced in the future, or bugs you didn't know about could be fixed, and so on. There's a lot of stuff that could go wrong. It's good to be able to just tar the source code.

Re: Canon's cloud platform has lost users' files and can't restore them

#374

Earlier quoted context omitted.

"If you can't protect people's shit, don't offer to hold onto it for money!" If you can't sell a reliable car, don't make one? Products fail sometimes and we in the tech-sphere get to laugh about it. No such thing as 100% failure proof. Customers that paid for the service would be angry but no doubt, they'd get their money back. Yes they might have lost more, but that's the business sometimes.

There's a price point out there for everybody, and the lower ones are not very secure I imagine. What's missing is, some certification level that tells you how secure the data is. So you know what you're buying into?

You mean like an SLA which says the availability of your data?

Re: Canon's cloud platform has lost users' files and can't restore them

#375
post #368

Earlier quoted context omitted.

Everyone entering a car today knows that given the wrong combination of events, they may kill someone, maybe even a toddler chasing a ball. Yet, millions of people still drive daily. Even cyclists are not free from that risk - when I leave later and take my cargo bike to pick up my kid, I might be distracted in the wrong instant and run over someone else’s child. I try hard not to, but the safest way to reduce that r…

We balance risks which have uncontrollable inputs. Not controllable ones for which we now have an answer. That's the difference. You note it right in your comment - that you in fact do take measures to be safe, and try to be specifically aware of elements of the dynamic situation which require attention. That's very different to say, if you planned on reversing a car and decided it took too much effort to look behind…

>Not controllable ones for which we now have an answer.

We do it all the time?

How far are you from the nearest fire extinguisher? Can't that number be reduced? That is but one example.

Re: Canon's cloud platform has lost users' files and can't restore them

#376

Earlier quoted context omitted.

There's a price point out there for everybody, and the lower ones are not very secure I imagine. What's missing is, some certification level that tells you how secure the data is. So you know what you're buying into?

You mean like an SLA which says the availability of your data?

SLAs are bullshit.

Refunds are fine for retail customers, and most SLAs I’ve ever had the need to discuss seem to be based on that model.

If I buy a computer from you for $1000 and it breaks, I’m out $1000, I want my money back. Vendor might be fine with that. If I buy a commercial oven from you for $40,000 and it breaks I’m out the price of the oven, plus half of my sales for a week while it gets repaired, wages for people who are now half useless to my business, and loss of trust and/or customers for my company since I had none of my signature baked goods for the entire week.

I don’t want SLAs, I want a warranty. Insurance.

Re: Canon's cloud platform has lost users' files and can't restore them

#377

Earlier quoted context omitted.

There is no such thing as lack of risk in this world. When a restaurant takes your reservation for Friday night, there's a 0.00001% (or so) risk that they will burn down on Thursday or that half the staff will get heart attacks or something. Should they not do reservations at all?

Losing a restaurant reservation on a single night when surrounded by alternatives (aka - lots of backup plans) is not the same as losing data that people trust as the backup plan for their data - the loss of which causes a lot more harm. Especially if it’s a respected brand.

“I lost your reservations” is so much less serious than “I lost the pictures of grandma’s birthday”

Re: Canon's cloud platform has lost users' files and can't restore them

#378
post #27

Earlier quoted context omitted.

The manager probably put five 9s onto the spec sheet, and provided absolutely no resources. You’ll never guess who’s going to get the heat for this, the manager or the poor person who didn’t follow the spec.

A reliability guarantee I like to call "nine fives".

There was an article about “five 8’s” during the Trough of Disillusionment for the original hype cycle.

Re: Canon's cloud platform has lost users' files and can't restore them

#379
post #51
post #28

Earlier quoted context omitted.

According to TFA (which apparently nobody reads) it most certainly was an engineering mistake. Files in long-term storage were accidentally subject to the same code that deletes files in short-term (30 day) storage.

It's sound similar to "junior dev had access to prod database" scenario which really isn't engineering mistake but process/administration.

I knew a guy who used a different color scheme for terminal sessions connected to production servers, so he didn’t do something in the wrong window.

A guy. One. How many people have I met with shell access to production servers?

Re: Canon's cloud platform has lost users' files and can't restore them

#380

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…

This was a fascinating, but also disturbing, read; it does sound like your solution (quasi legal as it might be) is the one that makes the most sense if you do this professionally
Post reply on HN