Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

Post reply on HN
Print view View original

81–90 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#81

Post by imdsm »

akerro wrote: post #51
Sharlin wrote: post #28

Earlier quoted context omitted.

According to TFA (which apparently nobody reads) it most certainly was an engineering mistake. Files in long-term storage were accidentally subject to the same code that deletes files in short-term (30 day) storage.

It's sound similar to "junior dev had access to prod database" scenario which really isn't engineering mistake but process/administration.

It seems more likely that there was some process that performed a "delete older than 30 days" on the long term storage. Likely untested code. That's what the article says anyway, and we all know that the public message is always the best spin on a scenario.

Re: Canon's cloud platform has lost users' files and can't restore them

#82

Post by plutonorm »

kohtatsu wrote: post #62
nabla9 wrote: post #49

Earlier quoted context omitted.

This can be just calculated risk. Incredible damage for some users can be negligible damage to the company. In this case just 10TB of data was lost, so maybe thousands of users for the long term storage option. Losing customer data has potential for big reputation or brand damage, but surprisingly often the damage is relatively small. Putting in too much effort when potential damage to business is minimal may not be…

This comment reads as "This is normal; nothing to see here move along". Which is more of an indictment on the state of things than on the comment itself. If you can't protect people's shit, don't offer to hold onto it for money!

Your comment is basically saying capitalism is bad.

I agree.

Re: Canon's cloud platform has lost users' files and can't restore them

#84

Post by wruza »

kohtatsu wrote: post #62
nabla9 wrote: post #49

Earlier quoted context omitted.

This can be just calculated risk. Incredible damage for some users can be negligible damage to the company. In this case just 10TB of data was lost, so maybe thousands of users for the long term storage option. Losing customer data has potential for big reputation or brand damage, but surprisingly often the damage is relatively small. Putting in too much effort when potential damage to business is minimal may not be…

This comment reads as "This is normal; nothing to see here move along". Which is more of an indictment on the state of things than on the comment itself. If you can't protect people's shit, don't offer to hold onto it for money!

Or what? They could and they did.

Re: Canon's cloud platform has lost users' files and can't restore them

#85

Post by msh »

visarga wrote: post #63
chrismatheson wrote: post #59

Earlier quoted context omitted.

I suppose that feature would be incompatible with user data in a GDPR world ??

Git is also incompatible with GDPR, you can't simply delete a file from all history.

Git is not really intended for personal data

Re: Canon's cloud platform has lost users' files and can't restore them

#86

Post by msh »

dannyw wrote: post #20
SturgeonsLaw wrote: post #7

I take nightly images of my computer's primary drive and replicate them offsite, with alerting if it fails and automated scheduled backup restore testing. It blows my mind to think that some random schmuck like me has better backup procedures than these multibillion dollar global corporations. Companies that don't invest in their IT end up paying for it tenfold down the track.

On topic topic, can I get HN’s advice on my backup strategy? All important files on Synology Ds218+ NAS, which has 2x12TB helium HDDs in mirror. Daily HyperBackup to Google Drive. I test restoring files on a monthly basis. Email alerts on failure. Anything I can improve?

Some way todo a offline snapshot a few times a year.

Re: Canon's cloud platform has lost users' files and can't restore them

#87

Post by SifJar »

Sharlin wrote: post #25
klodolph wrote: post #2

Lesson for those designing cold-storage solutions--design your system, as much as reasonable, to not support deletion operations. The reason why files get lost is usually due to software bugs, bad configurations, and operator errors. Design your system to protect against these things.

To be fair, image.canon is definitely not a cold-storage or backup solution, never mind primary storage. Its main use case is acting as a buffer for auto-uploading photos from camera and then later downloading them to a computer. As an afterthought it also offers a tiny long-term storage space, 10 GB per user, smaller than most memory cards available today.

It also appears that only images stored for >30 days were affected, so in theory most photos "should" have been downloaded and stored properly etc. if people are using the service as you describe

Re: Canon's cloud platform has lost users' files and can't restore them

#88

Post by wruza »

jedberg wrote: post #41
StreamBright wrote: post #26

Earlier quoted context omitted.

Or maybe an engineering mistake? I do not think you can claim anything without knowing the details. AWS lost customer data too, I would not try to assign blame without reading the post mortem.

An engineering mistake combined with an obvious lack of backups.

This reminds me of that oracle thread, where a guy had lots of backups, all encrypted with a key stored in RAM.

Re: Canon's cloud platform has lost users' files and can't restore them

#89

Post by eru »

CalRobert wrote: post #5
klodolph wrote: post #2

Lesson for those designing cold-storage solutions--design your system, as much as reasonable, to not support deletion operations. The reason why files get lost is usually due to software bugs, bad configurations, and operator errors. Design your system to protect against these things.

If memory serves, a class of Glacier storage (AWS) isn't practical to use in the EU because the inability to delete makes it impossible to serve "Right to Erasure" GDPR requests.

I suspect you can square the circle, if you encrypt all the files, and store the keys in storage you can delete quickly.

Re: Canon's cloud platform has lost users' files and can't restore them

#90

Post by nabla9 »

kohtatsu wrote: post #62
nabla9 wrote: post #49

Earlier quoted context omitted.

This can be just calculated risk. Incredible damage for some users can be negligible damage to the company. In this case just 10TB of data was lost, so maybe thousands of users for the long term storage option. Losing customer data has potential for big reputation or brand damage, but surprisingly often the damage is relatively small. Putting in too much effort when potential damage to business is minimal may not be…

This comment reads as "This is normal; nothing to see here move along". Which is more of an indictment on the state of things than on the comment itself. If you can't protect people's shit, don't offer to hold onto it for money!

I try to avoid normative statements because it leads to better discussion. Unfortunately this style of discussion leads to comments that project normative stance, or criticize me from not making my normative position clear.

(positive statement is statement of what without indication of approval or disapproval, normative statement expresses a value judgment)

Post reply on HN