Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

Post reply on HN
Print view View original

201–210 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#201

Post by tpolzer »

jacquesm wrote: post #160
tpolzer wrote: post #127

Earlier quoted context omitted.

The cloud is a backup. Just like any other backup you should test restore or at least verify the data regularly. I trust Google more than a random hard drive (to not lose random bits) in this respect. And unless your admin configured it away, GSuite allows takeout just like GMail afaik?

> GSuite allows takeout just like GMail afaik Hm, interesting ok, I will look into that. I didn't think of using takeout. Thank you. The only option I found before was to allow some random third party application access to all my files which was not an option.

It's a bit annoying for restore since it can take hours to be ready for download. Still better than Amazon Glacier though :D

Re: Canon's cloud platform has lost users' files and can't restore them

#202

Post by chongli »

jefftk wrote: post #189
deepstack wrote: post #150

Earlier quoted context omitted.

How hard can it be to have a set of data, make 2 copies and store the in different locations.

Reading their description of the problem, it wasn't that they lost one of their redundant servers, it was that their code that cleans up temporary storage accidentally was also cleaning up long-term storage. Increasing the redundancy of their storage wouldn't have helped.

If you have regular (daily, hourly) tape backups that are shipped off-site then you greatly mitigate the potential damage of software bugs. It sounds like they weren’t making backups and just treating failover replica servers as a “backup” (which they clearly aren’t).

Re: Canon's cloud platform has lost users' files and can't restore them

#203

Post by MaxBarraclough »

DaiPlusPlus wrote: post #54
dannyw wrote: post #20

Earlier quoted context omitted.

On topic topic, can I get HN’s advice on my backup strategy? All important files on Synology Ds218+ NAS, which has 2x12TB helium HDDs in mirror. Daily HyperBackup to Google Drive. I test restoring files on a monthly basis. Email alerts on failure. Anything I can improve?

Google, as a company, is not reliable - I refuse to host anything critical on Google Drive (even G Suite) because there's zero recourse or accountability if a bug in Google's software or if Google arbitrarily decides to ban you for life from their services (e.g. due to misidentifying you from a dodgy YouTube user or repeated Google Play Store policy violations) then they will delete all of your personal and business…

What do you make of comparable services from other providers, say, Amazon Glacier?

Re: Canon's cloud platform has lost users' files and can't restore them

#204

Post by jacquesm »

tpolzer wrote: post #201
jacquesm wrote: post #160

Earlier quoted context omitted.

> GSuite allows takeout just like GMail afaik Hm, interesting ok, I will look into that. I didn't think of using takeout. Thank you. The only option I found before was to allow some random third party application access to all my files which was not an option.

It's a bit annoying for restore since it can take hours to be ready for download. Still better than Amazon Glacier though :D

Better something than nothing. They must have changed this recently, even a few months ago attempts to bulk download Gsuite data led to error messages.

Re: Canon's cloud platform has lost users' files and can't restore them

#205

Post by ChuckNorris89 »

nix23 wrote: post #105
virgilp wrote: post #94

Earlier quoted context omitted.

They obviously do have _some_ backups (since were able to restore all videos and all photos - but not all photos at the original resolution). In the end it's all a tradeoff... ignoring cost, we all know what to do to get pretty safe storage. But how much are you really willing to pay, per month, for that safety?

>but not all photos at the original resolution Thats why you backup the original data and NOT the downscaled etc one. >But how much are you really willing to pay, per month, for that safety? You have to backup ALL your customers data, otherwise DONT go into file-hosting business.

With your argument, Google photos should also not be in business since they also don't back up your original resolution unless you pay.

Re: Canon's cloud platform has lost users' files and can't restore them

#206

Post by zxcvbn4038 »

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. I would not be surprised if all of this was running under someone’s desk.

Re: Canon's cloud platform has lost users' files and can't restore them

#207

Post by phone8675309 »

chrisseaton wrote: post #122
jacquesm wrote: post #113

Earlier quoted context omitted.

And there won't be at this rate. Besides, it is now a net negative for their users. 'The cloud is not a backup' should be a mantra that everybody that uses the cloud for their work and personal data be familiar with.

> The cloud is not a backup Of course the cloud is a backup. Why do you think it isn’t? Because it may break? All storage may break. The cloud is a backup. Like all backups, you need multiple independent backups.

321 rule for backups: 3 copies, 2 different media, at least 1 off site

Something is not a backup if you haven't tested that you can restore from it.

All backups not in your direct physical possession should be encrypted before being written.

Re: Canon's cloud platform has lost users' files and can't restore them

#208

Post by wegs »

toong wrote: post #101
visarga wrote: post #63

Earlier quoted context omitted.

Git is also incompatible with GDPR, you can't simply delete a file from all history.

Git allows you to rewrite history, that is not the problem. But you might have a hard time chasing down every copy, it's a distributed system by design.

Once you rewrite history, you've basically broken the git concept. It's an emergency feature if you've accidentally committed a security token or similar, or before you've pushed upstream, but if you're rewriting history with any sort of regularity, you're using git very, very wrong.

Re: Canon's cloud platform has lost users' files and can't restore them

#209

Post by galoisgirl »

jiofih wrote: post #102
nabla9 wrote: post #49

Earlier quoted context omitted.

This can be just calculated risk. Incredible damage for some users can be negligible damage to the company. In this case just 10TB of data was lost, so maybe thousands of users for the long term storage option. Losing customer data has potential for big reputation or brand damage, but surprisingly often the damage is relatively small. Putting in too much effort when potential damage to business is minimal may not be…

A “calculated risk” doesn’t make it okay when the risk is all on the consumer losing their memories. This corporate view on business risk alone is unethical.

There is no such thing as lack of risk in this world.

When a restaurant takes your reservation for Friday night, there's a 0.00001% (or so) risk that they will burn down on Thursday or that half the staff will get heart attacks or something. Should they not do reservations at all?

Re: Canon's cloud platform has lost users' files and can't restore them

#210

Post by galoisgirl »

jarym wrote: post #158

So today I learnt: 1. Canon don’t have any backup procedures in place for their cloud platform. Any hacker now will be salivating at the idea of pulling a ransomware hack I imagine. 2. Canon developers follow the ‘test in production’ methodology of continuous integration.

How do you know that? Gitlab had 5 backups and could restore none.
Post reply on HN