Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Canon's cloud platform has lost users' files and can't restore them

digitalcameraworld.com

Post reply on HN
Print view View original

321–330 of 419 posts

Re: Canon's cloud platform has lost users' files and can't restore them

#321

Post by fokinsean »

ghaff wrote: post #142
ChrisMarshallNY wrote: post #53

Earlier quoted context omitted.

I have a multi-dimensional backup strategy, which includes “cloudy” backups of my most important assets, using services like GitHub, local hourly NAS (Synology double-redundant RAID-like) incremental backups of my whole system, and running “hot” backups of my system and development volumes; automatically updated every four hours (or on-demand. It's a bootable CCC clone). I seldom need to use anything more than my “ho…

A problem with digital media as well is that it encourages you to just save everything but it's easier that way. But as I'm discovering well into the digital era, I wish I had more curated photos and video--though that comes with its own problems.

I dread the day I decide to go through all my photos and organize them. It's one reason why I'm so reluctant to move off of google photos, it's sooo easy to find stuff and it groups photos together nicely.

But I really want to move my photos off of google.

Re: Canon's cloud platform has lost users' files and can't restore them

#322

Post by nicoburns »

syshum wrote: post #305
nicoburns wrote: post #287

Earlier quoted context omitted.

Courts don't just mediate. Their judgements are backed by the power of the state (otherwise a thief could simply ignore them). And this is absolutely somebody having a positive right (to have their property ownership defended) that infringes on someone else's negative right (to take their property without the state interfering). It happens to be a right that I agree with. But I don't think you can differentiate it wi…

>>that infringes on someone else's negative right (to take their property without the state interfering) That is not a negative right, it is not a right at all. The entire concept of rights, as understood by common law, is based on Natural Rights under a Lockean philosophy of Self Ownership, that you own yourself. Under no conceptional understanding of natural rights would have have the right to take someone else's p…

Rights are a much broader concept than "natural rights". See: https://plato.stanford.edu/entries/rights/ I certainly recognise property rights (although in principle there's no reason why one couldn't believe in rights without that being one of them), but I also recognise a lot of other rights (including "positive" rights). And don't recognise negative rights as being more fundamental than positive rights.

The positive/negative distinction

Re: Canon's cloud platform has lost users' files and can't restore them

#323

Post by bscphil »

wtracy wrote: post #303
bscphil wrote: post #292

Earlier quoted context omitted.

> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…

By "raw" I think GP meant the files originally provided by the users, not actual RAW files.

That might be the case. Admittedly I don't know much about this cloud platform, and the article doesn't fill in a lot of details, but I would naively have assumed the target audience for something like this would be pro and semi-pro users, who would want to store RAW files, since amateurs and those just looking to share photos would get more out of Flickr, or even Google Photos / Facebook / Twitter / etc.

Re: Canon's cloud platform has lost users' files and can't restore them

#324

Post by klodolph »

IshKebab wrote: post #260
klodolph wrote: post #2

Lesson for those designing cold-storage solutions--design your system, as much as reasonable, to not support deletion operations. The reason why files get lost is usually due to software bugs, bad configurations, and operator errors. Design your system to protect against these things.

Good luck complying with the GDPA.

Do you mean GDPR? You don’t have to delete data immediately under the right to be forgotten.

Re: Canon's cloud platform has lost users' files and can't restore them

#325

Post by klodolph »

mschuster91 wrote: post #38
klodolph wrote: post #2

Lesson for those designing cold-storage solutions--design your system, as much as reasonable, to not support deletion operations. The reason why files get lost is usually due to software bugs, bad configurations, and operator errors. Design your system to protect against these things.

The problem with that is that under GDPR you have to support full deletion of an user's data.

Crucially, you do not have to do it immediately.

Re: Canon's cloud platform has lost users' files and can't restore them

#326

Post by JulianWasTaken »

munk-a wrote: post #313
JulianWasTaken wrote: post #307

Earlier quoted context omitted.

We don't think AWS does periodic offline backups of stuff stored in S3 so that they don't find themselves in this exact embarrassing scenario? Regardless of whether it's user-facing for AWS users, I'd hope they do, or certainly that they did before they got a good sense of the long-term reliability of S3 as a big system.

I think they do all sorts of backup stuff - and as a consumer of S3 you need to evaluate how resilient their backing up is. S3 is pretty murky about permanence guarantees so I'd always look at making sure there is a separately maintained replication script to some medium I have control over if that data is irreplaceable and the costs of losing it are significant to the business. Judging those costs and making that ca…

What would such a service then do as an end-user beyond what `aws s3 sync s3://my/bucket /mnt/my/backup/drive/` does?

(Honest question, not provocation. To me either you trust AWS or any party to never lose your data [unwise], or you basically ask them to offer a way to rsync, which AWS does)

Re: Canon's cloud platform has lost users' files and can't restore them

#327

Post by teawrecks »

bscphil wrote: post #292
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…

I'm not sold. In the cases where you lose the raw vs a post processed image, one of those is significantly more lost information. It may be non trivial to reproduce a post process image from a raw image, but the other way around is usually impossible.

In one case you've lost original sensor information, in the other case you've lost some parameters and possibly the algorithm used. The former is infinitely more difficult to synthesize from scratch.

Re: Canon's cloud platform has lost users' files and can't restore them

#328

Post by rsync »

crazygringo wrote: post #271
zxcvbn4038 wrote: post #206

This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…

Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…

"Do any cloud providers create backups on top of replication though?"

We do.[1]

For exactly 1.75x our normal pricing, we will replicate your entire account, nightly, to a geo-redundant site which is not open for normal customer use (and, therefore, has a lower risk profile). This GR site is the he.net core datacenter, in Fremont.

It's also worth pointing out that replication to rsync.net buys you malware/ransomware protection since your account is snapshotted, by ZFS, nightly, and those snapshots are immutable (read-only).

[1] rsync.net

Re: Canon's cloud platform has lost users' files and can't restore them

#329

Post by syshum »

nicoburns wrote: post #322
syshum wrote: post #305

Earlier quoted context omitted.

>>that infringes on someone else's negative right (to take their property without the state interfering) That is not a negative right, it is not a right at all. The entire concept of rights, as understood by common law, is based on Natural Rights under a Lockean philosophy of Self Ownership, that you own yourself. Under no conceptional understanding of natural rights would have have the right to take someone else's p…

Rights are a much broader concept than "natural rights". See: https://plato.stanford.edu/entries/rights/ I certainly recognise property rights (although in principle there's no reason why one couldn't believe in rights without that being one of them), but I also recognise a lot of other rights (including "positive" rights). And don't recognise negative rights as being more fundamental than positive rights. The positi…

It is not a matter of them being more "fundamental"

There is no way for a government to provide you with healthcare (positive rights) with out infringment of my negative rights to my labor (i.e they must take a part of my labor via theft what you call taxation in order to provide you with the right to a service). The only other way for a government to provide a service would be indentured servitude where they require a person to provide the service under threat of violence / imprisonment

That is why I reject the concept of positive rights, because they require that other people have their labor, property, or self ceased by the government to fulfill those rights that is fundamentally unethical to me

negative rights do not have this ethical problem

Re: Canon's cloud platform has lost users' files and can't restore them

#330

Post by munk-a »

JulianWasTaken wrote: post #326
munk-a wrote: post #313

Earlier quoted context omitted.

I think they do all sorts of backup stuff - and as a consumer of S3 you need to evaluate how resilient their backing up is. S3 is pretty murky about permanence guarantees so I'd always look at making sure there is a separately maintained replication script to some medium I have control over if that data is irreplaceable and the costs of losing it are significant to the business. Judging those costs and making that ca…

What would such a service then do as an end-user beyond what `aws s3 sync s3://my/bucket /mnt/my/backup/drive/` does? (Honest question, not provocation. To me either you trust AWS or any party to never lose your data [unwise], or you basically ask them to offer a way to rsync, which AWS does)

I think that's essentially what such a service would do. You might throw in some periodic less frequent syncs (maybe you sync down to the main backup every day and sync that backup to a secondary backup weekly or monthly) and maybe some of those successive syncs are done to hosts that are usually disconnected from the network to add in a firebreak.
Post reply on HN