301–310 of 419 posts
Earlier quoted context omitted.
Don't store your users PII in a git repository then.
The use case is to do dataset versioning for ML. The dataset itself is updated frequently. It would be nice to use a tool that can store efficiently when small changes are made yet allow versioning for reproducibility.
This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…
Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…
This sounds to me like the classic “replication is not backups” situation where (at best) all of the user files were stored in RAID array someplace and that is what the malware ate. If there had been actual backups and effective backups then it should have been trivial to restore non-corrupted files. It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually…
> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…
Earlier quoted context omitted.
>> Enforcing property laws is subject to limits on policing resources. Incorrect, in the US I have no right to policing either, I have the right to defend my property myself from people that wish to take it. Our Courts have held continuously that no individual has the right to protection from the police force. We do not have Positive Rights in the US, never have and hopefully never will because in order to have posit…
Courts don't just mediate. Their judgements are backed by the power of the state (otherwise a thief could simply ignore them). And this is absolutely somebody having a positive right (to have their property ownership defended) that infringes on someone else's negative right (to take their property without the state interfering). It happens to be a right that I agree with. But I don't think you can differentiate it wi…
That is not a negative right, it is not a right at all.
The entire concept of rights, as understood by common law, is based on Natural Rights under a Lockean philosophy of Self Ownership, that you own yourself.
Under no conceptional understanding of natural rights would have have the right to take someone else's property.
So if we are going to start from a completely foreign concept of negative rights then we have no basis for a conversation at all
https://www.learnliberty.org/videos/positive-rights-vs-negat...
Earlier quoted context omitted.
Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…
>> we're talking maybe exabytes I don't see the size as the big problem. If one cloud can handle the size, the backup storage system can too. For me, the real issue is timing. How do you backup a cloud full of constantly changing data? Do you draw a line in the sand, an image of the cloud state at a particular moment? Is that even possible? You have to do backups of smaller chunks, individual accounts, but eventually…
If you even do the straightforward "don't freeze time anywhere, just rolling copy all the files onto a hard medium", you may end up with logically inconsistent data, in that files from later in the backup may actually not have existed alongside ones from earlier ones, but if you write down the timestamps periodically, you end up with a way better end state than "I lost all the files".
At worst, you end up with "I may have lost the last 3 minutes worth of files from the last 3 minutes of the last backup I did".
Earlier quoted context omitted.
Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…
Just using AWS S3 as an example, you can copy a bucket to another bucket (preferably in a different region) with a policy of retaining all versions- preventing the problem of a delete whether malicious or accidental, from being reflected in the backup copy. There is also the option of using Glacier for lower cost long term storage. If you are asking does regular S3 provide a true backup capability out of the box, it…
Earlier quoted context omitted.
Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…
> Do any cloud providers create backups on top of replication though? Yes - I work at one of the FAANGs for a team that's doing precisely this. We develops an internal disaster recovery tool that creates backups of data files that can't be touched by the creating application, and that can be read back in a disaster event to recover the data.
Earlier quoted context omitted.
> It also sounds like someone made the decision not to backup the raw images because they were “big” - that is actually the one thing they should have backed up because all of the smaller files can be regenerated from the raw ones. Ironically my experience has been exactly the opposite. It's the demosaic-ed, fully developed copies of my photos that are larger and harder to preserve than the original RAWs. And these f…
By "raw" I think GP meant the files originally provided by the users, not actual RAW files.
Earlier quoted context omitted.
Do any cloud providers create backups on top of replication though? Backing up databases (terabytes) is feasible, they're not that big. But an entire cloud storage for photo and video for millions, we're talking maybe exabytes. The notion of making "separate" backups seems cost-prohibitive. I am curious, though -- for services like Dropbox or Google Drive, how many replicas are there of your files? I know there must…
Just using AWS S3 as an example, you can copy a bucket to another bucket (preferably in a different region) with a policy of retaining all versions- preventing the problem of a delete whether malicious or accidental, from being reflected in the backup copy. There is also the option of using Glacier for lower cost long term storage. If you are asking does regular S3 provide a true backup capability out of the box, it…