Earlier quoted context omitted.
That's analytics data, not file storage. If they have 34PB of analytics, they certainly have far more than that of blob storage.
I thought they were almost entirely on prem for the actual storage
S3 Strong Consistency
161–170 of 240 posts
Re: S3 Strong Consistency
#162This is really cool. We actually got burned on weak S3 consistency a few weeks ago when generating public download links for a customer. Took us a few hours of troubleshooting to realize they had downloaded a cached/older version of the software we had uploaded to the same URL just a few minutes prior. Resolution was to use unique paths per version to guarantee we were talking to the right files each time.
One potentially related item I was thinking about - How does HN feel about the idea of a system that has eventual durability guarantees which can be inspected by the caller? I.e. Call CloudService.WriteObject(). It writes the object to the local datacenter and asynchronously sends it to the remote(s). It returns some transaction id. You can then pass this id to CloudService.TransactionStatus() to determine if a Durable flag is set true. Or, have a convenience method like CloudService.AwaitDurabilityAsync(txnid). In the most disastrous of circumstances (asteroid hits datacenter 1ms after write returns to caller), you would get an exception on this await call after a locally-configured timeout in which case you can assume you should probably retry the write operation.
I was thinking this might be a way to give the application a way to decide how to deal with the concept of latency WRT cross-site replication. You could not await the durability for 4 nines or wait the additional 0-150 ms to see more nines. I wonder how this risk profile sits with people on the business side. I feel like having a window of reduced durability per object that is only ~150ms wide and up-front can be safely ignored. Especially considering the hypothetical ways in which you could almost instantaneously interrupt the subsequent activities/processing with the feedback mechanism proposed above.
Re: S3 Strong Consistency
#163Earlier quoted context omitted.
11 nines of reliability. I don't know anything else promising that (unclear if they actually reach it)
11 nines of _durability_. Uptime SLA for s3 is 99.9% AFAICR
And those 11 nines of durability are what it is designed for, they don't offer any sort of SLA for that.
Re: S3 Strong Consistency
#164If someone is wondering why this is huge. It's because of the problems you face, when you don't have strong consistency. See this picture to get an understanding of what kind of problems you might run into with eventual consistency: https://i.ibb.co/DtxrRH3/eventual-consistency.png
You just made my day :)
Re: S3 Strong Consistency
#165If someone is wondering why this is huge. It's because of the problems you face, when you don't have strong consistency. See this picture to get an understanding of what kind of problems you might run into with eventual consistency: https://i.ibb.co/DtxrRH3/eventual-consistency.png
I shall print this..
Re: S3 Strong Consistency
#166Earlier quoted context omitted.
I am curious about how much Dropbox pays for data ingress/egress, they migrated storage to their own on premise data center, then now moving data back to S3 for the data lake.
That's analytics data, not file storage. If they have 34PB of analytics, they certainly have far more than that of blob storage.
[deleted]
Re: S3 Strong Consistency
#167Earlier quoted context omitted.
I am curious about how much Dropbox pays for data ingress/egress, they migrated storage to their own on premise data center, then now moving data back to S3 for the data lake.
That's analytics data, not file storage. If they have 34PB of analytics, they certainly have far more than that of blob storage.
Definitely much more of user data storage. More about that https://dropbox.tech/infrastructure/inside-the-magic-pocket.
Happy to take other questions.
Re: S3 Strong Consistency
#168Earlier quoted context omitted.
Yup, I think it would.
I have started working on this actually (the vfs being developed in Rust). The design was complex but now this simplifies a lot. The vfs will also have snapshotting capabilities i.e. you can have multiple versions of sqlite db.
How are you handling latency (up to 800ms on S3) or the fact that you have to write the whole file each time?
Re: S3 Strong Consistency
#169Earlier quoted context omitted.
I have started working on this actually (the vfs being developed in Rust). The design was complex but now this simplifies a lot. The vfs will also have snapshotting capabilities i.e. you can have multiple versions of sqlite db.
How are you handling latency (up to 800ms on S3) or the fact that you have to write the whole file each time?
> you have to write the whole file each time
Will not write the whole file. Instead each page of the database (default = 4KB) will be separate put.
The use case is: one-writer multiple readers.
The readers will read an old snapshot of the database (from s3) and writer will write to the active one. This way a lot of readers can open the DB in read-only mode.
As for writer, each page will be a separate put. The writes will be appended to a log(s) locally and the page will be synced in the background. This helps to exploit lot of parallel PUTs to S3. Once the sync happens the disk space for that page will be reclaimed.
For the writer:
v = open_new_version()
//inserts + updates
//commit
close_version(v)
For readers:
open_version_for_read(v)
//select
Re: S3 Strong Consistency
#170Earlier quoted context omitted.
I have started working on this actually (the vfs being developed in Rust). The design was complex but now this simplifies a lot. The vfs will also have snapshotting capabilities i.e. you can have multiple versions of sqlite db.
How are you handling latency (up to 800ms on S3) or the fact that you have to write the whole file each time?
Obviously, a sane implementation would divide db file into blocks and write them into separate objects. Not as small as traditional FS blocks, 1-4 Mb, perhaps.