151–160 of 240 posts
“After a successful write of a new object or an overwrite of an existing object, any subsequent read request immediately receives the latest version of the object.” - being able to issue a read after a write has been the case in any system ever. AWS solving a fabricated issue, and the crowd cheering it as progress. Have to love herd mentality.
Earlier quoted context omitted.
Our image processing worker queues/servers write an S3 object and dispatch a follow up job. Currently we have to delay the next job (we use 5 seconds) otherwise then next job may start processing before the S3 object is available (it 404's if the next job is run straight away).
Out of curiosity, did you have retries on the GETs?
Where is the mistake.
The more I learn about S3 the more I feel the same awe I feel when looking at the Pyramids or Ankor Wat. > You can send 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix in an S3 bucket.
Ask HN: I have a question regarding strong consistency. The question is related to all geo distributed strong consistent storages, but let me pick Google Spanner as example. Spanner has a maximum upper bound of 7 milliseconds clock offset between nodes, thx to TrueTime. To achieve external consistency, Spanner simply waits 7 ms before committing a transaction. After that amount of time the window of uncertainty is ov…
The simple answer is that there are round trips required between datacenters when you have a quorum that spans data centers. Additionally, one replica of a split is the leader, and the leader holds write locks. So already you have to talk to the leader replica, even if it's out of the DC you're in. Getting the write lock overlaps with the transaction commit though. So for your example if we say the leader is in Canada and the client is in Australia, and we're doing a write to row 'Foo' without first reading (a so called blind write):
Client (Australia) -> Leader (Canada): Take a write write lock on 'Foo' and try to commit transaction
Leader -> other replicas: Start commit, timestamp 123
Other replicas -> Leader: Ready for commit
Leader waits for majority ready for commit and for timestamp 123 to be before the current truetime interval
Leader -> Other replicas: Commit transaction, and in parallel replies to client.
Of course there are things you can do to mitigate this depending on your needs, but there's no free lunch. If you have a client in Australia and a client in Canada writing to the same data you're going to pay for round trips for transactions.
I thought we had worked out in 2010-2012 that the CAP theorem, consistency, availability and partition tolerance ( https://en.wikipedia.org/wiki/CAP_theorem#History ) made this impossible? Where is the mistake.
“After a successful write of a new object or an overwrite of an existing object, any subsequent read request immediately receives the latest version of the object.” - being able to issue a read after a write has been the case in any system ever. AWS solving a fabricated issue, and the crowd cheering it as progress. Have to love herd mentality.
Ask HN: I have a question regarding strong consistency. The question is related to all geo distributed strong consistent storages, but let me pick Google Spanner as example. Spanner has a maximum upper bound of 7 milliseconds clock offset between nodes, thx to TrueTime. To achieve external consistency, Spanner simply waits 7 ms before committing a transaction. After that amount of time the window of uncertainty is ov…
Writes are implemented with pessimistic locking. A write in a Canada datacenter may have to acquire a lock in Australia as well. See page 12 of the paper for the F1 numbers (mean of 103.0ms with standard deviation of 50ms for multi-site commits).
The more I learn about S3 the more I feel the same awe I feel when looking at the Pyramids or Ankor Wat. > You can send 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix in an S3 bucket.
11 nines of reliability. I don't know anything else promising that (unclear if they actually reach it)