121–130 of 319 posts
Wouldn't batching lead to an increase in transaction latency, even if we achieve higher throughput?
The CAP theorem has been truly disastrous for databases. The CAP theorem simply says that if you have a database on 2 servers and the connection between those serves goes down, then queries against one server don't see new updates from the other server, so you either have to give up consistency (serve stale data) or give up availability (one of the servers refuses to process further requests). That's all that CAP is,…
For a longer explanation of the same idea which includes a concrete example of how you can get "high availability" in a CP system, see: https://apple.github.io/foundationdb/cap-theorem.html
Earlier quoted context omitted.
"Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition." Many of the distributed clusters I've maintained had crap infrastructure and no change control, and parts of the clusters were constantly going down from lack of storage, CPU and RAM, or bad changes. The only reason the applications kept working were either (1) the not-broken vnodes continue…
It seems you have other problems (crap infrastructure and no change control) to deal with before the issues in this article become your biggest concern, but are not the cases you list themselves partition problems?
The point is, resources cause problems, and the network is just one of many resources needed by the system. Other resources actually have more constraints on them than the network does. If a resource is constrained, it will impact availability in a highly-consistent model.
The author states that simply adding network redundancy would reduce partitions, and infrastructure problems are proof that this is very short-sighted. "You have bigger problems" - no kidding! Hence the weak-consistency model!
The point of AP systems is not 100% availability, but rather higher availability.
By the same reasoning, one should never do CP, because it is also not possible to have 100% consistency. Disk/memory/network corruption, even with ECC can overwhelm the ability to maintain consistency.
Earlier quoted context omitted.
Disagreement about what though? Does Spanner's solution have an objective problem? Do you or others in your community have specific reasons to believe that it cannot deliver on its promises?
Spanner's approach requires help from hardware and several full time employees maintaining and ensuring the uncertainty guarantees. This increases the cost of the maintaining the system, which for Cloud Spanner is partially passed on to the end users. If you can build a system that doesn't require time synchronization, yet doesn't have any significant drawbacks relative to what Spanner provides, you'd be better off u…
But you describe exactly the drawbacks of giving up time synchronization:
> The main downside of the first category is scalability. A server can process a fixed number of messages per second. If every transaction in the system participates in the same consensus protocol, the same set of servers vote on every transaction. Since voting requires communication, the number of votes per second is limited by the number of messages each server can handle. This limits the total amount of transactions per second that the system can handle.
How is "worse scalability" not a significant drawback?
This just sounds like an engineering tradeoff. I don't think engineering tradeoffs are the same as controversy. I get that your group's DB takes a different approach. But "blaming" Spanner for making a different trade-off doesn't come off well (I approached the article with an open mind).
> Systems that guarantee consistency only experience a necessary reduction in availability in the event of a network partition. As networks become more redundant, partitions become an increasingly rare event. And even if there is a partition, it is still possible for the majority partition to be available In my experience, yes network partitions are incredibly rare. However 99% of my distributed ststem partitions hav…
I may be a bit of an old fart, but this is the exact reasoning behind my decision to never go with "distributed X" if there's a "single-machine X" where you can just vertically scale. If you can afford 3-5 machines/VMs for a cluster you can almost certainly afford a single machine/VM with 2-4x the resources/CPU and chances are that it'll perform just as well (or better) because it doesn't have network latency to cont…
I also like the blogs point that availability is not ever 100%, but I think the added cost of availability levels when going from an eventual consistency system to a linearizable one is underestimated because performance is going to be a significant availability factor, not only failuremodes as discussed.
Earlier quoted context omitted.
24x7 coverage with a short time-to-repair costs at a minimum several million dollars per year. Interesting - what are the constituents of that cost? What sort of challenges do you face? Do you use PTP grandmaster clocks, or something else? How many sites, and how many clocks per site? Are the support issues mostly hardware failures, configuration problems, or something else? Is 24/7 support needed because the equipme…
I think it was meant that Google has such high costs. I read somewhere that Google operates two atomic clocks in each of its data centers, but I can't find a source for it right now, just this: https://www.wired.com/2012/11/google-spanner-time/
Earlier quoted context omitted.
People are going to notice a 300m deviation due to landmarks and their eyes.
Rarely, if you are navigating at sea or in the air or in the woods... and even on the road, it is not uncommon for my GPS device to be clearly off without justifying the conclusion that there is a fault in a satellite.
* Air and sea port operators and navigators
* Military personal running supply lines
* Military personal on foot in operations and training
* Space-X
* NASA
* River boats
* Fresh water fishermen
* Etc
Out of all the possible users who would notice a 300m deviation just based on visual reconciliation, I personally would not say it would be so rare that the USAF would not find out very quickly. Of course, this is ignoring the equipment that would likely detect the issue way before somebody in the Army started phoning the USAF.