Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Cloud Spanner, a Global Database Service

cloudplatform.googleblog.com

Post reply on HN
Print view View original

141–150 of 456 posts

Re: Introducing Cloud Spanner, a Global Database Service

#141

Post by edaemon »

abalone wrote: post #124
tedd4u wrote: post #114

Earlier quoted context omitted.

Aurora is very cool but won't help you much after you vertically scale your master and still need more write capacity. With Cloud Spanner you get horizontal write scalability out of the box. Critical difference.

So if I'm understanding you, with Aurora all writes go to one master and you're constrained by the biggest instance AWS offers. Is that right? Do you have a sense of what that limit is? There's a pretty big price difference between Spanner and Aurora at the entry level so it's useful to explore this.

Yes, Aurora has a single write master, though it does have automatic write failover -- i.e. if the Aurora primary dies, one of your read replicas is promoted to the primary and reads/writes are directed to the new instance. That does constrain your primary's capabilities to the largest instance size (currently a db.r3.8xlarge).

I don't have a good idea what the upper limit is for an Aurora database setup.

Re: Introducing Cloud Spanner, a Global Database Service

#142

Post by ex3ndr »

tedd4u wrote: post #33

The team here at Quizlet did a lot of performance testing on Spanner with one of our MySQL workloads to see if it's an option for us. Here are the test results: https://quizlet.com/blog/quizlet-cloud-spanner

> So a query that accesses 10 rows in disparate parts of the primary key space will take longer than one where the keys reside on the same splits. This is expected with a distributed system.

No, why? Query can be executed in parallel.

BTW, isn't 20k/sec is a very very small performance for 30 node installation. Cassandra can handle 50k+ (both writes and read) on a single node. When in most queries you are trying to collect data from many nodes it will scale almost linearly.

Re: Introducing Cloud Spanner, a Global Database Service

#143

Post by zzzcpan »

sudhirj wrote: post #140
sudhirj wrote: post #135

For those trying to compare this with AWS Aurora, Aurora is more a regular database (MySQL / Postgres) engine with a custom data storage plugin that's AWS/ELB/SSD/EFS-aware. Because of this the database engine can make AWS specific decisions and optimizations that greatly boost performance. It supports master-master replication in the same region, master-slave across regions. Global Spanner looks like a different bea…

From the technical blog post > Does this mean that Spanner is a CA system as defined by CAP? The short answer is “no” technically, but “yes” in effect and its users can and do assume CA. The purist answer is “no” because partitions can happen and in fact have happened at Google, and during some partitions, Spanner chooses C and forfeits A. It is technically a CP system. However, no system provides 100% availability,…

Except that A in CAP has nothing to do with overall system's availability over time and using it as such is just confusing.

Re: Introducing Cloud Spanner, a Global Database Service

#144

Post by tupshin »

rdtsc wrote: post #109
tupshin wrote: post #78

Earlier quoted context omitted.

Trivially, by never allowing either half of a partition to make progress while the partition is in place. Since the CAP theorem, by itself, doesn't put a cap(oof) on latency, it is valid to consider a system CA, if it if always available to listen to requests while partitioned, but never able to fulfill them. This, of course, is effectively useless in practice, and is dependent on an infinite buffer of pending operat…

> Trivially, by never allowing either half of a partition to make progress while the partition is in place. Doesn't availability mean getting a response on success or failure. If during a partition there is no response on success of failure how is the system available? It seems re-writing a term like "x will happen" to "x will happen after an infinite timeout" should not be valid

It does, but within what bounds? The CAP theorem doesn't specify. One could assume that it means before the partition is restored, but that is only one possible valid interpretation. The PACELC theorem, which is by no means the last word on the story, clarifies this well:

https://en.wikipedia.org/wiki/PACELC_theorem

"PACELC builds on the CAP theorem. Both theorems describe how distributed databases have limitations and tradeoffs regarding consistency, availability, and partition tolerance. PACELC however goes further and states that a trade-off also exists, this time between latency and consistency, even in absence of partitions, thus providing a more complete portrayal of the potential consistency tradeoffs for distributed systems."

And I would take that argument one step further and say that latency and partitioning are effectively identical, and from the point of view of any given operation, it is impossible to say whether the system is in partitioned state until max lateny (timeout) has elapsed, because failure to make progress within timeout is the only meaningful definition of partion-induced unavailability.

Re: Introducing Cloud Spanner, a Global Database Service

#145

Post by takeda »

jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

I'm not sure if what you said applies, they have severe restrictions and spanner offers subset of MySQL functionality which is already bare compared to other databases. Changes can be done by primary key only, so it almost feels like a KV store that can do joins...

I don't think it's easy to port existing applications to use it and in the end you will still need to accommodate shortcomings in your application.

Re: Introducing Cloud Spanner, a Global Database Service

#146

Post by scarmig »

hueving wrote: post #131
ChuckMcM wrote: post #89

Congratulations to the Spanner team for becoming part of the Google public cloud! And for those wondering, this is why Oracle wants billions of dollars from Google for "Java Copyright Infringement" because the only growth market for Oracle right now is their hosted database service, and whoops Google has a better one now. It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If…

>well that would be a lot of choice for the developers! A sad choice though. The centralization of computation is likely not a good thing in the long run.

The absolute level of computation available isn't changing at the consumer level. What's happening for the next decade is the destruction of businesses hosting their own IT infrastructure and moving it to a couple of core centers.

So, the computational "Gini index" is increasing, but no one is being thrown into computational poverty.

Re: Introducing Cloud Spanner, a Global Database Service

#147

Post by BinaryIdiot »

Oh this looks really compelling! Though I'm guessing this is targeted to companies? I'd love to use this for some personal projects but the pricing seems really high. Am I reading it right that a single node being used at least a tiny bit every hour is about $670 a month?

Maybe I'm misunderstanding how the pricing works here. Any clarification would be highly welcomed :)

Re: Introducing Cloud Spanner, a Global Database Service

#148

Post by u320 »

jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

The "old way" was sacrificing functionality such as transactions and joins to get scalability (BigTable, DynamoDB).

Google tried that a decade ago and found it lacking, this is why Spanner exists in the first place.

Re: Introducing Cloud Spanner, a Global Database Service

#150

Post by jedberg »

inopinatus wrote: post #139
jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

People used to make similar comparisons between the Russian and American space programs.

Oh yeah? Which was which in that comparison? I'm not familiar with that.
Post reply on HN