Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Cloud Spanner, a Global Database Service

cloudplatform.googleblog.com

Post reply on HN
Print view View original

201–210 of 456 posts

Re: Introducing Cloud Spanner, a Global Database Service

#201

Post by jedberg »

user5994461 wrote: post #197
jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

Amazon: Create usual services and sell them. Google: Make unique products that push the boundaries of what was previously thought possible. Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain. Google: Had to make all their core technologies efficient, performant, s…

Yeah that's all pretty accurate too. :)

Re: Introducing Cloud Spanner, a Global Database Service

#202

Post by krenoten »

sudhirj wrote: post #122
robryk wrote: post #98

Earlier quoted context omitted.

A CA system is not a system that doesn't have partitions, it's a system that works under the condition that there are no partitions (ie. it is a non-partition-resistant system).

A memcache cluster might qualify. If there's a partition, just forget about the missing nodes. It's just a cache anyway.

The system you describe would typically be referred to as AP, because it maintains availability in the presence of partitions.

Re: Introducing Cloud Spanner, a Global Database Service

#203

Post by zzzcpan »

DivineTraube wrote: post #190

"What if you could have a fully managed database service that's consistent, scales horizontally across data centers and speaks SQL?" Looks like Google forgot to mention one central requirement: latency. This is a hosted version of Spanner and F1. Since both systems are published, we know a lot about their trade-offs: Spanner (see OSDI'12 and TODS'13 papers) evolved from the observation that Megastore guarantees - tho…

> they do come at a cost Google does not tell in its marketing: high latency

Poor latency is fundamental for a CP system, it's kind of a given. Would be nice if they explained it to users though, how achieving consensus is necessary for global consistency and how it is impossible to do that without waiting.

Re: Introducing Cloud Spanner, a Global Database Service

#204

Post by CaptSpify »

cjmcqueen wrote: post #171
ocdtrekkie wrote: post #158

Earlier quoted context omitted.

The parent comment doesn't seem to specify "consumer level" and the loss of businesses having their own infrastructure is equally troubling. Everyone is putting a lot of eggs in a very small number of baskets.

I would disagree about the character of the situation. This isn't about people putting eggs in a few baskets, it's that it's more efficient to have centralized chicken coops instead of every family in the world owning their own chickens. Now, you could play with that analogy further and see some issues as well, but I don't think the issue here is centralized failure; all these data centers/"clouds" are at least good.…

> it's that it's more efficient to have centralized chicken coops instead of every family in the world owning their own chickens.

Only if the owner of the chicken-coop has everyone else's best interests in mind. Protip: They don't.

The Cloud isn't about efficiency, it's about data control. Getting people's systems and data into Google/AWS/etc helps with data mining, vendor lock-in, etc. Often times that can be efficient, but also it often isn't.

Re: Introducing Cloud Spanner, a Global Database Service

#205

Post by xapata »

jhugg wrote: post #168
xapata wrote: post #74

Earlier quoted context omitted.

No, he's saying it's effectively CAP because the A downtime is so small. It's one thing to do that for a key-value store. Entirely another to support joins on a globally distributed database. This ain't just one availability zone. Spanner is amazing. It took them a few years to make it a service, but when they announced its use internally a few years ago, it seemed like the nail in the coffin for in-house database ho…

I understand what he's saying. It's marketing. There's nothing wrong with saying it's CP, but since we control everything there's extremely rare P. Then he can show availability numbers (which he kinda does). Saying it's "effectively CA" defeats the point of the CAP theorem, which says you have to make tradeoffs. See: https://codahale.com/you-cant-sacrifice-partition-tolerance/

> It's marketing.

No, it's engineering. It's the recognition that if periods of unavailability are too small and too rare to be noticed, then the system behavior is indistinguishable from an "available" system in the sense of the CAP theorem.

It's like the "Retina" display you're probably reading from. There are pixels, you just can't see them.

Re: Introducing Cloud Spanner, a Global Database Service

#206

Post by user5994461 »

elvinyung wrote: post #85
xapata wrote: post #79

Amazon likes to respond to Google with it's own price drops and product launches. It's telling that their announcements are orthogonal instead of direct competition with Spanner. When Google announced Spanner back in 2012, I'm sure Amazon and Microsoft started teams to reproduce their own versions. Spanner is not just software. The private network reduces partitions. GPS and atomic clocks for every machine help synch…

Amazon already has Aurora: https://aws.amazon.com/rds/aurora/details/ You're right that there's literally nothing else out there that has tight synchronization using atomic clocks, though.

Aurora is a toy compared to Spanner.

Single region, limited backups and replication topologies, limited performances.

There is yet to see if Spanner can achieve the expectations, if it does, it's a game changer.

Re: Introducing Cloud Spanner, a Global Database Service

#207

Post by krenoten »

rdtsc wrote: post #109
tupshin wrote: post #78

Earlier quoted context omitted.

Trivially, by never allowing either half of a partition to make progress while the partition is in place. Since the CAP theorem, by itself, doesn't put a cap(oof) on latency, it is valid to consider a system CA, if it if always available to listen to requests while partitioned, but never able to fulfill them. This, of course, is effectively useless in practice, and is dependent on an infinite buffer of pending operat…

> Trivially, by never allowing either half of a partition to make progress while the partition is in place. Doesn't availability mean getting a response on success or failure. If during a partition there is no response on success of failure how is the system available? It seems re-writing a term like "x will happen" to "x will happen after an infinite timeout" should not be valid

Aphyr covers the impossibility of practical CA systems here: https://aphyr.com/posts/325-comments-on-you-do-it-too

Sometimes it's helpful to consider a distributed system through the lens of "harvest vs yield" where harvest is the proportion of information in a system reflected by a response, and yield which is the probability of receiving a response at all.

https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.24...

Re: Introducing Cloud Spanner, a Global Database Service

#208

Post by sandGorgon »

craigkerstiens wrote: post #173
sandGorgon wrote: post #169

Earlier quoted context omitted.

It would be very interesting to have your product at the 200-300$ pricepoint. Currently, the lowest tier starts at almost 2000$ per month for the high availability version. I'm not trying to compare on a per-mb level, but it would be nice for smaller scale workloads.

Helpful feedback, we do have a development plan for $99, but it's not really intended for production workloads. If you only have 10 GB of data we'd heavily recommend going with something like RDS or Heroku Postgres. At that amount of data single node Postgres works great.

RDS is not single node - its multi-AZ replicated. And that's what we are paying 300$ instead of 99$ for.

Imagine.. RDS is literally the ONLY place where you can buy a 10 GB data multi-AZ replicated, snapshotted and managed postgresql.

Its pretty much a monopoly, now that Google seems to have officially closed the book on ever supporting postgresql.

Re: Introducing Cloud Spanner, a Global Database Service

#209

Post by 010a »

theDoug wrote: post #41
runeks wrote: post #27

I wonder why they charge a minimum of $0.90 per node-hour when they offer VMs for as little as $0.008/hr. This is hugely useful even for single-person startups, so why charge a minimum of ~$8,000 per year?

Hugely useful but also hugely different from an engineering/coverage perspective, perhaps. Companies with more data than can fit in a single-instance RDBMS system (like >3TB of hot data, more throughput than a single node can handle) but still seeking transactional consistency are a clear use case. Single-person startups could definitely benefit, but it's a less-likely scenario that they would require the level of co…

But the most successful products tend to be ones that can scale from zero to global. DynamoDB is a great example.

Re: Introducing Cloud Spanner, a Global Database Service

#210

Post by xapata »

macintux wrote: post #196
edaemon wrote: post #141

Earlier quoted context omitted.

Yes, Aurora has a single write master, though it does have automatic write failover -- i.e. if the Aurora primary dies, one of your read replicas is promoted to the primary and reads/writes are directed to the new instance. That does constrain your primary's capabilities to the largest instance size (currently a db.r3.8xlarge). I don't have a good idea what the upper limit is for an Aurora database setup.

How does Aurora know that the primary is dead? Automatic failover is problematic in a distributed system.

Yeah, the latency on that failover isn't specified.
Post reply on HN