Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Cloud Spanner, a Global Database Service

cloudplatform.googleblog.com

Post reply on HN
Print view View original

271–280 of 456 posts

Re: Introducing Cloud Spanner, a Global Database Service

#271

Post by edaemon »

macintux wrote: post #231
CaveTech wrote: post #220

Earlier quoted context omitted.

AWS uses heartbeats for detecting liveliness. If x heartbeats fail the failover procedure is started. Generally 10s - 5minutes. In practice (for me) the failover has been less than 15s.

My concern was more around split brain. If you fail over while the write master is simply unreachable, pain results.

Aurora's read replicas share the underlying storage that the primary uses, so AWS claims that there's no data loss on failover. They also claim -- and I've never heard anyone say they were wrong -- that Aurora failovers take less than a minute. So the pain should be limited to under a minute of lost writes, which most applications can handle (with an error). It can still be painful depending on the application.

See here for more info: https://aws.amazon.com/rds/aurora/faqs/#high-availability-an...

Re: Introducing Cloud Spanner, a Global Database Service

#272

Post by hackuser »

dzhiurgis wrote: post #249
progman wrote: post #188

Earlier quoted context omitted.

> The centralization of computation is likely not a good thing in the long run. I agree. It only makes sense if you need special data for statistics, AI training, etc. In all other cases the classic way of programming on pc and notebook is smarter. If you do everything in the cloud, what if you lose Internet connection? I had that experience several times over the last years.

My internet connection is more stable than my computer.

I'm not sure that's widely true. Consider:

* Most Internet usage is via smartphone

* Computers are much more stable than they used to be

* Much of the world lives in places with less stable connections

* The most expensive spec in an Internet connection is availability. You can get a low-end 15 Mbps connection with no availability guarantee for $40/month; a T1 is one-tenth the speed and costs 10 times as much (all numbers are rough estimates).

Re: Introducing Cloud Spanner, a Global Database Service

#273

Post by sixdimensional »

karlding wrote: post #47

I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…

I am also interested in how it compares to NewSQL databases like NuoDB. NuoDB has been positioning itself as a very similar type solution (no compromise relational distributed database) to Cloud Spanner for a while (minus the cloud hardware provided for you).

Re: Introducing Cloud Spanner, a Global Database Service

#274

Post by hackuser »

scarmig wrote: post #146
hueving wrote: post #131

Earlier quoted context omitted.

>well that would be a lot of choice for the developers! A sad choice though. The centralization of computation is likely not a good thing in the long run.

The absolute level of computation available isn't changing at the consumer level. What's happening for the next decade is the destruction of businesses hosting their own IT infrastructure and moving it to a couple of core centers. So, the computational "Gini index" is increasing, but no one is being thrown into computational poverty.

> The absolute level of computation available isn't changing at the consumer level.

Maybe not, but consumers increasingly use centralized computation resources. I would guess that by now most applications used by consumers run in their web browser, such as Facebook.

Re: Introducing Cloud Spanner, a Global Database Service

#275

Post by xapata »

discodave wrote: post #266
xapata wrote: post #225

Earlier quoted context omitted.

> achieve the expectations It has for Google internally. No reason why they can't share the service externally.

Actually, there are lots of reasons why it's hard to share a service externally. Making a service public means you have to deal with lots of new problems like billing, abuse, dealing with lots of small users instead of a few large ones, stronger backwards compatibility requirements and so-forth.

Yes, but those are orthogonal to the scaling issues that I presume user5994461 was thinking of.

Re: Introducing Cloud Spanner, a Global Database Service

#276

Post by hackuser »

euyyn wrote: post #259
ocdtrekkie wrote: post #158

Earlier quoted context omitted.

The parent comment doesn't seem to specify "consumer level" and the loss of businesses having their own infrastructure is equally troubling. Everyone is putting a lot of eggs in a very small number of baskets.

That's like being sad about the emergence of banks, because everybody's money is being kept in a small number of vaults instead of under each one's mattress.

A good point, but there is an up and downside to everything. The centralization of IT does impact civil liberties and possibly innovation - unlike FOSS and other local systems, aspiring hackers can't tinker with Facebook code and see how it works.

Re: Introducing Cloud Spanner, a Global Database Service

#277

Post by edaemon »

xapata wrote: post #210
macintux wrote: post #196

Earlier quoted context omitted.

How does Aurora know that the primary is dead? Automatic failover is problematic in a distributed system.

Yeah, the latency on that failover isn't specified.

Do you mean the amount of time it takes to initiate a failover or the amount of time for a failover to complete?

For the former, I don't think they specify beyond "automatic".

For the latter, "service is typically restored in less than 120 seconds, and often less than 60 seconds": http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora...

Re: Introducing Cloud Spanner, a Global Database Service

#278

Post by floatboth »

nodesocket wrote: post #255
floatboth wrote: post #252

Earlier quoted context omitted.

That offering is just normal managed MySQL, the new thing is a custom built database for huge scalability, much more expensive of course.

Right that's what I was getting at, a typical web app does not and probably should not start with Cloud Spanner (ignoring the fact it costs $0.90 per hour per node). Cloud Spanner seems like it is attacking the big data market correct?

Something like that. But not necessarily the Big Data that's being analyzed, that's usually done with Hadoop/Spark/whatever is the big thing now. It seems to be aimed at, like, huge apps that actually need horizontal scalability.

Re: Introducing Cloud Spanner, a Global Database Service

#279

Post by cestith »

cjmcqueen wrote: post #171
ocdtrekkie wrote: post #158

Earlier quoted context omitted.

The parent comment doesn't seem to specify "consumer level" and the loss of businesses having their own infrastructure is equally troubling. Everyone is putting a lot of eggs in a very small number of baskets.

I would disagree about the character of the situation. This isn't about people putting eggs in a few baskets, it's that it's more efficient to have centralized chicken coops instead of every family in the world owning their own chickens. Now, you could play with that analogy further and see some issues as well, but I don't think the issue here is centralized failure; all these data centers/"clouds" are at least good.…

Let's look at your metaphor. It's more efficient for the raising of a large overall number of chickens. It's less efficient when I need fast access to a single egg.

Hence we get caching. There's the farms, then the inbound warehouses, then the distribution centers, then the grocer, then our refrigerators by the dozen or dozen and a half. When your local cache is empty of eggs, though, it requires a trip back out to the grocer to get an egg even if you need nothing else that trip. Then you generally have to buy at least half a dozen if not a dozen or more eggs just to get the one you wanted.

If I have my own couple of hens, I can go out into the yard and get an egg. If that's the whole of my fetch list, it's much more efficient for this single egg to have the hens laying right out back.

This whole few baskets metaphor breaks down from another point of view, though, when we consider that by the very nature of using a globally distributed hosted service we're actually eliminating a single basket problem. Yes, there's not much choice among just Google, Amazon, and Microsoft. (That they are the only options is a bit of a strawman, but lets grant this one legs.) However, putting just your own employees in charge of all your infrastructure in just your own datacenter(s) in just PostgreSQL or just MySQL is another single-basket problem. Spreading it out so that someone else gets to manage the hardware and the service and replicating your data widely within that service is from that point of view more baskets. You get more datacenter baskets, more employee baskets, and more software baskets. Using standard SQL means you can move among compliant software later, too, so you're not as tied to those baskets.

Now, back to your coop analogy. What's stopping me from having my application talk to Cloud Spanner and a local database proxy (or a work queue that sits between the app and the DB or whichever) so I can use Google's reliability for transactions and my local cached replicant for query speed when I'm querying older data? Why can't I keep a few eggs around?

Also, why would I be scared of Google or Amazon "having my data"? Why would I put sensitive data into my own database in plaintext and then replicate it among multiple datacenters that way?

Re: Introducing Cloud Spanner, a Global Database Service

#280

Post by dcosson »

jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

I would phrase it as:

AWS prioritizes building blocks that support very high throughput and avoid leaky abstractions at all costs, and they're happy to push forward as long as these criteria are met. IMO they really succeed at this goal. Minus specific bugs that they're generally good about acknowledging, their services reliably do what they say they're going to. And they definitely solve a lot of problems for you, even if sometimes you're still required to get further into the weeds than you might want.

I'll buy that Google Cloud is better at questioning underlying assumptions and sometimes succeeds in releasing higher-level abstractions than AWS without any leakiness (a great example of this now being Spanner vs Aurora). It also feels to me that with releases like this Google is leveraging the full value of their own experiences running their services, and seems to be more advanced than amazon in some areas so this has a lot of value, whereas AWS seems to build a broader range of products with a specific customer in mind which is not necessarily themselves (e.g. all of their move your on-prem stuff to the cloud helpers).

If you consider Spanner vs Dynamo, it definitely matches up as Google wrapping the old way and Amazon forcing a new way (though to be fair, Dynamo was released 5 years earlier). But on the other hand considering Spanner vs Aurora, Amazon is the one embracing the old way with full MySQL & Postgres compatibility whereas Spanner sounds like a pretty dramatically different subset of SQL in not supporting insert and update statements. It's a very reasonable compromise for basically getting to ignore the CAP theorem, but it is a a significant difference that every developer will have to learn.

Post reply on HN