Introducing Cloud Spanner, a Global Database Service • Hacker News Board

Re: Introducing Cloud Spanner, a Global Database Service

#261

Post by aaronblohowiak » Tue, Feb 14, 2017, 9:22 PM UTC

Godel_unicode wrote: post #212

aaronblohowiak wrote: post #181

Earlier quoted context omitted.

Microsoft has a history of sales and support that will allow them a certain longevity. They also have less "brand hate" than Oracle. I dont think MSSQL is going to be like Sybase any time soon, but I probably wouldn't focus on that stack starting now if you are into the startup or california scene. For many places in the USA, MS is the way to go. EDIT: Also, most DB users don't need global-scale databases.

Not sure if this is what you intended, but you are aware that SQL server was developed in partnership with Sybase until the mid-90s (when they were substantially the same product) right?

Neat history! I was not aware of that! I meant to imply that sybase's rdbms offering is no longer a big player and I would not want to bet the future of my career on being an expert in it.

Re: Introducing Cloud Spanner, a Global Database Service

#262

Post by lallysingh » Tue, Feb 14, 2017, 9:23 PM UTC

thesandlord wrote: post #182

sudhirj wrote: post #164

Earlier quoted context omitted.

But it makes sense in this case. The system guarantees CP. But as customer it looks like you're getting CA as well, because A is so high. If you drink the kool-aid, you get C & A & P. The kool-aid isn't too bad, though if they can measurably guarantee A > 99.999999%, I'm happy to round off to 100% and call it CAP.

The availability is "only" 99.999%, which IMO is still really high! (I work for Google Cloud)

https://uptime.is/99.999 puts that at:

Daily: 0.9s

Weekly: 6.0s

Monthly: 26.3s

Yearly: 5m 15.6s

Re: Introducing Cloud Spanner, a Global Database Service

#263

Post by deesix » Tue, Feb 14, 2017, 9:23 PM UTC

wcdolphin wrote: post #226

Is JSON data type support in the works? Seems to be a very commonly requested feature these days.

Yes, it's something we have on the roadmap. We will adjust priority based on the level of demand so thanks for your vote ;-)

(disclaimer: I work on Cloud Spanner)

Re: Introducing Cloud Spanner, a Global Database Service

#264

Post by jasonwatkinspdx » Tue, Feb 14, 2017, 9:23 PM UTC

krenoten wrote: post #218

suprgeek wrote: post #87

Really a CP system but with the Availability being five 9s or better (less than one failure in 10^6) How: 1)Hardware - Gobs and Gobs of Hardware and SRE experience "Spanner is not running over the public Internet — in fact, every Spanner packet flows only over Google-controlled routers and links (excluding any edge links to remote clients). Furthermore, each data center typically has at least three independent fibers…

MTBF of 2PC-strapped-to-quorum is no different from MTBF of a 2PC-strapped-to-spof replicas. MTTR is bounded by reelection latency, rather than replica recovery, although you still may eat a write amplification cost for rereplication. write amplification is 3-5x of non-quorum-backed 2PC system, depending on replication ensemble size. google further multiplies write amplification with geo-redundancy, so bump that WA b…

You're making some assumptions. For example if you use epaxos for the quorums there is no unavailability due to a leader failure and re-election. Even then, re-election is likely going to be a lot faster than any sort of fault tolerant 2PC coordinator recovery protocol. You're exaggerating the write amplification. The above offering is single region. Google says they do 5 way geo in their various papers/etc, and there's no way they're paying some cumulative 15 copy write amplification as you imply.

Re: Introducing Cloud Spanner, a Global Database Service

#265

Post by euyyn » Tue, Feb 14, 2017, 9:25 PM UTC

awj wrote: post #199

abalone wrote: post #124

Earlier quoted context omitted.

So if I'm understanding you, with Aurora all writes go to one master and you're constrained by the biggest instance AWS offers. Is that right? Do you have a sense of what that limit is? There's a pretty big price difference between Spanner and Aurora at the entry level so it's useful to explore this.

> Do you have a sense of what that limit is? Per their pricing page[1] it looks like the largest instance available is a "db.r3.8xlarge", which is a special naming of the "r3.8xlarge" instance type[2] which is 32 cpus and 244gb of memory. That's a hell of a lot of capacity to exhaust, especially if you're using read replicas to reduce it to only/mostly write workloads. Obviously it's possible to use more than this, b…

Wouldn't the write master be I/O-bound, rather than CPU- or memory-?

Re: Introducing Cloud Spanner, a Global Database Service

#266

Post by discodave » Tue, Feb 14, 2017, 9:27 PM UTC

xapata wrote: post #225

user5994461 wrote: post #206

Earlier quoted context omitted.

Aurora is a toy compared to Spanner. Single region, limited backups and replication topologies, limited performances. There is yet to see if Spanner can achieve the expectations, if it does, it's a game changer.

> achieve the expectations It has for Google internally. No reason why they can't share the service externally.

Actually, there are lots of reasons why it's hard to share a service externally. Making a service public means you have to deal with lots of new problems like billing, abuse, dealing with lots of small users instead of a few large ones, stronger backwards compatibility requirements and so-forth.

Re: Introducing Cloud Spanner, a Global Database Service

#267

Post by edblarney » Tue, Feb 14, 2017, 9:33 PM UTC

user5994461 wrote: post #197

jedberg wrote: post #110

This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable. Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it. I'm not saying either system…

Amazon: Create usual services and sell them. Google: Make unique products that push the boundaries of what was previously thought possible. Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain. Google: Had to make all their core technologies efficient, performant, s…

Not fair.

Amazon: IaaS

Google: PaaS

Amazon is philosophy is being 'close to the metal' to allow Enterprise customer to migrate 'regular apps' into a 'regular environment' in the cloud.

Most of Google's offerings are (at least were) novel, but proprietary ways of doing specific things.

Amazon is not a laggard: they have provided a number of interesting and useful 'helper' things to facilitate IaaS - as well as a number of 'pure cloud' type things.

Amazon is very, very customer focused. Their products come from customer demands.

Google often 'cool things they've done internally' and exposes them, hoping that they might have some use-case in the rest of the world.

Google and Amazon are equally interested in profit.

Re: Introducing Cloud Spanner, a Global Database Service

#268

Post by tazjin » Tue, Feb 14, 2017, 9:34 PM UTC

010a wrote: post #198

scarmig wrote: post #128

Earlier quoted context omitted.

Curious: is there any company in the world that could replicate its breadth, performance, and reliability in the next decade? Could any government? Has any government? My impression is that, infrastructure wise, Google is genuinely in a class of size one.

Its probably a class size of 2, with Amazon. Beyond those two though, no one else is close.

I honestly don't think Amazon is even close to Google.

How much more infrastructure do they have besides AWS? How much does Google have besides GCP?

Re: Introducing Cloud Spanner, a Global Database Service

#269

Post by richdougherty » Tue, Feb 14, 2017, 9:36 PM UTC

Some interesting stuff in https://cloud.google.com/spanner/docs/whitepapers/SpannerAnd... about the social aspects of high availability.

1. Defining high availability in terms of how a system is used: "In turn, the real litmus test is whether or not users (that want their own service to be highly available) write the code to handle outage exceptions: if they haven’t written that code, then they are assuming high availability. Based on a large number of internal users of Spanner, we know that they assume Spanner is highly available."

2. Ensuring that people don't become too dependent on high availability: "Starting in 2009, due to “excess” availability, Chubby’s Site Reliability Engineers (SREs) started forcing periodic outages to ensure we continue to understand dependencies and the impact of Chubby failures."

I think 2 is really interesting. Netflix has Chaos Monkey to help address this (https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey). There's also a book called Foolproof (https://www.theguardian.com/books/2015/oct/12/foolproof-greg...) which talks about how perceived safety can lead to bigger disasters in lots of different areas: finance, driving, natural disasters, etc.

Re: Introducing Cloud Spanner, a Global Database Service

#270

Post by deesix » Tue, Feb 14, 2017, 9:36 PM UTC

sandGorgon wrote: post #172

> If you have a MySQL or PostgreSQL system that's bursting at the seams Postgresql ? How does this work for people migrating from traditional SQL databases - typically people use ORM. How would this fit in with, say , Rails or SqlAlchemy ?

There is real work to migrate you application to use Cloud Spanner. Your schemas and queries will work with some tweaking but we don't have ORM support at beta.

We are going to produce some additional collateral on migrating from popular RDBMS, but the Quizlet post is the best reference right now for migrating from MySQL.

https://quizlet.com/blog/quizlet-cloud-spanner

We have released client libraries for Java, Go, Node and Python on Github, but we haven't used those clients to implement support for popular ORMs.

Basically, the open-source ecosystem will need to add support for these ORMs but we will contribute wherever we can to push these initiatives. If we get lots of demand for a specific ORM, we will look into doing something special for that.

Hope that helps.

(disclaimer: I work on Cloud Spanner)