Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Cloud Spanner, a Global Database Service

cloudplatform.googleblog.com

Post reply on HN
Print view View original

401–410 of 456 posts

Re: Introducing Cloud Spanner, a Global Database Service

#401

Post by inopinatus »

ec109685 wrote: post #377
krenoten wrote: post #218

Earlier quoted context omitted.

MTBF of 2PC-strapped-to-quorum is no different from MTBF of a 2PC-strapped-to-spof replicas. MTTR is bounded by reelection latency, rather than replica recovery, although you still may eat a write amplification cost for rereplication. write amplification is 3-5x of non-quorum-backed 2PC system, depending on replication ensemble size. google further multiplies write amplification with geo-redundancy, so bump that WA b…

9x write amplification isn't that high. If you are setup with a master slave configuration, that creates write amplification and for reliability, you probably have some sort of raid, that increase the number of places writes go to as well.

Not necessarily RAID, it might be an erasure coding.

Re: Introducing Cloud Spanner, a Global Database Service

#402

Post by lrem »

mistermann wrote: post #177
kornish wrote: post #129

Earlier quoted context omitted.

> It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If we get to the point where you have databases, compute, storage, and connectivity services from those three at equal scale, well that would be a lot of choice for the developers! There are also plenty of choices evolving for developers who aren't looking for hosted solutions (which can sometimes be a showstopper for ente…

As someone who works (in part) in the MS SQL field, is it irrational to be a bit worried about the effects some of these platform advances might have one one's career? For example, being a MSSQL performance tuning expert requires years of experience and probably pays very well, but just the other day I read an anecdotal story where someone switched a large BI database to use columnar indexes, allowing them to replace…

I would say: quite long enough, but the scale/importance of traditional deployments will be going down through that time. If you're looking around the market regularly, you should have ample time to notice that gigs are not what they used to be, so maybe it's time to change area.

Re: Introducing Cloud Spanner, a Global Database Service

#403

Post by lrem »

mistermann wrote: post #177
kornish wrote: post #129

Earlier quoted context omitted.

> It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If we get to the point where you have databases, compute, storage, and connectivity services from those three at equal scale, well that would be a lot of choice for the developers! There are also plenty of choices evolving for developers who aren't looking for hosted solutions (which can sometimes be a showstopper for ente…

As someone who works (in part) in the MS SQL field, is it irrational to be a bit worried about the effects some of these platform advances might have one one's career? For example, being a MSSQL performance tuning expert requires years of experience and probably pays very well, but just the other day I read an anecdotal story where someone switched a large BI database to use columnar indexes, allowing them to replace…

I would say: quite long enough, but the scale/importance of traditional deployments will be going down through that time. If you're looking around the market regularly, you should have ample time to notice that gigs are not what they used to be, so maybe it's time to change area.

Re: Introducing Cloud Spanner, a Global Database Service

#404

Post by behrangsa »

bdarnell wrote: post #189
karlding wrote: post #47

I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…

(Cockroach Labs CTO here) Google launching Spanner is generally a positive thing for our industry and our product. It's more proof that what we're aiming for is possible and that there's demand for it. We expect that in five years, all tech companies will be deploying technology like ours. One of the big differences is that Spanner only uses SQL for read-only operations, with a custom API for writes. We use standard…

Would Cockroach 1.0 comply with SQL:2011?

Re: Introducing Cloud Spanner, a Global Database Service

#405

Post by ngaut »

xapata wrote: post #130
karlding wrote: post #47

I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…

https://www.cockroachlabs.com/blog/living-without-atomic-clo... > A simple statement of the contrast between Spanner and CockroachDB would be: Spanner always waits on writes for a short interval, whereas CockroachDB sometimes waits on reads for a longer interval. How long is that interval? Well it depends on how clocks on CockroachDB nodes are being synchronized. Using NTP, it’s likely to be up to 250ms. Not great, b…

If the clock offset is too high (more than 250ms), we should use another transaction model, Google Percolator is a good fit before the unforeseeable improvement in the hardware. Based on the monitoring of clock offset on cloud, TiDB chose to use the timestamp oracle to allocate timestamp, which is much faster.

Re: Introducing Cloud Spanner, a Global Database Service

#406

Post by mallipeddi »

Few questions from reading the docs:

1) How big can all the colocated data for a single primary key get before they don't fit within a split? Can I implement a GMail-like product where all the data for a single user resides within one split?

2) Is there a way to turn off external consistency and fall back to serializability? In return you get better write latencies. This is similar to what CockroachDB provides?

Re: Introducing Cloud Spanner, a Global Database Service

#407

Post by grandinj »

xapata wrote: post #130
karlding wrote: post #47

I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…

https://www.cockroachlabs.com/blog/living-without-atomic-clo... > A simple statement of the contrast between Spanner and CockroachDB would be: Spanner always waits on writes for a short interval, whereas CockroachDB sometimes waits on reads for a longer interval. How long is that interval? Well it depends on how clocks on CockroachDB nodes are being synchronized. Using NTP, it’s likely to be up to 250ms. Not great, b…

Maybe you could help fund Eric S. Raymond to improve NTPD, he might have some good ideas about improving normal PC-class hardware cheaply too.

https://www.ntpsec.org/

Re: Introducing Cloud Spanner, a Global Database Service

#408

Post by throwawaydbfif »

pvg wrote: post #387
throwawaydbfif wrote: post #361

Earlier quoted context omitted.

Have you run your own servers in a colo? I've done it myself. One person, with maybe 3 hours a week of time investment after a few weeks of setup and hardware purchase. Using containers I can move between the cloud and my own servers seamlessly, and long as I never bite the golden apple and use any of the cloud's walled garden "services" like S3. If I need more power I can spin up some temporary servers at any cloud…

Did you ever read about how Netflix tried to run their own hardware but can't because they have so much data in AWS that it would basically bankrupt them to extract it? Where did you read this? You can have Amazon send you a truck full of hard drives. I doubt it costs more than Netflix can afford.

Nevermind, I misremembered the story I read about them. They moved the main site to AWS with the huge omission of their movie streaming system. Their own Open Connect servers are far cheaper to use for this becuase of massive AWS outbound data costs.

Also, the truck is for data in, not data out. Getting data out of AWS is far more expensive than putting it in. That's the lock in.

Re: Introducing Cloud Spanner, a Global Database Service

#409

Post by vgt »

anjanb wrote: post #399
tyingq wrote: post #325

Earlier quoted context omitted.

>the only growth market for Oracle right now is their hosted database service, and whoops Google has a better one now It could end up that way, but lacking INSERT and UPDATE will likely limit this to a niche market for now.

I thought SQL compatible means INSERT and UPDATE available. Why aren't these statements available ?

Technically those are dml statements.

Re: Introducing Cloud Spanner, a Global Database Service

#410

Post by the_cap_theorem »

>> Remarkably, Cloud Spanner achieves this combination of features without violating the CAP Theorem.

This is the best weasel PR language I have seen in a long time.

Note that the sentence does not actually proclaim that they solved (the previously "unsolvable") problem of achieving distributed consensus with unreliable communication while maintaining partition tolerance and availability.

The blog only says they don't "violate" the CAP theorem -- whatever that means. So the statement is technically correct. Still the intention is obviously to mislead the casual reader (why else would you start the sentence with "Remarkably"?).

A litmus test: The same statement is true for MySQL - or _any other_ database in fact:

  >> "Remarkably, MySQL achieves this combination of features without violating the CAP theorem"
It's a bit like saying

  >> "Remarkably, MySQL is not a perpetuum mobile"
Post reply on HN