Spanner's paper: https://static.googleusercontent.com/media/research.google.c...
Introducing Cloud Spanner, a Global Database Service
121–130 of 456 posts
Re: Introducing Cloud Spanner, a Global Database Service
#122Earlier quoted context omitted.
> If you believe that partitions are inevitable, any distributed system How does that answer it? Are they implying that partitions will not happen if you don't believe in them?
A CA system is not a system that doesn't have partitions, it's a system that works under the condition that there are no partitions (ie. it is a non-partition-resistant system).
A memcache cluster might qualify. If there's a partition, just forget about the missing nodes. It's just a cache anyway.
Re: Introducing Cloud Spanner, a Global Database Service
#123Congratulations to the Spanner team for becoming part of the Google public cloud! And for those wondering, this is why Oracle wants billions of dollars from Google for "Java Copyright Infringement" because the only growth market for Oracle right now is their hosted database service, and whoops Google has a better one now. It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If…
As a bit of a veteran in the database industry, I concur (at least about the impact on Oracle's database business). There is a lot of pent-up demand for anything that offers distributed consistency.
We've been seeing this demand at Fauna. FaunaDB offers distributed consistency, based on Raft and the Calvin protocol instead of depending on specific networking and clock hardware. We've seen a big part of our appeal is the ability to run FaunaDB across multiple cloud services.
Re: Introducing Cloud Spanner, a Global Database Service
#124Earlier quoted context omitted.
Amazon's Aurora databases seem to be solving the same problem, and are MySQL or Postgres compatible to boot.
Aurora is very cool but won't help you much after you vertically scale your master and still need more write capacity. With Cloud Spanner you get horizontal write scalability out of the box. Critical difference.
So if I'm understanding you, with Aurora all writes go to one master and you're constrained by the biggest instance AWS offers. Is that right?
Do you have a sense of what that limit is?
There's a pretty big price difference between Spanner and Aurora at the entry level so it's useful to explore this.
Re: Introducing Cloud Spanner, a Global Database Service
#125I wonder why they charge a minimum of $0.90 per node-hour when they offer VMs for as little as $0.008/hr. This is hugely useful even for single-person startups, so why charge a minimum of ~$8,000 per year?
A Spanner node is not related to a VM instance, they are entirely different concepts. A Spanner node refers to a collection of regional (and eventually global) compute and storage resources, while a VM instance is just a virtual machine running on a single physical host.
Re: Introducing Cloud Spanner, a Global Database Service
#126The team here at Quizlet did a lot of performance testing on Spanner with one of our MySQL workloads to see if it's an option for us. Here are the test results: https://quizlet.com/blog/quizlet-cloud-spanner
> Not every application can handle Spanner's ~5ms minimum query time, but if you can, then you can have that latency for a very high-throughput workload
This is the tradeoff we've all been looking for. Cool product, anyway!
Re: Introducing Cloud Spanner, a Global Database Service
#127Earlier quoted context omitted.
> the only growth market for Oracle right now is their hosted database service This is not true. Oracle is far more than a database company nowadays in the same way that Microsoft is more than Windows. Oracle has been acquiring high-growth startups at a significant rate.
I was going by this document: http://s1.q4cdn.com/289076952/files/doc_financials/quarterly... Which has their 'cloud services' doubling their contribution to revenue year over year and licenses losing 50% of their contribution to revenue year over year. There 'cloud' collateral is pretty opaque though.
Yes, the "cloud" includes all of their non-database offerings too which have been the focus of recent growth/acquisitions.
Re: Introducing Cloud Spanner, a Global Database Service
#128I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…
The main sales pitch of Cloud Spanner is Google's network infrastructure. No startup will be able to replicate that anytime soon, a lot of time (and money) has been put into it by a lot of people over a long time.
Curious: is there any company in the world that could replicate its breadth, performance, and reliability in the next decade?
Could any government? Has any government?
My impression is that, infrastructure wise, Google is genuinely in a class of size one.
Re: Introducing Cloud Spanner, a Global Database Service
#129Congratulations to the Spanner team for becoming part of the Google public cloud! And for those wondering, this is why Oracle wants billions of dollars from Google for "Java Copyright Infringement" because the only growth market for Oracle right now is their hosted database service, and whoops Google has a better one now. It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If…
> It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If we get to the point where you have databases, compute, storage, and connectivity services from those three at equal scale, well that would be a lot of choice for the developers!
There are also plenty of choices evolving for developers who aren't looking for hosted solutions (which can sometimes be a showstopper for enterprise on-prem deployments). There's a growing ecosystem of distributed open-source databases to look out for too.
Take Citus, for instance – a Postgres-compatible distributed store which automatically parallelizes normal SQL queries across machines. It's as easy to set up as adding an extension, and people are doing some staggering things in prod with it.
Different audience from BigQuery and Spanner, but no less exciting.
Disclaimer: no professional association, but love their product and the team.
Re: Introducing Cloud Spanner, a Global Database Service
#130I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2]. [1] https://www.cockroachlabs.com/docs/frequently-asked-questi…
https://www.cockroachlabs.com/blog/living-without-atomic-clo...
> A simple statement of the contrast between Spanner and CockroachDB would be: Spanner always waits on writes for a short interval, whereas CockroachDB sometimes waits on reads for a longer interval. How long is that interval? Well it depends on how clocks on CockroachDB nodes are being synchronized. Using NTP, it’s likely to be up to 250ms. Not great, but the kind of transaction that would restart for the full interval would have to read constantly updated values across many nodes. In practice, these kinds of use cases exist but are the exception.
CockroachDB is waiting for time keeping hardware to improve.