Live data from Hacker News

Spanner vs. Calvin: distributed consistency at scale

fauna.com

11–20 of 52 posts

Re: Spanner vs. Calvin: distributed consistency at scale

#11

No pricing information for the self-hosted version. Gonna pass on this. Why even bother reading shitloads of text for something I can't try?

FaunaDB on-premises pricing will be by core and comparable to other commercial databases like Aerospike, Datastax DSE, etc.

Daniel's post is relevant to more than just FaunaDB though.

Re: Spanner vs. Calvin: distributed consistency at scale

#12
post #5
post #4

Welcome Dan. What other production implementations of Calvin are out there?

I said in my post: "influenced the design of several modern “NewSQL” systems" --- I'm not aware of other production implementations of Calvin. But VoltDB's command logging feature came directly from Calvin. So basically I had in mind FaunaDB and VoltDB when I wrote that sentence. Neither is an exact version of Calvin, but FaunaDB is closer to Calvin than VoltDB. Obviously, the Calvin paper has been cited many hundred…

How does Volt's transaction resolution mechanism compare? It sounds like that would be a third model yet.

Re: Spanner vs. Calvin: distributed consistency at scale

#13
Tangentially, while FaunaDB and Spanner both seem great, and I love consuming the technical details in articles like these, neither I or my companies are ever going to commit to using something which isn't open source.

I have a real problem understanding why Google (and now FaunaDB) are so clueless and/or inconsistent about software licensing. Google seemed to finally get it with Kubernetes, which has grown into a real and vibrant ecosystem; I will (and do) happily use Google's hosted Kubernetes product, while I know, at the same time, that I am not locked into a single vendor. With Spanner (and FaunaDB), all I get is lock-in. (The cynic in me suspects that Google's open-sourcing of Kubernetes is a cunning strategy to dominate the cloud by commoditizing container orchestration, and thus undermining competitors such as Amazon. After all, it's not like Google is open-sourcing Cloud Storage or BigQuery either.)

The fact that I can't run these products locally on my dev machine is another major downside unrelated to the philosophical aspects of free software.

I'm sure Google and Fauna will make truckloads of money on enterprises that are willing to bet on them, of course. Personally, I am holding out for CockroachDB.

Re: Spanner vs. Calvin: distributed consistency at scale

#14

Tangentially, while FaunaDB and Spanner both seem great, and I love consuming the technical details in articles like these, neither I or my companies are ever going to commit to using something which isn't open source. I have a real problem understanding why Google (and now FaunaDB) are so clueless and/or inconsistent about software licensing. Google seemed to finally get it with Kubernetes, which has grown into a re…

Understand. Some of the market is committed to open source, and so be it.

Most companies we talk to are more worried about being locked into specific infrastructure than specific software, since they would pay for the enterprise or hosted version of the open source database anyway. FaunaDB solves that problem by not being tied to any specific cloud.

There is a developer edition of FaunaDB on the way that you will be able to run locally for free.

Re: Spanner vs. Calvin: distributed consistency at scale

#15
post #5
post #4

Welcome Dan. What other production implementations of Calvin are out there?

I said in my post: "influenced the design of several modern “NewSQL” systems" --- I'm not aware of other production implementations of Calvin. But VoltDB's command logging feature came directly from Calvin. So basically I had in mind FaunaDB and VoltDB when I wrote that sentence. Neither is an exact version of Calvin, but FaunaDB is closer to Calvin than VoltDB. Obviously, the Calvin paper has been cited many hundred…

> But VoltDB's command logging feature came directly from Calvin.

VoltDev here. Huh? We added this feature in 2011 and read the Calvin paper sometime later IIRC.

Re: Spanner vs. Calvin: distributed consistency at scale

#17
post #6
post #3

Earlier quoted context omitted.

I can't comment on Google's decision here. I don't see any technical reason why they can't support client-side interactive transactions. But I'm happy to comment on technical points I made in the post ...

(Disclaimer: I work on Cloud Spanner). Cloud Spanner fully supports interactive read-write transactions. I'm not sure what the source of the confusion here is. Maybe Daniel is using a new definition of "client-side interactive transactions" that I'm unfamiliar with. :)

Sorry about that. As Evan said -- that sentence was based on something he told me. The post has now been fixed. My apologies.

Re: Spanner vs. Calvin: distributed consistency at scale

#18
post #15
post #5

Earlier quoted context omitted.

I said in my post: "influenced the design of several modern “NewSQL” systems" --- I'm not aware of other production implementations of Calvin. But VoltDB's command logging feature came directly from Calvin. So basically I had in mind FaunaDB and VoltDB when I wrote that sentence. Neither is an exact version of Calvin, but FaunaDB is closer to Calvin than VoltDB. Obviously, the Calvin paper has been cited many hundred…

> But VoltDB's command logging feature came directly from Calvin. VoltDev here. Huh? We added this feature in 2011 and read the Calvin paper sometime later IIRC.

"The case for determinism in database systems" paper (which described the technology that became Calvin) was written in VLDB 2010. At least one VoltDB developer told me that command logging came from a talk we gave about this paper to your team.

Re: Spanner vs. Calvin: distributed consistency at scale

#19

Tangentially, while FaunaDB and Spanner both seem great, and I love consuming the technical details in articles like these, neither I or my companies are ever going to commit to using something which isn't open source. I have a real problem understanding why Google (and now FaunaDB) are so clueless and/or inconsistent about software licensing. Google seemed to finally get it with Kubernetes, which has grown into a re…

> The fact that I can't run these products locally on my dev machine is another major downside unrelated to the philosophical aspects of free software.

You can't run a full spanner cluster unless you have the Google Hardware. Spanner needs to have an accurate clock with a known error bound. This makes it difficult to open source without open sourcing a bunch of hardware.

Re: Spanner vs. Calvin: distributed consistency at scale

#20
The all-to-all dependency step between Calvin's sequencer layer and scheduler layer seems like it will be a problem as things scale, because it means that a single stalled sequencer [edit, orig: scheduler] blocks all writes in the system whether they conflict or not. This is the kind of dependence structure that magnifies outlier latencies and unavailability at scale.

In Spanner's design, on the other hand, a transaction can only be blocked by another one on which it actually conflicts. It will have worse average performance, but better tail latency as things scale.

Perhaps it is just a nit, but the blog post is somewhat inaccurate when it says that Spanner uses the commit timestamp to order transactions. It uses locks to order the transactions, then holds the locks for extra time to ensure that TrueTime order agrees with the lock-based order.

Post reply on HN