Why CockroachDB and PostgreSQL Are Compatible

I cannot help but being saddened by this. It is really hard to understand how not knowing how to interface directly to the part of the system that holds the ultimate reason we write a program (handling data), and probably the performance bottleneck of that system when it scales, is beneficial to a professional developer.

While I agree that that can work for maintaining existing and valuable projects, I have some doubts that that can work to fund projects getting to that point. Postgres, additionally, grew out of a proprietary commercial enterprise in Ingress and then became usable with a sizable payroll from the University of California (as far as I understand it). I do think that public investment like large grants to fund teams to work on open source software would be a great thing.

I think another factor here is the timing and the context of the moment. There is a turning tide in the data systems world whereby just having something you can run isn't good enough because the overhead to figure out the operations just isn't in the budget when there are hosted solutions out there.

> Sure, CRL may be interested in partnering and playing nice- now. But they hold the keys to the kingdom, and can lock the door anytime they want to.

This is only somewhat true. The BSL license cockroach uses converts all code to Apache after 3 years. While in the short term this likely means that CRL holds the keys, if this investment builds the quality of product we hope and believe it will, that corpus of code will be available for decades to come. I do appreciate the quality product that is postgres, but I can also say that I've built services that have worked quite well on Postgres 9 which was released over 10 years ago. I'm not saying that good stuff hasn't happened, but that if cockroach is able to fund its way to a somewhat finished project that proves its worth in enterprise deployments, it will have to have been successful and valuable for more than 3 years.

Open core can be okay. CRL engages in some of that too. It's hard to know where to draw the line. I much prefer a 3 year, permissive BSL to the open core enterprise code. Maybe that's just me.

> RedHat is pretty successful at selling support and consulting.

Was pretty successful at it. It's an IBM brand now. Also, it grew up and thrived in the era when you needed a lot of investment and expertise to build, run, and manage datacenters. That world is ending.

Another thing I'll note is that Postgres is fundamentally simpler than crdb, or at least than cockroach would like to ultimately be. WordPress is way way simpler than both of them. Hundreds of engineer years is a pretty steep cost price to get something to that bar of really being valuable. The opportunity cost landscape for software developers alone has shifted what it might take to make a postgres-scale database happen again: my guess is that if the same calliber programmers from the 90s at Berkeley were tinkering on systems getting paid public university wages today, they'd end up in jobs elsewhere pretty quickly.

So yeah, I'd love for it to make sense for everything every company did to be at least big O Open Source, if not even big F Free Software. That'd be a cool world. Imagine the world where everything running Google, Amazon, Microsoft were building blocks we could all learn from and shape to our needs. That'd be sweet. In that world, crdb and crl might not need to exist, and that'd be totally cool too.

Note also that Google and Amazon not just have hosted versions of Postgres, they also have adapted them into new products and they make a heck of a lot of money from those products without sharing any of the tech.

It's not fair to draw direct comparisons from what has worked to what might work today. A 3 year delay for fully permissive licensing is something that lets me sleep pretty well while still leaving me way way more privileged than sometimes feels reasonable.

What are the salient differences in your mind? Under the hood, CockroachDB executes writes to and reads from such a table in the same way that you would against a key-value store. You can explore this for yourself with the "kv trace" functionality of CockroachDB's SQL shell, which logs of all of the KV API calls that a SQL query emits:

  $ ./cockroach demo
  # Welcome to the CockroachDB demo database!
  #
  # You are connected to a temporary, in-memory CockroachDB cluster of 1 node.
  # ...
  #
  demo@127.0.0.1:26257/test> CREATE TABLE kv (k STRING PRIMARY KEY, v STRING);
  CREATE TABLE
  
  Time: 5ms total (execution 5ms / network 0ms)
  
  demo@127.0.0.1:26257/test> \set auto_trace=on,kv
  demo@127.0.0.1:26257/test> INSERT INTO kv VALUES('a', 'b');
  INSERT 1
  
  Time: 2ms total (execution 2ms / network 0ms)
  
                 timestamp              |       age       |                     message                      |                            tag                             |                location                 |    operation     | span
  --------------------------------------+-----------------+--------------------------------------------------+------------------------------------------------------------+-----------------------------------------+------------------+-------
    2020-12-17 23:17:46.626696+00:00:00 | 00:00:00.001123 | CPut /Table/53/1/"a"/0 -> /TUPLE/2:2:Bytes/b     | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/row/writer.go:207                   | flow             |    6
    2020-12-17 23:17:46.626754+00:00:00 | 00:00:00.001181 | querying next range at /Table/53/1/"a"/0         | [n1,client=127.0.0.1:49216,hostssl,user=demo,txn=dcce3954] | kv/kvclient/kvcoord/range_iter.go:159   | dist sender send |    8
    2020-12-17 23:17:46.626792+00:00:00 | 00:00:00.001219 | r36: sending batch 1 CPut, 1 EndTxn to (n1,s1):1 | [n1,client=127.0.0.1:49216,hostssl,user=demo,txn=dcce3954] | kv/kvclient/kvcoord/dist_sender.go:1851 | dist sender send |    8
    2020-12-17 23:17:46.627281+00:00:00 | 00:00:00.001708 | fast path completed                              | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/plan_node_to_row_source.go:145      | flow             |    6
    2020-12-17 23:17:46.627322+00:00:00 | 00:00:00.001749 | rows affected: 1                                 | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/conn_executor_exec.go:622           | exec stmt        |    4
  (5 rows)
  
  Time: 1ms total (execution 1ms / network 0ms)
  
  demo@127.0.0.1:26257/test> SELECT * FROM kv WHERE k = 'a';
    k | v
  ----+----
    a | b
  (1 row)
  
  Time: 6ms total (execution 6ms / network 0ms)
  
                 timestamp              |       age       |                message                 |                            tag                             |                location                 |    operation     | span
  --------------------------------------+-----------------+----------------------------------------+------------------------------------------------------------+-----------------------------------------+------------------+-------
    2020-12-17 23:17:54.402735+00:00:00 | 00:00:00.003116 | Scan /Table/53/1/"a"{-/#}              | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/row/kv_batch_fetcher.go:337         | materializer     |    7
    2020-12-17 23:17:54.402763+00:00:00 | 00:00:00.003144 | querying next range at /Table/53/1/"a" | [n1,client=127.0.0.1:49216,hostssl,user=demo,txn=d30bcbc9] | kv/kvclient/kvcoord/range_iter.go:159   | dist sender send |    9
    2020-12-17 23:17:54.404565+00:00:00 | 00:00:00.004946 | r36: sending batch 1 Scan to (n1,s1):1 | [n1,client=127.0.0.1:49216,hostssl,user=demo,txn=d30bcbc9] | kv/kvclient/kvcoord/dist_sender.go:1851 | dist sender send |    9
    2020-12-17 23:17:54.405091+00:00:00 | 00:00:00.005472 | fetched: /kv/primary/'a'/v -> /'b'     | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/colfetcher/cfetcher.go:888          | materializer     |    7
    2020-12-17 23:17:54.405895+00:00:00 | 00:00:00.006276 | rows affected: 1                       | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/conn_executor_exec.go:622           | exec stmt        |    4
  (5 rows)
  
  Time: 1ms total (execution 1ms / network 0ms)
  
  demo@127.0.0.1:26257/test>

  2020-12-17 23:17:46.626696+00:00:00 | 00:00:00.001123 | CPut /Table/53/1/"a"/0 -> /TUPLE/2:2:Bytes/b     | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/row/writer.go:207                   | flow             |    6

  2020-12-17 23:17:54.402735+00:00:00 | 00:00:00.003116 | Scan /Table/53/1/"a"{-/#}              | [n1,client=127.0.0.1:49216,hostssl,user=demo]              | sql/row/kv_batch_fetcher.go:337         | materializer     |    7

Getting your boss to buy into a new tech is hard enough