I wonder if it's better in a non sharded use of something like Cockroach to push the replication and HA outside the database, let the control plane(e.g. kubernetes) handle failure and restarts on different nodes, and let something like Ceph with it's efficient write path and replication handle data durability.
Why CockroachDB and PostgreSQL Are Compatible
31–40 of 62 posts
Re: Why CockroachDB and PostgreSQL Are Compatible
#32I wonder if it's better in a non sharded use of something like Cockroach to push the replication and HA outside the database, let the control plane(e.g. kubernetes) handle failure and restarts on different nodes, and let something like Ceph with it's efficient write path and replication handle data durability.
You need to integrate transaction coordination with replication to ensure that the replication respects transaction atomicity (so that cross-shard queries get all their reads and writes isolated from each other, and rolled back atomically when a txn is aborted).
So separating the layers like you do is only possible if there is an XA protocol between the layers. Neither K8s nor Ceph support that.
Re: Why CockroachDB and PostgreSQL Are Compatible
#33How compatible? Can I just expect things like full text search to work? https://www.postgresql.org/docs/13/textsearch.html What about additional but supplied modules like ltree? https://www.postgresql.org/docs/current/ltree.html I ask as I saw a related article https://www.cockroachlabs.com/blog/full-text-indexing-search... recently and damn it looks close... has anyone migrated a prod system that uses the above? Wha…
Be very careful - even if things are compatible, in my experience some things do not perform as well in CockroachDB which seems a bit counter intuitive but can be very true... We had problems making date range queries fast with tiny amounts of data, for example. I would seriously consider if you need CockroachDB - if you need that level of scaling you should also consider Cassandra (or Cassandra like) solutions as th…
[deleted]
Re: Why CockroachDB and PostgreSQL Are Compatible
#34The title is deceitful since it is not really compatible.
Not every postgresql installation can perform every postrgresql feature (via plugins). That doesn't make postgresql incompatible with itself.
I use a postgresql connector to execute some postgresql specific sql statement on CockroachDB. That's a baseline quality for "compatible".
Re: Why CockroachDB and PostgreSQL Are Compatible
#35I am truly looking forward for CockroachDB to become the next PostgreSQL for planet-spanning database workloads. In our ecosystem we get more and more requests for CRDB integration. Generally though, I would not say that full compatibility should even be desired. A k/v database simply works differently from a strictly relational database. There are things like shard IDs, avoiding hot spots, deciding on how to paginat…
CockroachDB is not a key-value store though.
Re: Why CockroachDB and PostgreSQL Are Compatible
#36I am truly looking forward for CockroachDB to become the next PostgreSQL for planet-spanning database workloads. In our ecosystem we get more and more requests for CRDB integration. Generally though, I would not say that full compatibility should even be desired. A k/v database simply works differently from a strictly relational database. There are things like shard IDs, avoiding hot spots, deciding on how to paginat…
CockroachDB is not a key-value store though.
Unless I misunderstood the docs, it is a KV store underneath the abstraction layers:
> At the highest level, CockroachDB converts clients' SQL statements into key-value (KV) data, which is distributed among nodes and written to disk.
https://www.cockroachlabs.com/docs/stable/architecture/overv...
Re: Why CockroachDB and PostgreSQL Are Compatible
#37TL;DR: Because the Postgres folks have outstanding taste, even better documentation, a good community, and a compatible license. Which are all the same reasons that it's my goto database. I don't build software anymore, but the last project I worked on for money was a query engine, and the project founders were pretty open about the fact that whenever they were unclear about the behavior required by the SQL spec, the…
OT: your woodworking is amazing, always enjoyable to see someone transition to a more fulfilling line of work.
Thanks!
Re: Why CockroachDB and PostgreSQL Are Compatible
#38Earlier quoted context omitted.
(From the FAQs) How does the change to the BSL affect me as a CockroachDB user? It likely does not. As a CockroachDB user, you can freely use CockroachDB or embed it in your applications (irrespective of whether you ship those applications to customers or run them as a service). The only thing you cannot do is offer CockroachDB as a service without buying a license. - - - - - Cockroach is asking not to take open sour…
I know how their license negatively impacts me, I don't need some company to tell me that "all is well". I know that thier license means that if my requirements and Coackroach Labs business model diverge, I will be unable to fork the code and build a proper open source community around it. I know that I will (likely) never be able to buy a managed coackroach DB service from any vendor other than CoakroachLabs, no mat…
I don't think your points are totally wrong. I especially think that your points hold water if you were to feel the need to set out to build your community fork or competing hosted offering in the next couple of years. Secondarily, I think that if your company had the resources to staff developers to build out a whole hosted offering, I can tell you that CRL would love to be a partner and understand what's going wrong. Maybe that's unacceptable for some sort of other reason but it's not obvious to me what that is. Yes the license means that real-time development of the project, on some level, relies on coordinating with its developers, which do, on the whole, work for a company.
All this being said, it sounds like you've got a better vision for how projects like this should get funded and built. How should society fund projects like CRDB? I think there's a very wide range of reasonable answers that fall in different parts of the fantasy spectrum. For now, in 2020s American capitalism, the BSD license approach, at least to my tastes (which are hecka biased), feels pretty good.
Re: Why CockroachDB and PostgreSQL Are Compatible
#39Earlier quoted context omitted.
>Cockroach is asking not to take open source and start offering it as a service. Which sounds like completely reasonable thing to do! But i just wrote "be aware of it", with that license i don't consider it "Free Software", just imagine Apache or Nginx would do that. On the other hand I'm totally fine if they want to protect themself from leeches like Amazon/Google/Oracle/IBM.
> imagine Apache or Nginx would do that. Nginx will likely do something similar, very soon. It's not that long ago when F5 bought them, with clear plans to offer paid-only features while the core itself remains open. Give it another 5 years and I expect nginx to propagate some of those special features to the open-source version, but with the restriction that you can't offer them as a service vendor without a specifi…
I never understood the appeal of Nginx. I mean, I understand the web server, it's much easier than setting up apache, and before apache introduced event based mpm it was also much faster.
What I mean is the load balancing functionality of F5. It's in the same league as proxy support in apache[1]. Hardly comparable to for example what HAProxy provides, yet for some reason F5 (which is a leader in hardware load balancing, sadly though with public cloud that market is shrinking) found it interesting. Is it because of the name recognition, and they plan on turning Nginx into a proper load balancer? Because in terms of functionality Nginx is not even close to their LTM (even HAProxy is missing a lot of its features).
[1] I'm talking about the non paid version, there are some health checks in paid although still the support is rather weak.
Re: Why CockroachDB and PostgreSQL Are Compatible
#40If you want to use the CockroachDB be aware of the license: https://www.cockroachlabs.com/docs/stable/licensing-faqs.htm...
(From the FAQs) How does the change to the BSL affect me as a CockroachDB user? It likely does not. As a CockroachDB user, you can freely use CockroachDB or embed it in your applications (irrespective of whether you ship those applications to customers or run them as a service). The only thing you cannot do is offer CockroachDB as a service without buying a license. - - - - - Cockroach is asking not to take open sour…
So if I write OfftopAPI as a service which basically deploys Cockroach for clients , but adds a thin Node JS API on top , is that ok ?
Feels like you could play a game of chicken with them. I know many big companies would probably stick to postgres with this in mind.
And also makes it feel like they might just change the license later