PolarDB, yet another open source database system based on PostgreSQL • Hacker News Board

Re: PolarDB, yet another open source database system based on PostgreSQL

#91

Post by rad_gruchalski » Sun, May 30, 2021, 5:32 PM UTC

zinclozenge wrote: post #90

rad_gruchalski wrote: post #85

Earlier quoted context omitted.

Licensing. CockroachDB does not allow as-a-service.

Yugabyte does allow it though? I could take it and create my own company based around providing yugabyte as a service?

Yes. The core database is fully Apache 2 licensed with no strings attached: https://docs.yugabyte.com/latest/legal/. Maybe it will change if the company behind feels they're ripped off. Who knows, but for now, yes, you could.

CockroachDB has an explicit clause is the licensing: Yes, employees and contractors can use your internal CockroachDB instance as a service, but no people outside of your organization will be able to use it without purchasing a license: https://www.cockroachlabs.com/docs/v21.1/licensing-faqs.html....

Re: PolarDB, yet another open source database system based on PostgreSQL

#92

Post by jeff-davis » Sun, May 30, 2021, 5:33 PM UTC

mb4nck wrote: post #73

pella wrote: post #65

Earlier quoted context omitted.

IMHO: with the latest fix -> more stable search for "crash": https://www.postgresql.org/docs/release/11.3/ ( 11x ) https://www.postgresql.org/docs/release/11.4/ ( 4x ) https://www.postgresql.org/docs/release/11.5/ ( 1x ) https://www.postgresql.org/docs/release/11.6/ ( 3x ) https://www.postgresql.org/docs/release/11.7/ ( 14x ) https://www.postgresql.org/docs/release/11.8/ ( 5x ) https://www.postgresql.org/docs/release…

OTOH the PolarDB specific changes seem to be contained enough that if you decide to run it in production, you can probably just apply most of the changes from the v11 branch yourself. But I agree it's not a very good look to code-drop something on a .2 release when there's been 2,5 years of fixes.

The diffs are non-trivial (EDIT: only included changes in directories where conflicts are most likely to occur):

    $ git log --oneline REL_11_2..REL_11_12 -- src/backend src/include ':!src/backend/po' | wc -l
    536
    $ git diff --stat REL_11_2..REL_11_12 -- src/backend src/include ':!src/backend/po' | tail -1
     352 files changed, 14651 insertions(+), 7078 deletions(-)

Even if the conflicts are minor, it's going to be annoying to try to work it out. If you are hitting a specific crash, there's a good chance you can backport the fix cleanly, but I doubt you can just pull in all of the fixes proactively without some knowledge of the details of the fork.

I haven't really looked at the details... perhaps PolarDB already has many (or all) of the fixes since 11.2. Also I haven't actually tried a merge, I'm just assuming the difficulty based on the number of diffs (and my experience doing minor version merges in the past).

(Disclaimer: I work for Citus Data. Citus takes the approach of a pure extension, which means it works on unmodified Postgres, and minor upgrades typically don't interfere at all.)

Re: PolarDB, yet another open source database system based on PostgreSQL

#93

Post by pella » Sun, May 30, 2021, 6:13 PM UTC

jeff-davis wrote: post #88

mb4nck wrote: post #54

Earlier quoted context omitted.

Looks like it, yeah. Also, Postgres 12 introduced pluggable storage, which might help to implement a shared-nothing architecture without huge changes to vanilla Postgres (I haven't looked at how large their delta is)

Citus Data enables scale out, while also being a pure extension. That means you can upgrade Postgres like normal to the latest point release using whatever normal upgrade process you want (e.g. OS packages). It has worked for a long time without the need for Postgres 12. However, the new APIs introduced in v12 did enable us to offer columnar compression as an option, which complements a lot of scale-out use cases. Se…

> I work for Citus, and on columnar compression.

:-)

I am waiting for the basic index support for columnar tables !

https://github.com/citusdata/citus/pull/4950

:-)

Re: PolarDB, yet another open source database system based on PostgreSQL

#94

Post by speedgoose » Sun, May 30, 2021, 6:26 PM UTC

bob1029 wrote: post #68

speedgoose wrote: post #5

Earlier quoted context omitted.

Sometimes I dream about a highly available multi primary postgresql.

Do you also dream of loss of certain ACID semantics or crippling performance issues? In the real world, it is extremely hard to provide all the same guarantees you get out of a single instance of if you turn around and spread it across the internet. If you have super deep control over the physical & temporal environment around your system, you can cheat the rules a little bit (i.e. Google).

Yes I know that the perfect database cannot exist. I think some trade-off are possible though. Today I use postgresql with a primary and replicas, together with couchdb in the same application. They complement each other, but I think something good between is possible.

Re: PolarDB, yet another open source database system based on PostgreSQL

#95

Post by speedgoose » Sun, May 30, 2021, 6:31 PM UTC

rad_gruchalski wrote: post #79

speedgoose wrote: post #5

Earlier quoted context omitted.

Sometimes I dream about a highly available multi primary postgresql.

I would recommend having a look at Yugabyte.

Thanks. I should try it out.

Re: PolarDB, yet another open source database system based on PostgreSQL

#96

Post by sometimes666la » Mon, May 31, 2021, 1:17 AM UTC

iampims wrote: post #67

arcturus17 wrote: post #60

Earlier quoted context omitted.

I’m glancing over the Heroku Postgres docs and they seem to offer easy to set up read replicas called “followers”. Wouldn’t this work for you?

Heroku is only available in EU and US east. Latency from Australia is somewhat unavoidable. You can maybe reduce it by using Cloudflare or similar to terminate TLS as close to your users as possible.

Cloudflare give different Anycast IPs for each plan and they don't always hit the local ingestion point. Some people shot themselves in the foot with free/pro plan in certain markets (India, aus etc..) I don't believe in May 2021 local ingestion (TLS termination) is happening on anything less than Business plan in MEL or SYD

Re: PolarDB, yet another open source database system based on PostgreSQL

#97

Post by DaiPlusPlus » Mon, May 31, 2021, 5:07 AM UTC

speedgoose wrote: post #47

DaiPlusPlus wrote: post #12

Earlier quoted context omitted.

CockroachDB?

Perhaps. It's missing the 30 years of experience, the PostGreSQL reputation, an opensource license, and a better name.

> a better name

NGL, I've had to explain to enough people by now that the "funny name" is because it was made by ex-Googlers with a view towards resiliency, as per the idiom that only cockroaches will survive global nuclear war.

Re: PolarDB, yet another open source database system based on PostgreSQL

#98

Post by funny_falcon » Mon, May 31, 2021, 7:19 AM UTC

unityByFreedom wrote: post #51

greatpatton wrote: post #37

Earlier quoted context omitted.

They did it only after ES decided to abandon the Apache license. I doubt that Amazon would have done it if they were not forced. Redshift is also Postgres based and Amazon never released something. So based on that metric Alibaba is more open.

There would be no open source Elastic had Amazon not taken it on. It was a smart move and benefits their balance sheet to do so. The point though is that move was greater than replicating existing open source work.

? Team on my previous workplace patched ElasticSearch to add functionality for years. Could thay do it if there were no open source Elastic?

Re: PolarDB, yet another open source database system based on PostgreSQL

#99

Post by arthurcolle » Tue, Jun 01, 2021, 2:10 AM UTC

manigandham wrote: post #55

arthurcolle wrote: post #41

Earlier quoted context omitted.

I think Citus is one option. Or possibly TimescaleDB

Those are meant for a single location, not cross-region distribution.

I don't think that's right.

Citus has a post from 2017 talking about allowing for horizontal scaling of a postgres instance. [0]

Same thing with Timescale. [1]

--------------------------------------------

[0] https://citusdata.com/blog/2017/02/16/citus61-released/

[1] https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node...

Re: PolarDB, yet another open source database system based on PostgreSQL

#100

Post by yftsui » Tue, Jun 01, 2021, 4:59 AM UTC

foobarbazetc wrote: post #70

wanghq wrote: post #15

Oh, what's going on here? > OceanBase, the database of Alibaba's fintech company Ant Group, will be open-source soon, possibly as early as June 1, according to Sina Tech. https://cntechpost.com/2021/05/27/ant-groups-in-house-databa... Also the team published many papers about PolarDB. https://scholar.google.com/scholar?hl=en&as_sdt=0%2C48&q=%22...

Alibaba is huge and different teams have different goals. That one is for fintech.

These are just KPI projects as their performance are judged by meaningless metrics. OceanBase WAS open sourced, after the news reports and publicity, Alibaba deleted all code and replaced it with a Chinese announcement says it will no longer be an open source project.