Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Citus Unforks from PostgreSQL, Goes Open Source

citusdata.com

Post reply on HN
Print view View original

131–140 of 156 posts

Re: Citus Unforks from PostgreSQL, Goes Open Source

#131

Post by greggyb »

satygeek wrote: post #108

Does CitusDb fit in olap analytical workloads to do aggregations on hundreds millions of records using varying order and size of dimensions (eg druid) in max of 3 seconds response time using as few boxes as possible - Or there are other techniques have to be used along with Citusdb? Can you shed a light on your experience with CloudFlare in terms of cluster size and queries perf?

Hundreds of millions of records in I work for a BI consultancy and we don't even bat an eye until we hit billions of records in a primary fact table.

Certainly the DB server does need to scale vertically to some extent as you pass through the orders of magnitude > 10M. A good columnstore engine is also worthwhile to consider.

Re: Citus Unforks from PostgreSQL, Goes Open Source

#133

Post by gtrubetskoy »

hercynium wrote: post #130
wstrange wrote: post #109

Earlier quoted context omitted.

Cloudera and Hortonworks have raised a ton of VC money, but it is not clear (yet) that they have viable, profitable business models.

I hope that at least Cloudera will profit and thrive. Having used plain Apache components, then Hortonworks, then Cloudera... They are far and away the superior distribution, regardless of whether or not you are an "enterprise" customer. My $employer paid Hortonworks for a support contract and I have no qualms declaring publicly that it was a total and utter scam. (We are a Java shop and know our shit) If I go into t…

I have no experience with Hortonworks, but Cloudera certainly tries hard, even though it too has plenty of issues, but then the whole "Hadoop Echosystem" is in such fast-moving flux, it's understandable.

Cloudera also makes (Apache Licensed) Impala, which IMHO is a pretty cool product.

Another company worth mentioning is Databricks, which leads Apache Spark development.

Re: Citus Unforks from PostgreSQL, Goes Open Source

#134

Post by ioltas »

frn wrote: post #47
faizshah wrote: post #38

So this sounds similar to Pivotal's Greenplum which is also open source, can anyone compare the two?

Greenplum is based on postgres 8.2, with the featureset you'd expect from pg 8.2 - basically none of the additions after 2006 have merged to GP.

Since the move to open source, more recent upstream changes have been slowly merged in the code base, though they seem to be still on a 8.3 base, still a couple of years worth of code to go through.

Re: Citus Unforks from PostgreSQL, Goes Open Source

#135

Post by gtrubetskoy »

rch wrote: post #101
gtrubetskoy wrote: post #18

Earlier quoted context omitted.

Which means there is no chance this would ever become part of PostgreSQL proper.

That's correct, and with such a significant license change I think the term 'unfork' is being used inappropriately in the title. Edit: the PostGIS extension is GPL, and that license choice has been very successful. Hopefully the AGPL works out at least as well for Citus, I'm just not familiar enough to know what the implications will be in this context.

>> the PostGIS extension is GPL

But then it doesn't really belong or aim to be in PostgreSQL proper, at least IMO, so I think that's fine.

The functionality (i.e. distributed processing and horizontal scaling) that Citus has done is something that I predict will eventually be part of standard PostgreSQL, but it will not be Citus's code (unless they change the license, of course).

Re: Citus Unforks from PostgreSQL, Goes Open Source

#136

Post by albasha »

I recently switched back to MariaDB because I didn't see a clear/easy path for Postgres scalability in case the project i am working on takes off. I am under the assumption there are at least two fairly simple approaches to scale MySQL; master-master replication using Galera and Aurora from AWS. What do you guys think? Am I right in thinking MySQL is easier to scale given I want to spend the least amount of time maintaining it.

Re: Citus Unforks from PostgreSQL, Goes Open Source

#137

Post by mslot »

satygeek wrote: post #108

Does CitusDb fit in olap analytical workloads to do aggregations on hundreds millions of records using varying order and size of dimensions (eg druid) in max of 3 seconds response time using as few boxes as possible - Or there are other techniques have to be used along with Citusdb? Can you shed a light on your experience with CloudFlare in terms of cluster size and queries perf?

Yes, Citus may be a good fit, for a complete example see:

https://www.citusdata.com/blog/15-marco-slot/402-interactive...

Re: Citus Unforks from PostgreSQL, Goes Open Source

#138

Post by singlow »

belorn wrote: post #125
singlow wrote: post #66

Earlier quoted context omitted.

U.S. Copyright code allows you to make copies into memory or otherwise if it is required in order to utilize the software. https://www.law.cornell.edu/uscode/text/17/117

I remember that exception, and it was very explicit that you needed to already have obtained copyright permission to use the software. The legal theory behind the "copy to memory" exception is that if a end-user already have been granted permission to use a copyrighted work, then it make sense that they also have permission to "copy" it to their computers memory. If you have a copyright license that adds a condition…

It says, explicitly, that making copies for the purpose of executing it is not an infringement. I do not need permission if it is not an infringing act.

Re: Citus Unforks from PostgreSQL, Goes Open Source

#139

Post by lobster_johnson »

This is great!

One thing I'm having trouble with is finding information about transactional semantics. If I make several updates (to differently sharded keys) in a single transaction, will the transaction boundaries be preserved (committed "locally" first, then replicated atomically to shards)? Or will they fan out to different shards with separate begin/commit statements? Or without transactional boundaries at all?

In fact, I can't really find any information on how CitusDB achieves its transparent sharding for queries and writes. Does it add triggers to distributed tables to rewrite inserts, updates and deletes? Or are tables renamed and replaced with foreign tables? I wish the documentation was a bit more extensive.

Post reply on HN