“Big data” features coming in PostgreSQL 9.5 • Hacker News Board

Re: “Big data” features coming in PostgreSQL 9.5

#11

Post by lukaslalinsky » Sat, Aug 15, 2015, 6:51 PM UTC

kodablah wrote: post #8

I would love to go back to relational, but I can't get over the linear scaling and HA ease of Cassandra and I'm willing to write my applications to work around the data model limitations. Is there any open source SQL database (i.e. "newsql") that can linearly scale with Cassandra's ease right now? I look forward to CockroachDB but it isn't done yet.

There are some SQL databases build on top of HBase. HBase and the whole Hadoop ecosystem is not what I'd call easy, but once you are familiar with it, scaling it is relatively easy (probably still not as easy as Cassandra, thanks to the fact that Cassandra always hashes things and does not allow sorted range scans).

I'm currently experimenting with Phoenix, because it's a relatively thin layer above HBase, but it doesn't provide many features that you expect in relational databases like transactions. SELECT support is quite complete, it can do JOINs, subqueries, etc.

I have recently discovered Trafodion which looks like a much more complete relational database, but at the expense of no longer being just a HBase client.

http://phoenix.apache.org/ https://wiki.trafodion.org/wiki/index.php/Main_Page https://vimeo.com/128190098

Re: “Big data” features coming in PostgreSQL 9.5

#12

Post by tsuraan » Sat, Aug 15, 2015, 6:59 PM UTC

warbiscuit wrote: post #10

I love PG, and don't plan to ever move away from it. But I'm starting to wish they had in-place upgrades. Application I develop is getting to the point of "too large to make a copy of the whole db whenever I want to upgrade Postgres". I can postpone things for a little while by replacing all our large objects with a external file storage -- but the tables themselves are growing quickly. That'd be my favorite "big dat…

You can run pg_upgrade with the --link option to have it hardlink most of the tables into the new cluster's location. It'll still re-write some things, but I believe that most of the actual table data remains unaltered.

Re: “Big data” features coming in PostgreSQL 9.5

#13

Post by ericfrenkiel » Sat, Aug 15, 2015, 7:55 PM UTC

kodablah wrote: post #8

I would love to go back to relational, but I can't get over the linear scaling and HA ease of Cassandra and I'm willing to write my applications to work around the data model limitations. Is there any open source SQL database (i.e. "newsql") that can linearly scale with Cassandra's ease right now? I look forward to CockroachDB but it isn't done yet.

It's commercial, but MemSQL Community Edition scales horizontally. It's free and there are no capacity or scaling limits.

Re: “Big data” features coming in PostgreSQL 9.5

#14

Post by ozgune » Sat, Aug 15, 2015, 8:49 PM UTC

kodablah wrote: post #8

I would love to go back to relational, but I can't get over the linear scaling and HA ease of Cassandra and I'm willing to write my applications to work around the data model limitations. Is there any open source SQL database (i.e. "newsql") that can linearly scale with Cassandra's ease right now? I look forward to CockroachDB but it isn't done yet.

(Disclaimer: I'm one of the authors of pg_shard.)

pg_shard extends PostgreSQL without forking it, and it's worth a look: https://github.com/citusdata/pg_shard

pg_shard v2.0 will target linear scaling and higher HA through metadata replication. If you have any questions or feedback for the project, we'd be happy to hear from you.

If you're on AWS, you can already get linear scale and full HA. This post shows a proof-of-concept: https://goo.gl/3c2GYc

Re: “Big data” features coming in PostgreSQL 9.5

#15

Post by polskibus » Sat, Aug 15, 2015, 9:23 PM UTC

Does anyone know about TABLESAMPLE equivalents in other RDBMS, for example in MS SQL?

Re: “Big data” features coming in PostgreSQL 9.5

#16

Post by chucksmash » Sat, Aug 15, 2015, 9:45 PM UTC

Joyfield wrote: post #5

I should really learn some PG and move away from MySQL. But at 40 years old i have a problem learning how to sit properly. sigh

It's certainly possible to use Postgres productively without ever doing anything advanced.

My use case can pretty well be summed up as "things an ORM can do for you out of the box." As a developer living in the shallow end of the RDBMS pool, switching a hobby project over is a matter of hours. If your usage is similar, learning it well enough to get by could just be a matter of:

  apt-get install postgres

Re: “Big data” features coming in PostgreSQL 9.5

#17

Post by bmh100 » Sat, Aug 15, 2015, 9:52 PM UTC

It's nice to PostgreSQL adding all of these analytics and business intelligence features. BI platforms have had these sorts of features for a least a decade.

Re: “Big data” features coming in PostgreSQL 9.5

#18

Post by pfg » Sat, Aug 15, 2015, 9:59 PM UTC

polskibus wrote: post #15

Does anyone know about TABLESAMPLE equivalents in other RDBMS, for example in MS SQL?

MS SQL has TABLESAMPLE support: https://technet.microsoft.com/en-us/library/ms189108(v=sql.1...

I don't think it has something like TABLESAMPLE BERNOULLI.

Re: “Big data” features coming in PostgreSQL 9.5

#19

Post by threeseed » Sat, Aug 15, 2015, 10:07 PM UTC

These aren't "big data" features but rather EDW/BI features.

If PostgreSQL can't trivially scale horizontally like Cassandra or Riak or integrate nicely with Spark/Hadoop like MongoDB then it isn't particularly useful in typical big data roles.

Re: “Big data” features coming in PostgreSQL 9.5

#20

Post by mdaniel » Sat, Aug 15, 2015, 10:57 PM UTC

minimaxir wrote: post #7

Joyfield wrote: post #5

I should really learn some PG and move away from MySQL. But at 40 years old i have a problem learning how to sit properly. sigh

Syntax-wise, Postgres is not much different from MySQL. You just have to learn how to use window functions which takes a bit.

I don't think people have a concern that Postgres is the one with the wacky syntax. And it's been my experience that unless the team is very disciplined, if a language offers a thing, you'll find it in the codebase. Thus, ripping out all the backticks where you've used "select" or "from" as a column name isn't a postgres problem.

To really go out on a limb, I would posit that MySQL's view on RDBMS functionality influences its use, which I bet does not square with more traditional systems. Anytime one finds a shift in world view, the code becomes hard to rewrite and still behave the same way.