Live data from Hacker News

Postgresql 9.6: Support parallel aggregation

postgresql.org

51–60 of 64 posts

Re: Postgresql 9.6: Support parallel aggregation

#51
post #25

It looks like in 9.4 PG went after the document databases and now they are after Oracle. I don't know why I get excited with point releases of postgres, but I do.

Because in the PostgreSQL world those are actually major versions, not "point updates" ;-)

Apt user name

Re: Postgresql 9.6: Support parallel aggregation

#52

Short of doing a deep dive into Postgresql, do any universities use Postgresql as a basis for their DB implementation courses? I wanted to dive into Postgres and was hoping for some training wheels. Thanks.

CS186 at Berkeley was entirely in the guts of PostgreSQL. But then, I suppose that's not too surprising given the history.

Personally, I found it a delight to work with, and that has translated into a great respect for the product itself. It is no great surprise that the pace of development stays strong and that engineers gravitate toward it; it's set up for exactly that.

Re: Postgresql 9.6: Support parallel aggregation

#53
post #40

A slight tangent, but I am still shocked that many open source projects lack downloads via https (including PostgreSQL). Sure you can offer some layer of security by signing the distributions, but ultimately users are lazy.

As a project, I don't think PostgreSQL wants (or should want) anybody running a DB from the source tarball that's (eventually) stamped as 9.6 - it's existence is a mere artifact of the actual packaging work that goes on during a release. In all cases you should be installing packages via your OS's installation mechanism, and those packages will be authenticated (or not, as the case may be, but it's not in-scope for P…

> you should be installing packages via your OS's installation mechanism.

My OS's install mechanism downloads the source tarball. It does authenticate the download, of course. The point being you don't know what platform someone is using, and the source may be the only way they can install PostgreSQL.

> PostgreSQL wants

It doesn't matter what they want - if the source is available for download, it will be used. Unauthenticated downloads are an "attractive nuisance" that puts users at risk. The actual download links[1] at www.postgresql.org do use https, but the HTML that contains the download URLs irresponsibly redirects https requests back to http. While the download of the actual source tarball is authenticated, the URL to that download can easily be modified in transit.

[1] https://www.postgresql.org/ftp/source/v9.5.1/

Re: Postgresql 9.6: Support parallel aggregation

#54
post #35

I am totally impressed by the work 2ndQuadrant is doing: many of the recent innovations to Postgres have been done by them and all of that without any obligation for them to be doing so. The BSD license would allow them to add all of these things to a proprietary fork that they could be selling. Or they could just release their own fork under an open license and focus on just adding features. But that's not how they…

While I do work for 2ndQuadrant and think that David did a very fine job on this great patch, it'd be a bit unfair not to mention this patch stands on parallel infrastructure built by EDB (and Robert Haas & Amit Kapila in particular) and others.

Kudos to them.

Re: Postgresql 9.6: Support parallel aggregation

#55
post #14
post #6

Earlier quoted context omitted.

It would help a lot for typical data warehouse queries that involves aggregating millions to billions of rows. SELECT EXPLAIN ANALYZE will tell if its relevant for you ;-)

> SELECT EXPLAIN ANALYZE will tell if its relevant for you That assumes I run PostgreSQL already, which I don't. I am interested in possibly switching at some point if it's worthwhile, but it's hard to muster the effort to do concerted testing of a representative sample of my data, including possibly changing how queries are done to take advantage of specific features, when I have little information to go on. Not tha…

If you aren't using it now, and thus don't know how fast or slow it would be on your data and queries, why would you care about percentage speed ups?

It could be a 1% speed up from insanely fast to slightly more insanely fast, or a 100% speed up from unbearably slow to just extremely annoyingly slow.

I gather that people considering migrating who want some reassurance before investing time in testing should be more interested in such things things as benchmarks that show whether PostgreSQL can saturate your hardware for various query types, benchmarks that compare its query planner against that of competitors, and benchmarks that show how well it works under load.

Re: Postgresql 9.6: Support parallel aggregation

#56

FWIW, a simple benchmark by David Rowley (one of the authors of the patch) are here: http://blog.2ndquadrant.com/parallel-aggregate/

The article claims a near linear speedup for a very large query. On a 64 core machine, the query takes:

1375s with 0 workers

131s with 10 workers

56s with 30 workers

Re: Postgresql 9.6: Support parallel aggregation

#57
post #30

Earlier quoted context omitted.

First hand experience, I work for a bank which is a Fortune 100, and our department was heavily relying on Oracle Exadata, paying between 1-2 millions (don't know exact figure), we moved away to PostgreSQL and Casandra. Cost factor was not the only motive (1-2 millions was actually penny for this bank) but our Director was convinced that exadata was unnecessary.

Out of interest, what are the benefits of Exadata, or is it really a bit of emperorer's new clothes? Can you not get similar performance with similar good hardware?

Exadata:Hardware used to be competitively priced (at least in the x4 age), its the licensing for the software on top that makes it really expensive.

The other thing is that some oracle features only work on exadata or other engineered systems. The hybrid columnar compression for one and query offload as well as some forms of the in memory stuff. The problem is that these are very high end features and most of us live happily without these. Postgres is going after the bread in the oracle db market while exadata is the exclusive cheese shop. Much smaller market even if the margins are better.

Re: Postgresql 9.6: Support parallel aggregation

#58
post #35

I am totally impressed by the work 2ndQuadrant is doing: many of the recent innovations to Postgres have been done by them and all of that without any obligation for them to be doing so. The BSD license would allow them to add all of these things to a proprietary fork that they could be selling. Or they could just release their own fork under an open license and focus on just adding features. But that's not how they…

I wish they would build a RDS alternative. I would use them in a heartbeat. There are all these companies/startups who are not in the target market for consulting... But who would gladly pay for hosted postgres.

Just a little heads-up: Postgres is very easy to run on your own and even in the default configuration runs well even for a considerable amount of users.

Over here, we only started to seriously thinking about what we're doing once we were handling in the order of 10K transactions per second.

Once you are at that level, you're probably going to need optimizations specific to your application and a generic database hoster might not be able to help you anyways.

I get that as a startup you don't have people for everything, but can you really afford to outsource the knowledge about the central piece of your application where all the value is stored at?

Re: Postgresql 9.6: Support parallel aggregation

#59
post #26

Earlier quoted context omitted.

Out of curiosity what does it matter if your postgres download is via http or https? Is there any benefit of hiding your download from prying eyes?

To put it another way it's less about prying eyes and more about MitM attacks.

This is true, and MitM attacks are becoming less and less theoretical as more traffic moves to the various wireless protocols. Rogue AP's and cell towers are a thing in these modern times.

Re: Postgresql 9.6: Support parallel aggregation

#60

A slight tangent, but I am still shocked that many open source projects lack downloads via https (including PostgreSQL). Sure you can offer some layer of security by signing the distributions, but ultimately users are lazy.

It's OSS so there's not much to hide.

I'm sure that someone who has capability to not only tap into but also modify traffic on the fly, won't have problem obtaining valid certificate from one of the hundreds CAs that everyone has in their web browsers.

If anything I feel https actually hurts, because it gives you false sense of security and you're more likely not to validate files with GPG, which you should do whether it is https or not.

Post reply on HN