Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL Parallel Query v2

rhaas.blogspot.com

Post reply on HN
Print view View original

11–20 of 44 posts

Re: PostgreSQL Parallel Query v2

#11

Post by rodgerd »

fleetside72 wrote: post #2

The relationship between EnterpriseDB and the at-large Postgres community has always intrigued me. Why do they turn over to the community what they could make part of their proprietary fork? Maybe they make a lot of money consulting and not just licenses and general at-large adoption is good for that side of business.

Well, Red Hat have demonstrated you can make a lot of money by open sourcing things (including buying proprietary software like ManageIQ and oVirt and open sourcing it). Given that EDB has ex-RHAT folks involved in setting it up, I imagine they're trying to replay the model.

Re: PostgreSQL Parallel Query v2

#12

Post by colanderman »

malisper wrote: post #10

Besides parallel queries (specifically parallel index scans), the Postgres 10 feature I'm most looking forward to is the work Andres Freund has done on using LLVM to JIT compile expressions[0]. Based on the the layout of the database I work on[1], an insane amount of CPU goes to processing of partial index predicates. If the improvements are anywhere near what is being described in the mailing list, we're going to st…

Don't forget native logical replication.

Re: PostgreSQL Parallel Query v2

#13

Post by hendzen »

rodgerd wrote: post #11
fleetside72 wrote: post #2

The relationship between EnterpriseDB and the at-large Postgres community has always intrigued me. Why do they turn over to the community what they could make part of their proprietary fork? Maybe they make a lot of money consulting and not just licenses and general at-large adoption is good for that side of business.

Well, Red Hat have demonstrated you can make a lot of money by open sourcing things (including buying proprietary software like ManageIQ and oVirt and open sourcing it). Given that EDB has ex-RHAT folks involved in setting it up, I imagine they're trying to replay the model.

RedHat Q4 FY2016 profit: $68M

Oracle Q4 FY2016 profit: $2B

Re: PostgreSQL Parallel Query v2

#14

Post by malisper »

colanderman wrote: post #12
malisper wrote: post #10

Besides parallel queries (specifically parallel index scans), the Postgres 10 feature I'm most looking forward to is the work Andres Freund has done on using LLVM to JIT compile expressions[0]. Based on the the layout of the database I work on[1], an insane amount of CPU goes to processing of partial index predicates. If the improvements are anywhere near what is being described in the mailing list, we're going to st…

Don't forget native logical replication.

Logical replication doesn't matter as much to us. We are using CitusDB (in other words sharded Postgres), and we have our own code for handling replication.

Re: PostgreSQL Parallel Query v2

#15

Post by zzzzzzzza »

hendzen wrote: post #13
rodgerd wrote: post #11

Earlier quoted context omitted.

Well, Red Hat have demonstrated you can make a lot of money by open sourcing things (including buying proprietary software like ManageIQ and oVirt and open sourcing it). Given that EDB has ex-RHAT folks involved in setting it up, I imagine they're trying to replay the model.

RedHat Q4 FY2016 profit: $68M Oracle Q4 FY2016 profit: $2B

will oracle be around in 10-20 years? will red hat? which will experience growth?

Re: PostgreSQL Parallel Query v2

#16

Post by fnord123 »

malisper wrote: post #10

Besides parallel queries (specifically parallel index scans), the Postgres 10 feature I'm most looking forward to is the work Andres Freund has done on using LLVM to JIT compile expressions[0]. Based on the the layout of the database I work on[1], an insane amount of CPU goes to processing of partial index predicates. If the improvements are anywhere near what is being described in the mailing list, we're going to st…

It should be interesting, but LLVM jit brings in a substantial up front cost. Maybe Andres found a way to mitigate it.

Re: PostgreSQL Parallel Query v2

#17

Post by qeternity »

malisper wrote: post #14
colanderman wrote: post #12

Earlier quoted context omitted.

Don't forget native logical replication.

Logical replication doesn't matter as much to us. We are using CitusDB (in other words sharded Postgres), and we have our own code for handling replication.

Citus should already be saturating i/o during a scan unless you're only hitting one shard. What am I missing about your use case?

Re: PostgreSQL Parallel Query v2

#18

Post by temp246810 »

zzzzzzzza wrote: post #15
hendzen wrote: post #13

Earlier quoted context omitted.

RedHat Q4 FY2016 profit: $68M Oracle Q4 FY2016 profit: $2B

will oracle be around in 10-20 years? will red hat? which will experience growth?

All things being equal it would take red hat ~30 quarters to make 2bn dollars, or roughly 7 years.

7 years to make what Oracle makes in a quarter.

I don't really have a dog in this fight but just FYI I think you missed ops point.

Re: PostgreSQL Parallel Query v2

#19

Post by throwaway91111 »

qeternity wrote: post #17
malisper wrote: post #14

Earlier quoted context omitted.

Logical replication doesn't matter as much to us. We are using CitusDB (in other words sharded Postgres), and we have our own code for handling replication.

Citus should already be saturating i/o during a scan unless you're only hitting one shard. What am I missing about your use case?

Doesn't this inherently depend on the query? Shard-local i/o saturation affects the latency of the entire query if you're not expecting the i/o to the client to saturate.

Re: PostgreSQL Parallel Query v2

#20

Post by jordanthoms »

fnord123 wrote: post #16
malisper wrote: post #10

Besides parallel queries (specifically parallel index scans), the Postgres 10 feature I'm most looking forward to is the work Andres Freund has done on using LLVM to JIT compile expressions[0]. Based on the the layout of the database I work on[1], an insane amount of CPU goes to processing of partial index predicates. If the improvements are anywhere near what is being described in the mailing list, we're going to st…

It should be interesting, but LLVM jit brings in a substantial up front cost. Maybe Andres found a way to mitigate it.

Is it significant when you are talking about​ analytical queries which are running for minutes or hours? Seems like it wouldn't be a huge amount of code to JIT compared to the payoff?
Post reply on HN