Live data from Hacker News

PostgreSQL Magic

goto.project-a.com

41–50 of 67 posts

Re: PostgreSQL Magic

#41
post #37
post #29

Earlier quoted context omitted.

I don't disagree in the general case, but I'm particularly interested in window functions here, as they are part of the SQL standard and seem equivalent between MS SQL Server and Postgres (I don't have near enough experience in other RDBMS's to speak to their implementations), despite being called out with some regularity as a Postgres-specific nicety.

The only one I know of is that PostgreSQL has some nice ordinary aggregate functions which can be used well with window functions, like bool_and() and array_agg(). I can't think of an example on top of my head where it is useful but I know I have used them both.

Thanks. There are definitely features in Postgres that I'm jealous of in the MS SQL Server world, and you've listed a couple of them.

Re: PostgreSQL Magic

#42
post #4

I have to admit that I'm still not quite sure about arrays in relational databases. Don't get me wrong, I use them all the time, but it kinda feels like when you've got tables that you just know could be normalized more thoroughly. Also: If someone has a good version-control wrapper for stored procedures, that would be swell. And while I'm doing the wishful thinking shtick, maybe a Coffeescript-like preprocessor and…

They are incredibly useful in stored procedures. Also at indexes, pg's full text search is completely reliant on array indexes, ditto for the json storage. I think I've never used them as column type, but I can imagine a few uses there too.

> I think I've never used them as column type, but I can imagine a few uses there too.

A lot of cases where you would use a one:many are useful to store in an array instead. Tags would be a good example, multi-select lists, etc.

Re: PostgreSQL Magic

#43
post #22
post #2

As a fellow Postgres amateur wizard, love the positive attention that postgres seems to be getting more and more, and some half databases less and less (unless you actually need map-reduce, ofcourse. You probably don't. /trollface) What I miss though in this article, and where I think postgres shines majorly compared to other rel dbs, are window functions. It allows you to apply a partition to a set. You can do some…

Window functions are not a Postgres thing, but part of the SQL standard with (varying, of course) support across most of the major RDBMS's. I've seen comments like this several times that seem to imply that Postgres is exceptional either due to having window functions, or like in this one where the tone sounds as if Postgres does them particularly better. I'm curious, as I do almost 100% of my production work in MS S…

This may be related to how long Postgres has had support for various window functions, vs. MSSQL only gaining equivalent support in v2012 (AFAIK). Also, the various query optimization struggles older versions of MSSQL had.

http://sqlperformance.com/2013/03/t-sql-queries/the-problem-...

Re: PostgreSQL Magic

#44
post #6
post #4

I have to admit that I'm still not quite sure about arrays in relational databases. Don't get me wrong, I use them all the time, but it kinda feels like when you've got tables that you just know could be normalized more thoroughly. Also: If someone has a good version-control wrapper for stored procedures, that would be swell. And while I'm doing the wishful thinking shtick, maybe a Coffeescript-like preprocessor and…

We use them a lot where the join table could be over a million rows. Saves a ton of performance and a join. Were talking seconds here on a 3-4s query before, ~1s after.

Yes, it can make a huge difference. I did this once when I had arrays of ~1 million floats and had to compute statistics on them. I wound up implementing a bunch of stats functions as C stored procedures that operate on arrays, and brought query times down from ~12 seconds to ~20 milliseconds:

https://github.com/pjungwir/aggs_for_arrays/

Another time arrays are handy is when you don't know how many "columns" you need to return. SQL can't do this, but a variable-length array can. Here is a writeup for one time that came in handy:

http://illuminatedcomputing.com/posts/2013/03/fun-postgres-p...

Another time they are helpful is to throw an `array_agg` into an aggregate query to see what values are getting rolled up. This can be really useful if you're trying to debug weird behavior.

Also `(array_agg(...))[1]` is a poor-man's `first` function. :-)

Re: PostgreSQL Magic

#45
I have seen magic done with custom types and aggregates. For example, I know of projects with run-length-encoded bitsets and custom aggregates for doing set operations and counts, sort of like Redis' bitset commands but built into PG. This is a massive space optimization, because otherwise you'd just have a join table with zillions of tiny rows.

Re: PostgreSQL Magic

#47
post #11

Earlier quoted context omitted.

Isn't a CTE in Postgres (unlike in MS SQL, AFAIR) also an optimization fence? Just something to keep in mind when using it as a substitute for subquery, readability vs performance and all that :)

Yes, at present, the query executor can't optimize across CTEs. Sometimes, that's even the behavior you want.

> Sometimes, that's even the behavior you want.

Sadly, in most cases, this mean you have to decide between ugly and performant, or nice and slow code.

I hope this gets fixed soon. Nobody should have to write queries like this:

      select blah blah blah
      from x, (select blah blah blah
        from y, (select blah blah blah 
          from z, (select blah blah
            from w
            where a=b
            and c=d)
          where z.id = w.id
          and p = 2
          and q = 4)
        where z.id = y.different_id
        and r = 3
        and t = 'BLAH'
        and u not in (select u from w)) l
      where l.id = x.id
CTEs allow you to build "lisp-like" pipeline where you transform your data as you go and are able to give the intermediate results useful names.

Re: PostgreSQL Magic

#48
So many amazing features that are incredibly relevant to modern web development. I've switched over a personal project from MySQL to PG recently and this time I'm not looking back.

On 9.4 I have the HSTORE and JSONB types as well as range types which are incredibly useful. If you have a solid language library wrapper for PG you can spend far less time mangling data from one format to the next and just get to work. I love it.

Re: PostgreSQL Magic

#49
post #23
post #22

Earlier quoted context omitted.

Window functions are not a Postgres thing, but part of the SQL standard with (varying, of course) support across most of the major RDBMS's. I've seen comments like this several times that seem to imply that Postgres is exceptional either due to having window functions, or like in this one where the tone sounds as if Postgres does them particularly better. I'm curious, as I do almost 100% of my production work in MS S…

I think because Postgres is commonly being considered as an alternative to MySQL rather than MSSQL or Oracle.

I dunno, getting the hell off Oracle to Postgres is fashionable as hell. We're doing it and EVERYTHING IS BETTER.

Re: PostgreSQL Magic

#50
post #47
post #11

Earlier quoted context omitted.

Yes, at present, the query executor can't optimize across CTEs. Sometimes, that's even the behavior you want.

> Sometimes, that's even the behavior you want. Sadly, in most cases, this mean you have to decide between ugly and performant, or nice and slow code. I hope this gets fixed soon. Nobody should have to write queries like this: select blah blah blah from x, (select blah blah blah from y, (select blah blah blah from z, (select blah blah from w where a=b and c=d) where z.id = w.id and p = 2 and q = 4) where z.id = y.dif…

Of course, you can first write the query with CTEs, then convert to the nested form. It's just substitution.

(For bonus points, keep the original CTE form, but commented out, to help with troubleshooting further down the road.)

Post reply on HN