Live data from Hacker News

PostgreSQL Count(*) Performance Improvements

cybertec-postgresql.com

31–35 of 35 posts

Re: PostgreSQL Count(*) Performance Improvements

#31
post #20

Earlier quoted context omitted.

I'm not sure I understand your statement. On databases without an intrinsic row count property the only way to find the row count is to go to every page and count the rows there. The query planner will use the same plan for both `count(*)` and `count(1)`.

Not according to that blurb. There have been other databases where count(1) was much faster. Here they are saying that in Postgres, count(1) is slower because star is treated as no arguments, and count(1) is more complex.

That is not what "they" are saying. count(1) and count(*) are equally fast because they are doing the same thing.

Re: PostgreSQL Count(*) Performance Improvements

#32
post #9

Sorry for being a bit off topic, but anyone out there who is using PostgreSQL in production, how do you manage tables with lots of updates? Is auto-vacuum doing good enough job for you or do you have to run “vacuum full” regularly?

We have busy tables with lots of record churn some which receive 2000 xacts/s. I’ve found that they will never shrink down until you do a full vacuum but regular vacuum is good at reusing pages so tables won’t outgrow their max size during busiest periods. We never run full vacuums because we’ve never found it to be necessary.

Re: PostgreSQL Count(*) Performance Improvements

#33
post #10

Earlier quoted context omitted.

I think the "the opposite is true" is in reference to the assertion that "count(⧆) appears to consult the data for a whole row", not to the proposition that "one might think that count(1) would be faster [than count(⧆)]". count(⧆) just counts the tuples themselves, which is fast; it's like counting heap-allocated data structures by counting their pointers (which you're already walking), without dereferencing those po…

> Postgres's count(1) isn't slower than the one in any other DBMS. count( * ) is faster on Postgres than count(1). But both are fundementally slow because of MVCC. And count( * ) on postgres (the optimized one on Postgres) is much slower than count(1) on other databases (the optimzed one on other databases). So practically speaking, counting rows is slower on Postgres than on other databases. That said, I love Postgr…

Yes, I was trying to make a finer point—the difference between count(⧆) and count(1) comes down to the cost of filtering the row, and in Postgres count(1) is a regular filtering operation—taking the same filtering cost that count(1) has in other DBMSes—while count(⧆) has a filtering cost that is lower than that of other DBMSes, because it has been specifically optimized†.

Separately, there's an MVCC cost of walking the rows to filter them, and other DBMSes optimize walking rows for counting [usually causing both count(1) and count(⧆) to be faster], while Postgres does not do this optimization. (And, as stated in the article, in those DBMSes, this isn't a pure optimization per se, but is rather a trade-off, trading write speed for all INSERTs/DELETEs for read speed for this particular case.)

(† Technically, the filtering cost of count(⧆) hasn't been specifically optimized; the relative speed of filtering tuples for count(⧆) is an emergent property of the general fact that Postgres treats any mention of `⧆` as a reference to the row-tuple object itself. i.e. If `foo` is a table (x int, y text), then in actuality, `foo` is first created as a type [a pg_class] defined as the tuple (int, text); and then the table `foo` is defined as a relation persisting a rowset of `foo`-tuple-typed rows [in est making a table['s triggerable operations] each into a stored procedure with a `foo`-tuple-typed-rowset return type.] Then, the expressions `(SELECT ⧆ FROM foo)`, and `(SELECT f.⧆ FROM foo f)` both evaluate to rowsets type `foo`, which means that Postgres doesn't need to dereference the pointer to each `foo` heap tuple to build those rowsets. It only needs to dereference the pointers when it comes time to actually serialize and emit the row over the wire—which in case of a `count(⧆)` operation, never happens.)

Re: PostgreSQL Count(*) Performance Improvements

#35

I appreciate approaches that acknowledge when “close” is good enough. It would be interesting to graph how far off those numbers are to put a real metric to it.

Only one data point, but I've got a custom search engine for a community that has pretty close to the ideal usage pattern for this: append-only with no update or deletes. A pure count(*) over 6.8M rows on a slow VPS takes 135,015ms and the n_live_tup technique¹ takes 214ms. The latter was 00.009557% high before an analyze and 00.001414% low afterward. Definitely good enough for me.

¹ SELECT n_live_tup FROM pg_stat_all_tables WHERE relname = 'comments';

Post reply on HN