How PostgreSQL aggregation works and how it inspired our hyperfunctions’ design • Hacker News Board

Re: How PostgreSQL aggregation works and how it inspired our hyperfunctions’ design

#31

Post by sharps_xp » Fri, Aug 06, 2021, 4:13 PM UTC

djk447 wrote: post #18

codeismath wrote: post #11

The continuous aggregates portion of this blog (along with the breakdown of Transition, Combine, and Final Functions) reminded me of "A Theory of Changes for Higher-Order Languages: Incrementalizing Lambda-Calculi by Static Differentiation" (Giarrusso et. al.) [0]. Particularly the part of getting logically-consistent results via different routes of computation. David Kohn says this in the blog post: "But, you have t…

(NB: Post author here) Glad you liked the GIFs! Will have to take a look at this. Thanks for sharing!

I also like the GIFs, which tools did you use to make these?

Re: How PostgreSQL aggregation works and how it inspired our hyperfunctions’ design

#32

Post by nlehuen » Sat, Aug 07, 2021, 9:59 PM UTC

djk447 wrote: post #23

nlehuen wrote: post #22

You can see the same pattern of two-step aggregation with the HLL_COUNT family of functions in BigQuery: https://cloud.google.com/bigquery/docs/reference/standard-sq... This is really useful for this kind of metric (distinct count) that can't be trivially aggregated. It allows generating pre-aggregated tables containing the intermediate aggregation state. Now you can compute your DAUs, WAUs and MAUs from the same dai…

Absolutely! We're actually developing a lot of that: https://github.com/timescale/timescaledb-toolkit/tree/main/d... A number of the things you're looking for we've done experimentally and we'll be stabilizing over the next few releases. So we'd love some feedback while we're still able to futz with the API without making breaking changes. But the two you're asking about are, I think, going to be covered by hyperlogl…

The stats_agg + extractor function pattern is exactly what I was thinking about in my last paragraph :-). Very cool!

Re: How PostgreSQL aggregation works and how it inspired our hyperfunctions’ design

#33

Post by LoriP » Mon, Aug 09, 2021, 1:31 PM UTC

sharps_xp wrote: post #31

djk447 wrote: post #18

Earlier quoted context omitted.

(NB: Post author here) Glad you liked the GIFs! Will have to take a look at this. Thanks for sharing!

I also like the GIFs, which tools did you use to make these?

The GIFs were made with Adobe XD