Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andy Pavlo joins ClickHouse to establish ClickHouse Labs

clickhouse.com

Post reply on HN
Print view View original

51–60 of 82 posts

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#51

Post by shimman »

AbbeFaria wrote: post #29
apavlo wrote: post #19

Earlier quoted context omitted.

They will continue. New seminar series starts next month (announcement coming this week).

I “audited” CMU 15-445 in April 2026, a few months ago. I then applied to Azure Hyperscale DB team, and as my pitch to get hired I cold DM’d the hiring manager whatever I finished in the assignments (BPM, Concurrent B-tree, Transaction support etc). With that, I was able to get the interview although I ended up failing it. My lack of experience in C++ was probably one of the main reasons. Your course helped me stand…

Why did you think it was appropriate to nag the hiring manager while you finished a MOOC?

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#52

Post by strbean »

shimman wrote: post #51
AbbeFaria wrote: post #29

Earlier quoted context omitted.

I “audited” CMU 15-445 in April 2026, a few months ago. I then applied to Azure Hyperscale DB team, and as my pitch to get hired I cold DM’d the hiring manager whatever I finished in the assignments (BPM, Concurrent B-tree, Transaction support etc). With that, I was able to get the interview although I ended up failing it. My lack of experience in C++ was probably one of the main reasons. Your course helped me stand…

Why did you think it was appropriate to nag the hiring manager while you finished a MOOC?

I read it as "After finishing the course, I DM'd the hiring manager with a list of the stuff I did for the course."

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#53

Post by pdhborges »

jandrewrogers wrote: post #46
bedman12345 wrote: post #24

Earlier quoted context omitted.

Could you explain what practical research there is to be done? The heavy theory I know does not seem to be very useful in practice. Optimal join algorithms, Yannakakis adjacent algorithms, tree decomposition of queries all seem to be worse than well implemented naive algorithms. But maybe the implementations of the new algorithms just are not good? I really don’t know.

There are several interesting foundational problems around scalability, generality, efficiency, and adaptability. Historically use cases haven't always been there to justify focusing on these problems but that is changing. In my opinion, some of the more important open problems: Ideally a table should be index-organized across all relevant columns. No public system works anything like this. We don't have single index…

To be honest none of these problems feels remotely "solvable" beyond having a ton of knobs and engineering tradeoffs like we already have now.

In a sense they don't seem very exciting because all of this required generality.

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#55

Post by hodgesrm »

jimmyl02 wrote: post #36
a34729t wrote: post #28

I'm very curious about the convergence of the best in class fast OLAP products (StarRocks, ClickHouse) with Trino. It sounds like everybody is going for decoupled compute/storage, using S3 or similar as the storage layer, and thus forgoing colocated joins (ok I know ClickHouse joins suck)... So what does this mean for ingestion (and indexing)? Iceberg V3? Paimon? Bespoke ingestion through the DB engine to do the inde…

It's also interesting how Clickhouse / Starrocks can now also act as a query planner and executor on top of non-native formats (ex. Iceberg). I assume the native formats will always be faster / more optimized but the need for Trino as a separate executor while running either of these databases seems to be close to gone.

> I assume the native formats will always be faster / more optimized

Actually query on Iceberg is end-to-end faster on ClickHouse than MergeTree native format on disk. It's mostly a matter of how much compute you throw at it. Native MergeTree still wins in cases that depend on use of indexes to reduce I/O but scan speed is no longer an issue.

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#56

Post by kirugan »

andriy_koval wrote: post #26
jandrewrogers wrote: post #18

Earlier quoted context omitted.

The databases we use today in production have severe limitations and are not even close to what is theoretically possible. Many traditional parts of a database (indexing, caching, scheduling, et al) are AI-complete algorithm problems. Entire sub-classes of database (e.g. graph or spatial) famously have persistently poor scalability and performance because of open questions in the foundational computer science. Just t…

I personally built quite several specialized hyper-performant DB engines, and studied most popular OSS projects, and believe most DB questions are theoretically answered long ago(decade back). The puzzle is mostly to assemble pieces together to fit specific tradeoff of performance/simplicity/functionality and not overengineer system.

What is the best resource for finding good academic papers on databases that are actually practical to implement?

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#58

Post by astrange »

bijowo1676 wrote: post #23
astrange wrote: post #14

> ClickHouse had features that at the time were only found in a handful of closed-source, commercial analytical DBMSs. For example, ClickHouse was written in C++ and supported vectorized query execution using SIMD in 2016. Most prominent open-source analytical DBMSs in 2016 were JVM-based and did not support SIMD optimizations until years later. Performance is a feature. "Written in C++" is a strange idea of a featur…

zero cost abstraction is one of the many features of C++, its just you are not familiar with the language

I am familiar with C++.

Re: Andy Pavlo joins ClickHouse to establish ClickHouse Labs

#60

Post by dgacmu »

bedman12345 wrote: post #24
jandrewrogers wrote: post #18

Earlier quoted context omitted.

The databases we use today in production have severe limitations and are not even close to what is theoretically possible. Many traditional parts of a database (indexing, caching, scheduling, et al) are AI-complete algorithm problems. Entire sub-classes of database (e.g. graph or spatial) famously have persistently poor scalability and performance because of open questions in the foundational computer science. Just t…

Could you explain what practical research there is to be done? The heavy theory I know does not seem to be very useful in practice. Optimal join algorithms, Yannakakis adjacent algorithms, tree decomposition of queries all seem to be worse than well implemented naive algorithms. But maybe the implementations of the new algorithms just are not good? I really don’t know.

There's been a lot of fun, practical work recently on optimal join algorithms! (Full disclosure, this advisor on this paper is one of Andy and my former students, so I'm slightly biased)

RPT: https://people.iiis.tsinghua.edu.cn/~huanchen/publications/r...

Like most academic work, this one builds on some work that's been done over the last few years on ways to make the Yannakakis algorithm actually practical.

Post reply on HN