Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training a 4B model to produce 81% faster query plans than Postgres

rohanbansal.com

Post reply on HN
Print view View original

131–136 of 136 posts

Re: Training a 4B model to produce 81% faster query plans than Postgres

#131

Post by __alexs »

withinboredom wrote: post #114
dragontamer wrote: post #102

Earlier quoted context omitted.

You know that GPUs are more flexible than just linear algebra, right? GPUs are simply faster at fundamental algorithms like sorting (which has huge parallelism), and hashing. This is because both sorting and hashing benefit from endless growth of parallelism, offering enough "work" for these 10,000 SIMD-core systems to crunch work upon. And because of modern algorithms/libraries with 'Mergepath sort' (a GPU-SIMD para…

The GPU isn’t connected to the disk though. Usually. So you’d still have to load from disk, to ram, then from ram to the GPU.

PCI-E is really very flexible https://developer.nvidia.com/gpudirect

Re: Training a 4B model to produce 81% faster query plans than Postgres

#132

Post by zacmps »

I am skeptical of these results given that the end has:

> Favorite settings The model regularly used enable_sort=off and random_page_cost=1.1

If random_page_cost wasn't set correctly for the default cases postgres's query planner can generate terrible plans (unless you're running on a spinning disk).

That could easily explain the difference by itself.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#133

Post by williamdclt »

soerxpso wrote: post #58
dragontamer wrote: post #56

Earlier quoted context omitted.

With a 4B parameter model that probably ran through 8GBs of RAM multiple times to run. At a certain point we should seriously talk about CUDA accelerating Postgres instead.

I would think it's possible to make it so that the 4B model only needs to be called during an initial phase, and then the same queries it constructed can just be re-used with values replaced, unless you're generating a lot of unique on-the-fly query shapes.

Postgres takes the actual values into account when generating a query plan. The same query with different params can (and should) result in different query plans. It looks at statistics on the actual data stored.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#134

Post by karambahh »

HackerThemAll wrote: post #130

Two things. One is that to remind folks that PostgreSQL has used a tiny form of "artificial intelligence", that is GEQO - Genetic Query Optimizer, since 2001. Second. How would that LLM-based query optimizer work in a real-world 10,000 qps ERP system with very large shape of queries? I'm not saying it's useless, it just won't replace a real query planner soon. Latencies would skyrocket.

I'm also quite skeptical of performance/determinism over a real world load but if I'm not mistaken, a 10 000 qps with large shape of queries would not benefit much either from genetic algos, would it?

Re: Training a 4B model to produce 81% faster query plans than Postgres

#136

Post by tempfile »

egeozcan wrote: post #109
Tanjreeve wrote: post #108

Earlier quoted context omitted.

That bug is fixable and verifiable. The LLM you cross your fingers till the next time the same thing happens.

> fixable and verifiable By people with a specific skill set. LLMs generation can also be fixed and verified by people with a certain skill set, and non-deterministic computing doesn't automatically mean unpredictable. When people say that the LLMs are a black box, it means unpredictability in unknown situations. You do structured output, input validation, output validation, lower temperature, limit decisions, RL, et…

None of the things you mention are guaranteed to increase the probability of correctness. You can run the LLM output through as many deterministic programs as you like, but "the query plan runs in acceptable time" is not something you can verify with such a tool. Nobody knows how the LLM does it, so they cannot know how to make the LLM do it better.
Post reply on HN