Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training a 4B model to produce 81% faster query plans than Postgres

rohanbansal.com

Post reply on HN
Print view View original

71–80 of 118 posts

Re: Training a 4B model to produce 81% faster query plans than Postgres

#71

Post by galkk »

2001zhaozhao wrote: post #52

Engineer: "HELP, our production DB is frozen on this query that worked fine before!" Infra: "Hmm, let's check... Well would you look at that, it seems like your LLM query planner usually works and produces fast queries, but this time when you changed a variable name to trigger query rebuild, it happened to hallucinate and miss an index, would you mind re-running the LLM a few times until you get a faster query?"

Our database 12.34 was working great, but 12.35 deployment had some optimizer changes that had regression on exact scenario that you have in your statistics. Shit happens, sorry. Use this hint.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#72

Post by bt1a »

dragontamer wrote: post #56
refibrillator wrote: post #29

“81% faster query plans than Postgres”…on an 8 GB dataset that fits entirely in memory, with shared_buffers constrained to a fraction of that, queries warmed before measuring, and read-only SELECTs. I would be cautious about over fitting, it’s tough to say if those query plans would really be more optimal than Postgres heuristics at scale and with a bit more realistic OLTP workloads. In any case, such is life with pr…

With a 4B parameter model that probably ran through 8GBs of RAM multiple times to run. At a certain point we should seriously talk about CUDA accelerating Postgres instead.

pardon but aren't disks usually the bottleneck? im all for CUDA acceleration and CUDA accelerating culture

Re: Training a 4B model to produce 81% faster query plans than Postgres

#73

Post by mistrial9 »

Someone wrote: post #10

> I paid ~$800 to rent a 2x H100 SXM node from Lambda for ~95 hours, and ~$400 in OpenAI API fees to generate the Astra trajectory demonstrations. > a tiny 4B model went from not being able to understand the harness it was wrapped in, to achieving a 1.81x geometric mean speedup and a summed latency decrease of 44.7% across a workload of join-heavy SQL queries I can’t find it in the article (may have skimmed it too mu…

the currency of the counts also, right.. actually recounting table contents is done from time to time

Re: Training a 4B model to produce 81% faster query plans than Postgres

#76

Post by perrygeo »

Nice article about how to train/fine-tune a language model.

However, it misses the whole point of database query planning. You can't just ignore the planning time itself, as if the database query were a static entity to be optimized once at a leisurely pace.

The real constraint on live query planners is quite different: they must improve the combined time - planning + query - based on live database statistics. You can amortize the planning with prepared statements, but that too is fraught since optimal plans can change quite frequently and based on input parameters. "Live" and "faster than the queries themselves" are the hard requirements to be considered a viable database query planner. This project does neither.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#77

Post by setr »

soerxpso wrote: post #58
dragontamer wrote: post #56

Earlier quoted context omitted.

With a 4B parameter model that probably ran through 8GBs of RAM multiple times to run. At a certain point we should seriously talk about CUDA accelerating Postgres instead.

I would think it's possible to make it so that the 4B model only needs to be called during an initial phase, and then the same queries it constructed can just be re-used with values replaced, unless you're generating a lot of unique on-the-fly query shapes.

With query hints finally being added it’d probably be doable as an extension

Re: Training a 4B model to produce 81% faster query plans than Postgres

#79

Post by abirch »

globalnode wrote: post #44
devsda wrote: post #12

> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.

Yeah don't steal my stolen stuff.

Like this scene from Pirates of Silicon Valley between Bill Gates and Steve Jobs: https://www.youtube.com/watch?v=CBri-xgYvHQ Bill was saying that Steve was stealing from Xerox so it's okay if Bill stole from Xerox first.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#80

Post by voganmother42 »

bt1a wrote: post #72
dragontamer wrote: post #56

Earlier quoted context omitted.

With a 4B parameter model that probably ran through 8GBs of RAM multiple times to run. At a certain point we should seriously talk about CUDA accelerating Postgres instead.

pardon but aren't disks usually the bottleneck? im all for CUDA acceleration and CUDA accelerating culture

I remember projects like PG-Strom back in the day, very cool stuff
Post reply on HN