Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training a 4B model to produce 81% faster query plans than Postgres

rohanbansal.com

Post reply on HN
Print view View original

31–40 of 124 posts

Re: Training a 4B model to produce 81% faster query plans than Postgres

#31

Post by btown »

dvt wrote: post #15
Someone wrote: post #10

> I paid ~$800 to rent a 2x H100 SXM node from Lambda for ~95 hours, and ~$400 in OpenAI API fees to generate the Astra trajectory demonstrations. > a tiny 4B model went from not being able to understand the harness it was wrapped in, to achieving a 1.81x geometric mean speedup and a summed latency decrease of 44.7% across a workload of join-heavy SQL queries I can’t find it in the article (may have skimmed it too mu…

> ⇒ this may be useful for some workloads, but even then, can you afford to spend hours every now and then to update your 4B model to ensure it still picks a good query plan? I think this would be likely comparable to a scheduled backup, so I think it would be an acceptable maintenance window. However, deterministic algorithms would likely beat re-training (or re-fine-tuning) the model. For example, one could analyze…

The Postgres query planner has had to operate, for those same decades, in a much more realtime-sensitive and restricted environment than compilers. It can only draw its conclusions from summary statistics on tables in isolation, not on their relationships with each other (and even less so when filters are involved). For many cases this is fine! For many others it isn't.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#32

Post by hamilyon2 »

Optimal plan construction is math-heavy, algorithm-heavy and vary even by workload. There are options like creating just-in-time indexes, so solution space grows even faster than article presents. Sometimes it is the query planner which is the slow part of total execution time.

LLM is kind of blunt weapon to use here. I am waiting rather for alphago style neural net heuristic.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#34

Post by Muromec »

kelvinjps10 wrote: post #25
p1necone wrote: post #22

Earlier quoted context omitted.

Having the moral high ground matters less than having a big warchest of money to spend on lawyers.

Does it matter if they companies doing are not in the jurisdiction or even if they are, maybe the can't prove it?

They can't prove it but can force you to spend time and money to disprove it.

Re: Training a 4B model to produce 81% faster query plans than Postgres

#35

Post by krisoft »

cowboylowrez wrote: post #14
kingjimmy wrote: post #8

Aren't optimizations suppose to be deterministic?

I'd like to contribute my amateur hour entry into this thread, although I did administer and develop mssql stuff for awhile. sure optimizations based on stats, but the stats are the wildcard, in my experience query plans can change suddenly. Queries are translated into plans according to statistics. However the transforms will be deterministic and should only change one valid plan to another. I could very easily see…

> Obviously ai still can't play chess

I believe you are wrong on that. Do you mean large language models can’t play chess?

Re: Training a 4B model to produce 81% faster query plans than Postgres

#36

Post by pyaamb »

devsda wrote: post #12

> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.

On the flip side, its important to know just how much we would be sacrificing if big frontier gets their way in convincing the courts that distillation is a bad thing

Re: Training a 4B model to produce 81% faster query plans than Postgres

#37

Post by jwpapi »

foota wrote: post #24

Funny enough I was thinking about something very similar to this based on the Jev model posted yesterday.

I’ve played with it already. I don’t think this is the use case. I think Jev’s use case is fast, cheap and somewhat easy classification. It’s not trainable in the way you would want here. Even though it’s fast it wont be faster than pgs query optimizer.

At least as I understand things.

How did you plan to use Jev for query optimization?

Re: Training a 4B model to produce 81% faster query plans than Postgres

#39

Post by cowboylowrez »

krisoft wrote: post #35
cowboylowrez wrote: post #14

Earlier quoted context omitted.

I'd like to contribute my amateur hour entry into this thread, although I did administer and develop mssql stuff for awhile. sure optimizations based on stats, but the stats are the wildcard, in my experience query plans can change suddenly. Queries are translated into plans according to statistics. However the transforms will be deterministic and should only change one valid plan to another. I could very easily see…

> Obviously ai still can't play chess I believe you are wrong on that. Do you mean large language models can’t play chess?

I don't believe that a pure neural network can currently abide by the rules of chess, even with unreasonable amounts of training. Do you have a counter example? I never mind having beliefs challenged with facts lol

edit: I think you could provide an AI with a service or skill that asks "is this move legal" but given all the overhead for llms or whatever to call a "legal move" service external to its process, well then you aren't really searching the tree very efficiently lol.

However if you just let a neural network score boards and the neural network is in the same process well then I think thats the working solution for using neural networks in chess. The net does not need to score all boards either, simple value based heuristics can obviously provide a preliminary list of good boards (moves) at a certain depth or ply and then select the move that produces the board that the neural net scores highest. I kinda sorta think thats whats done today but as usual I could be full of it lol

Re: Training a 4B model to produce 81% faster query plans than Postgres

#40

Post by justonenote »

make3 wrote: post #17
devsda wrote: post #12

> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.

Frontier model trainers stole almost all the data they've trained on (the whole internet, all copyrighted). It's very hard for them to claim the moral high ground here. It's like stealing an apple from the British Colonial Empire.

You'd be shocked at some of the punishments handed out from various empires over history for offences as minor as stealing an apple. Whether they can claim the moral high ground is not question of ethics but a question of power.
Post reply on HN