Aren't optimizations suppose to be deterministic?
They are not. Choice among several query plans depends on various summary statistics about the data, which might not be the most recent.
Training a 4B model to produce 81% faster query plans than Postgres
11–20 of 118 posts
Re: Training a 4B model to produce 81% faster query plans than Postgres
#12> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere
Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#13Aren't optimizations suppose to be deterministic?
They are not. Choice among several query plans depends on various summary statistics about the data, which might not be the most recent.
Including the input parameters.
It's not unusual for us to end up with bad query plans because the shape of our data can vary pretty greatly. In many cases, a Foo has 1 Bar. But in some cases, a Foo has a million Bars. That can cause the query optimizer to treat lookups on the bar table as if there are few elements there (causing a scan instead of a seek).
For the general case, the optimizer gets it right. However, the fringe case is one that causes the entire system to crash. It's a bit akin to how an insertion sort can be faster than quick sort when n is small. The optimizer might make a bad assumption about the size of n which makes it pick an expensive n lookup when log(n) is available (but slower for small n).
Re: Training a 4B model to produce 81% faster query plans than Postgres
#14Aren't optimizations suppose to be deterministic?
I'd like to contribute my amateur hour entry into this thread, although I did administer and develop mssql stuff for awhile.
sure optimizations based on stats, but the stats are the wildcard, in my experience query plans can change suddenly.
Queries are translated into plans according to statistics. However the transforms will be deterministic and should only change one valid plan to another. I could very easily see a neural network manipulate transforms the same way the current programming does, its just that the neural networks are by nature really nicely suitable because the "decisions" are based on training, and this training can be closed world type things like the ai assists that chess engines are now getting. Obviously ai still can't play chess but apparently its very good at ranking board positions just by developing that much statistical info because its training comes not from reading the web, but playing a gazzilian games against itself in a "closed" chess world of its own.
I'm thinking that the ai does "this legal transform of the query plan should be applied to this pattern of data (statistics, cardinality, etc)" simply because the ai encountered it in closed world training, much like the chess thing.
Just a theory tho feel free to correct!
Re: Training a 4B model to produce 81% faster query plans than Postgres
#15> I paid ~$800 to rent a 2x H100 SXM node from Lambda for ~95 hours, and ~$400 in OpenAI API fees to generate the Astra trajectory demonstrations. > a tiny 4B model went from not being able to understand the harness it was wrapped in, to achieving a 1.81x geometric mean speedup and a summed latency decrease of 44.7% across a workload of join-heavy SQL queries I can’t find it in the article (may have skimmed it too mu…
> ⇒ this may be useful for some workloads, but even then, can you afford to spend hours every now and then to update your 4B model to ensure it still picks a good query plan?
I think this would be likely comparable to a scheduled backup, so I think it would be an acceptable maintenance window. However, deterministic algorithms would likely beat re-training (or re-fine-tuning) the model. For example, one could analyze actual distributions or whatever (instead of assuming uniform), and then some plans would automatically be eliminated.
Imo a good thought experiment is to look at places that are hyper-optimized, like compilers. Would LLMs bring anything to the table (architecturally or performance-wise) to a piece of software that has been carefully crafted for decades? (Methinks no.)
Re: Training a 4B model to produce 81% faster query plans than Postgres
#16> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.
I don't think anybody's denying that small models are doing distillation, the issue at hand is frontier lab vs frontier lab when it comes to big models.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#17> Frontier intelligence is extremely powerful; the distillation I did off Astra trajectories is proof enough that large models are not going anywhere Wouldn't admitting this invite trouble due to accusations of distillation flying around between closed and open models.
Frontier model trainers stole almost all the data they've trained on (the whole internet, all copyrighted).
It's very hard for them to claim the moral high ground here.
It's like stealing an apple from the British Colonial Empire.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#18why is this write-up so long?
Need 5 days just to go through it.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#19But how will you know that the query plan actually does what your query asked for?
Because P!=NP (very probably IMHO) there's a huge class of problem for which LLMs are useful on the expensive and heuristic-y generation side because the verification is relatively inexpensive.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#20why is this write-up so long? Need 5 days just to go through it.
They forgot to run a (same) model to optimize article for reading speed