An 8GB dataset? That’s literally a few seconds worth of records generated in my world, and any speedup at that scale is completely meaningless.
Let’s talk when you are looking at double digit TB at a minimum.
Training a 4B model to produce 81% faster query plans than Postgres
91–100 of 124 posts
Re: Training a 4B model to produce 81% faster query plans than Postgres
#92[dead]
Re: Training a 4B model to produce 81% faster query plans than Postgres
#93TL:DR; for people. Index your data properly.
It’s 8G of data in memory. You can raw dog that no problem.
(Indexes would be nice though)
Re: Training a 4B model to produce 81% faster query plans than Postgres
#94I have a theory that soon enough every code library will ship a CLI and a very tiny finetune for that specific lib alongside it
Re: Training a 4B model to produce 81% faster query plans than Postgres
#95[dead]
Re: Training a 4B model to produce 81% faster query plans than Postgres
#96Really creative use of distillation here.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#97[flagged]
Re: Training a 4B model to produce 81% faster query plans than Postgres
#98“81% faster query plans than Postgres”…on an 8 GB dataset that fits entirely in memory, with shared_buffers constrained to a fraction of that, queries warmed before measuring, and read-only SELECTs. I would be cautious about over fitting, it’s tough to say if those query plans would really be more optimal than Postgres heuristics at scale and with a bit more realistic OLTP workloads. In any case, such is life with pr…
With a 4B parameter model that probably ran through 8GBs of RAM multiple times to run. At a certain point we should seriously talk about CUDA accelerating Postgres instead.
What would you accelerate? Is there a lot of linear algebra you could throw cuda at in Postgres?
Re: Training a 4B model to produce 81% faster query plans than Postgres
#99Earlier quoted context omitted.
The immediate problem: How do you know which one is better without running them?
Could you A/B at random, use that to collect data and eventually feed that back in to prefer A or B depending on the shape of the query?
Customers love it when their queries sometimes run a lot longer.
Re: Training a 4B model to produce 81% faster query plans than Postgres
#100Engineer: "HELP, our production DB is frozen on this query that worked fine before!" Infra: "Hmm, let's check... Well would you look at that, it seems like your LLM query planner usually works and produces fast queries, but this time when you changed a variable name to trigger query rebuild, it happened to hallucinate and miss an index, would you mind re-running the LLM a few times until you get a faster query?"
I’ve seen this happen to SQL Server many times. Every time the solution is a stored proc with the recompile option enabled.
The answer to every problem is to rebuild statistics. (And never use stored procedures. It's just a shitty API layer in the worst language imaginable, sitting outside of source control. If you need an API layer, write it in a real language, ideally the one you're already using.)