Getting AI to write good SQL

Post by lmeyerov » Sat, May 17, 2025, 2:59 PM UTC

blibble wrote: post #268

lmeyerov wrote: post #262

Earlier quoted context omitted.

I can bring data here: We recently did the first speed run where Louie.ai beat teams of professional cybersecurity analysts in an open competition, Splunk's annual Boss of the SOC. Think writing queries, wrangling Python, and scanning through 100+ log sources to answer frustratingly sloppy database questions: - We get 100% correct for basic stuff in the first half that takes most people 5-15 minutes per question, and…

is a competition/speed run a realistic example?

Post by ses1984 » Sat, May 17, 2025, 3:11 PM UTC

Spivak wrote: post #226

tclancy wrote: post #112

Earlier quoted context omitted.

Mother of God. I can write JSON instead of a language designed for querying. What is the advantage? If I’m going to move up an abstraction layer, why not give me natural language? Lots of things turn a limited natural language grammar into SQL for you. What is JSON going to {do: for: {me}}?

I find it funny people are making fun of this while every ORM builds up an object representing the query and then compiles it to SQL. SQL but as a data structure you can manipulate has thousands of implementations because it solves a real problem. This time it's because LLMs have an easier time outputting complex JSON than SQL itself.

Post by [deleted] » Sat, May 17, 2025, 3:13 PM UTC

theropost wrote: post #239

wewewedxfgdf wrote: post #42

Earlier quoted context omitted.

It is absolutely the greatest golden age in programming ever - all these infinitely wealthy companies spending bajillions competing on who can make the best programming companion. Apart from the apologising. It's silly when the AI apologises with ever more sincere apologies. There should be no apologies from AIs.

I wish my AI would tell me when I'm going in the wrong direction, instead of just placating my stupid request over and over until I realize.. even though it probably could have suggested a smarter direction, but instead just told me "Great idea! "

Post by [deleted] » Sat, May 17, 2025, 3:14 PM UTC

wewewedxfgdf wrote: post #26

Can I just say that Google AI Studio with latest Gemini is stunningly, amazingly, game changingly impressive. It leaves Claude and ChatGPT's coding looking like they are from a different century. It's hard to believe these changes are coming in factors of weeks and months. Last month i could not believe how good Claude is. Today I'm not sure how I could continue programming without Google Gemini in my toolkit. Gemini…

Post by sgarland » Sat, May 17, 2025, 3:16 PM UTC

mritchie712 wrote: post #211

ljm wrote: post #13

Earlier quoted context omitted.

> you get the added benefit of writing queries in JSON instead of raw SQL. I’m sorry, I can’t. The tail is wagging the dog. dang, can you delete my account and scrub my history? I’m serious.

LLMs are far more reliable at producing something like this: { "dimensions": [ "users.state", "users.city", "orders.status" ], "measures": [ "orders.count" ], "filters": [ { "member": "users.state", "operator": "notEquals", "values": ["us-wa"] } ], "timeDimensions": [ { "dimension": "orders.created_at", "dateRange": ["2020-01-01", "2021-01-01"] } ], "limit": 10 } than this: SELECT users.state, users.city, orders.stat…

    SELECT
      users.state,
      users.city,
      orders.status,
      COUNT(*)
    FROM users
    JOIN orders ON user.id = orders.user_id
    WHERE
      users.state != 'us-wa' AND
      orders.created_at BETWEEN '2020-01-01' AND '2021-01-01'
    GROUP BY 1, 2, 3
    LIMIT 10;