Generating SQL with LLMs for fun and profit

One possible trick you might consider is to (manually!) add objects to the DB's schema that explicitly indicate its environment, e.g. `CREATE TABLE dbo.ThisIsProductionYouSillyPerson ( DummyCol int NOT NULL );` for prod and `CREATE TABLE dbo.ThisIsTestFeelFreeToMessAround ( DummyCol int NOT NULL );` - these tables would be excluded from the automated DB deployment code - and write test scripts that all start by checking that the `dbo.ThisIsTestFeelFreeToMessAround` table exists and that the `dbo.ThisIsProductionYouSillyPerson` table does not exist in the DB before continuing.

DB automation is great for preventing mistakes during common routine operations, but because DB automation can also go haywire and delete drop all-by-itself unless you set-up out-of-band (if that's the right term?) safeguards. Having airgapped dev-test-staging-and-prod won't help you if if you forgot the `WHERE` in an `UPDATE` in a little-used script that the prod automation uses, that testing never discovered (which happens all the time, it's scary).

I do appreciate how MySQL does come with an `UPDATE-without-key` guard, but I'm surprised none of the other RDBMS have safety-guards like that - just a simple `RequireManualConfirmationForMultiRowDml` flag on a table would help.

In my tool I let users generate SQL using ChatGPT: https://www.timestored.com/pulse/tutorial/chatgpt-sql-query-... However it's their own hosted database and I show them the query for them to run beforehand. For anyone interested in pushing this further, the best paper I found was "Evaluating the Text-to-SQL Capabilities of Large Language Models". It examines which prompts work best and was how I decided on sending schema / create details etc. as part of the initial prompt. Since I create the UI and show the schema as a tree, I can generate that part of the prompt without the users involvement.

On a related note, is anyone aware of good resources for using LLMs to generate user analytics queries on the fly where the LLM has schema/domain context?

> 1) You don’t sound too sure about this. Your previous comment sounded like speculation also. Do you actually read these blogs and/or journals?

I have read some of them before. Not Frontiers in Gastroenterology, but I have spent a lot of time reading psychology/psychiatry journals, since they have some personal relevance. Some of my favourite papers are https://link.springer.com/article/10.1007/s40489-016-0085-x and https://www.nature.com/articles/s41398-019-0631-2 and also (not a paper, a letter to the editor) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9054657/

I have in the past read https://dontforgetthebubbles.com/ which is a paediatrics blog–again, has some personal relevance.

I also am very interested in law. I actually applied to law school once, but didn't get in, and gave up on the idea after that. If they'd accepted me, I might have been a lawyer right now rather than a software engineer. Public international law was always an area of particular fascination for me. I remember being at university, and I was supposed to be at my CS lecture, but instead I was in the library reading books like Restatement (Third) of the Foreign Relations Law of the United States and some textbook (I forget its name now) I found on the EU treaties and ECJ case law. So yes, I do read law blogs sometimes. I went through a period when I was reading SCOTUSblog a lot. Not a blog, but I actually enjoy reading stuff like this: https://legal.un.org/ilc/texts/instruments/english/draft_art...

> And now you provided a quote where the law blog is explicitly explaining a very basic thing even to their _qualified target audience_. If “most readers” already know something, then what’s the point of re-explaining it? You’re proving my point instead.

Even that quoted sentence is assuming the reader already knows what "international humanitarian law" and "international human rights law" are, and what is the difference between them. There are also many cases in that post in which (unlike lex specialis) the author uses technical legal terminology without ever explaining it: for example, his repeated invocation of jus ad bellum, or his mention of the "Inter-American System". Another example is where he cites the Vienna Convention on the Law of Treaties, which assumes the reader understands its significance.

> Even if an “unqualified” person were to read a _professional_ law or medical blog/journal, what’s the worst that could happen? Nothing.

For a medical journal – a person reads an article about using drug X to treat condition Y. They then proceed to misdiagnose themselves with condition Y, and then somehow acquire drug X without having been prescribed it, and start taking it. A person could cause quite serious medical harm to themselves in this way. Reading medical journals can also contribute to the development of illness anxiety disorder – you don't need to be a medical student to develop medical student's disease.

For a law journal - a criminal defendant reads something in a law journal and thinks it helps their case. Their lawyer tries to explain to them that they are misunderstanding it and it isn't actually relevant to their situation, but they refuse to listen. They fire their lawyer and then try to argue in court based on that misunderstanding. It is easy to see how they could end up with a significantly worse outcome as a result, maybe even many extra years in prison.

Conversely, our 10 year old sometimes write Python programs. They aren't anything special, but better than I could do at his age. I bet you his Python programs are full of security holes and nasty bugs and bad practices. Who cares, what possible harm could result? And he isn't at the stage yet of reading development blogs, but I've seen him before copying code off random websites he found, so maybe he has stumbled on to one of them. My brother is a (trainee) oncologist, but he did an introductory programming course as an undergrad, and he wrote some Python programs in that too, although he hasn't done any programming in years–what harm could have his programs done? If he started trying to modify the software in one of the radiation therapy machines, I'd be worried (but he's too responsible for that); if he decided to try writing a game in Python for fun, why should anyone worry, no matter what the quality of his code is?