It's not empowering to hand off the details

AI is the new "excel spreadsheet with ridiculous formulas on top of VB script on top of pivot tables" for the non-programmers. Yes, the AI might not do it efficiently or even always give the right answers, but the spreadsheets didn't either. For non-programmers, the computer has always been a black box, data goes in, data comes out. Usually it's right, sometimes it's wrong and the users mostly had no idea why or how. But now they don't need the middleware app. They don't need to take their data and split it up into very specially structured files, and put just the right pieces of information on just the right lines and in the right spaces. They don't need to remember to do "(foo and (bar or baz))" to get the right answer, as opposed to "(foo and bar or baz)" and get the wrong one. AI for non-programmers is the closest we've ever come to "natural language programming" for people to use. It doesn't matter that it gets things wrong sometimes, or does it inefficiently or makes something unmaintainable unless you use the AI again. What matters to them is they can say "take all this data here and swizzle the frobs" in their native language instead of learning a specific programming syntax and system. And they can point it at some arbitrary data in whatever format they happen to have the data in, and they have a halfway decent chance of getting something that does what they want.

It's not going to be pretty, it's not going to be perfect and it's probably going to break in subtle and unexpected ways. But as long as most of the time its mostly good enough, people will still use it enthusiastically. We're effectively looking at "worse is better" for the modern age and for people that aren't and never were going to be programmers.

Now, I'm back to rubber-ducking until I've built my own mental model over a few days of thinking.

I don't think that's the case and it's becoming more obvious every day. From people who cannot distinguish truth or fiction on online platforms, good code from bad, to insecure software that causes supply chain attacks, to investment vehicles or fake medicine peddled that does not work, the idiotic MBA manager who does not understand the workers they manage and so on. That is why we're living on BS metrics and crappy reviews, because to the people who manage organizations their own workplace is a Blackbox. The critique that "AI" systems are semantically blind token processors is correct, it's just not bothering anyone because it already applied to most modern companies.

When people become passive consumers rather than broadly competent users how something works cannot be evaluated so by definition all they have to go off is appearance, which is why we live in the age of influencers and marketing, because that is now the only factor that actually matters.

Ironically, that I don't have to skip the details is the one thing I like. Mind you, what I use LLM for is toy stuff compared to what people who use them at their jobs for big projects.

But FWIW, I wrote this comment recently: https://news.ycombinator.com/item?id=49049006 And then I figured I should actually do that, I wanna see what it looks like. So I described it to the best of my abilities, and was polite about it: https://pastebin.com/sAbPzi05 Claude did not ask for clarification, it just knocked it out of the park, free tier, Sonnet, medium effort. https://i.imgur.com/YPwr6aT.png The only changes I made since then are not worth talking about.

Today I made another little tool, one I'll actually use but that is too specific and boring to explain, and because of that previous experience I spent nearly an hour writing that prompt, rereading it, making sure to use the same terms for the same concepts everywhere, moving things around so what logically belongs together actually is together, instead of a stream of consciousness, and the result is exactly what I had in mind. It was like imagining something and then seeing that exact thing, except the colour scheme had a nice topical little quirk even.

The plan now is to keep chatting about the result as I make experiences with it in practice, make notes for version 2.0, and then update and extend that first prompt and use it in a fresh context. And that is very fun, actually, and it's still a puzzle. What do I even want, exactly, and how do I describe it? That's not trivial, and there is hardly a "skill ceiling" here, though of course there are diminishing returns. I also assume that if I put some vague amount "less effort" into it, the result would be the same, but I don't care, because I enjoy using this brain muscle.

And directing the effort towards just a prompt, not a whole "system" around it pleases me too, since I generally don't like vendor lock-in, and prefer things to just be a "a bunch of files" that run almost anywhere. A clear English description is as close as it gets to that, and I don't mind that I roll the dice with every LLM I give a specific prompt to, since at least it doesn't mutate the prompt.

I hate the formatting of the the LLM code of course, I wouldn't have named many variables that way, so I still look forward to and love making things where I put every character myself. But since I code voluntarily I see it as two separate activities that don't really encroach upon each other unless I make them.