Earlier quoted context omitted.
> Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent. He would be the very first person to say that they are flawed, that is the intent of them. He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.
I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem. Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want…
I’d never just turn on the heater silently if someone said this to me. I think it means something else.