Live data from Hacker News

Three Inverse Laws of AI

susam.net

201–210 of 388 posts

Re: Three Inverse Laws of AI

#201

> An AI system is a tool and like any other tool, responsibility for its use rests with the people who decide to rely on it Doesn't that argument backfire though? If I use a chainsaw then to a certain extend I will need to rely on it not blowing up in my face or cutting my throat. If I drive a car I need to rely on that its brakes work and the engine doesn't suddenly explode. If a pilot flies an airplane which sudden…

I'm gonna push the responsibility up a level in the ladder:

A competent adult using a tool ought to understand the inherent pitfalls of using that tool.

Chainsaws are dangerous, in obvious and non obvious ways. The tool can operate as designed and still amputate your foot.

Re: Three Inverse Laws of AI

#202
post #124

Earlier quoted context omitted.

Look at any recent CoT output where the model is trying to infer from an underspecified prompt what the user wants or means. It is generally the first thing they do — try to figure out what did you mean with this prompt. When they can’t infer your intent, good models ask follow-on questions to clarify. I am wondering if this is a semantics issue as this is an established are of research, eg https://arxiv.org/pdf/2501…

Right, and then look at any number of research papers showing that CoT output has limited impact on the end result. We've trained these models to pretend to reason.

If it's only pretending to reason, then how is it that the CoT output improves performance on every single benchmark/test?

Re: Three Inverse Laws of AI

#203
post #176

Earlier quoted context omitted.

Recognising a stock cultural script isn't the same as capturing intent. Ask it something where no script exists. For example: "A man thrusts past me violently and grabs the jacket I was holding, he jumped into a pool and ruined it. Am I morally right in suing him?" There's no way for the LLM to know that the reason the jacket was stolen was to use it as an inflatable raft to support a larger person who was drowning.…

If your example for an exception to LLM's ability to infer intent is a deliberately misleading trick question that leaves out crucial contextual details, then I'm not sure what you're trying to prove. That same ambiguity in the question would trip up many humans, simply because you are trying as hard as possible to imply a certain conclusion. As expected, if I ask your question verbatim, ChatGPT (the free version) re…

If you want to convince yourself that they can infer intent despite the fundamental limitations of the systems literally not permitting it then you can be my guest.

Faking it is fine, sure, until it can’t fake it anymore. Leading the question towards the intended result is very much what I mean: we intrinsically want them to succeed so we prime them to reflect what we want to see.

This is literally no different than emulating anything intelligent or what we might call sentience, even emotions as I said up thread...

Re: Three Inverse Laws of AI

#204
post #6

I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can con…

I agree Asimov's laws are intentionally flawed/ambiguous (which makes the stories so good) but a slight difference to LLMs is the laws aren't just software, the positronic brain is physically structured in such a way (I'm hazy on the details) that violating the laws causes the robot to shutdown or experience paralysing anxiety. So if an LLM's safety rules fail or are subverted it can still generate dangerous output, while an Asimov robot will stop working (or go insane...)

Re: Three Inverse Laws of AI

#205

This is sound advice but isn't really about AI: Humans must not anthropomorphise {non-humans} Humans must not blindly trust the output of {anything} Humans must remain fully responsible and accountable for consequences arising from the use of {anything} Naturally, none of this advice matters at all as humans will do what they do. This just documents a subset of the ways real humans consistently make choices to their…

I kind of agree with 1, but not really with 2 and 3. It's easy to come up with trivial examples where it is both unreasonable and not feasible to follow those two, both for AI and non-AI scenarios.

Re: Three Inverse Laws of AI

#206
post #85

Earlier quoted context omitted.

“LLMs can capture intent now” reads to me the same as: AI has emotions now, my AI girlfriend told me so. I don’t discredit you as a person or a professional, but we meatbags are looking for sentience in things which don’t have it, thats why we anthropomorphise things constantly, even as children. We are easily fooled and misled.

What do you think it means to “capture intent” and where do current models fall short on this description? From my perspective the models are pretty good at “understanding” my intent, when it comes to describing a plan or an action I want done but it seems like you might be using a different definition. Tell me, what’s your intent? :)

[dead]

Re: Three Inverse Laws of AI

#208
post #72

Earlier quoted context omitted.

> Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent. He would be the very first person to say that they are flawed, that is the intent of them. He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.

LLM's now can capture intent. I think the issue now is that the full landscape of human values never resolves cleanly when mapped from the things we state in writing as being human values. Asimov tried to capture this too, as in, if a robot was tasked with "always protect human life", would it necessarily avoid killing at all costs? What if killing someone would save the lives of 2 others? The infinite array of micro…

> LLM's now can capture intent No they can’t. Here is an example: Ask an llm to write a multi phase plan for a very large multi file diff that it created, with least ambiguity, most continuity across plans; let’s see if it can understand your intent.

Re: Three Inverse Laws of AI

#209
post #72
post #6

I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can con…

> Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent. He would be the very first person to say that they are flawed, that is the intent of them. He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.

I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem.

Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want you to turn on the heater. But the literal semantics is just "the ambient temperature in the room is low" and it has nothing to do with heaters. Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do, often so quickly and effortlessly that we don't notice the complexity of the calculation we did.

Or suppose I say to a bot "tell me how to brew a better cup of coffee". What is encoded in the literal meaning of the language here? Who's to say that "better" means "better tasting" as opposed to "greater quantity per unit input"? Or that by "cup of coffee" I mean the liquid drink, as opposed to a cup full of beans? Or perhaps a cup that is made out of coffee beans? In fact the literal meaning doesn't even make sense, as a "cup" is not something that is brewed, rather it is the coffee that should go into the cup, possibly via an intermediate pot.

If the bot only understands literal language then this kind of query is a complete nonstarter. And yet LLMs can handle these kinds of things easily. If anything they struggle more with understanding language itself than with inferring intent.

Re: Three Inverse Laws of AI

#210
post #6

I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can con…

> It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines

You mean like stopping at a red light?

Post reply on HN