Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Three Inverse Laws of AI

susam.net

Post reply on HN
Print view View original

271–280 of 388 posts

Re: Three Inverse Laws of AI

#271

Post by BuyMyBitcoins »

giancarlostoro wrote: post #88
miyoji wrote: post #6

I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can con…

> Humans WILL anthropomorphize the AI r/myboyfriendisai Is quite... an interesting subreddit to say the least. If you've never seen this, it was really something when the version that followed GPT4o came out, because they were complaining that their boyfriend / girlfriend was no longer the same.

The whole “I can fix him” trope takes on a whole new meaning.

Re: Three Inverse Laws of AI

#273

Post by eranation »

Two of these laws I see being violated repeatedly, but it’s not always as obvious as one would hope.

Claude Code, Cursor, Codex etc impersonate your GitHub user. Either via CLI or MCP or using your git credentials. It’s perfectly reasonable that a piece of code made it to production where not a single human actually looked at it (Alice wrote it with AI, Bob “reviewed it” with AI, including posting PR comments as Bob, Alice “addresses” these comments, e.g. fixes / pushes back, and back and forth using the PR as an inefficient yet deceptive mechanism for AI to have a conversation with itself, while adding a false sense of process. Eventually Bob will prompt “is it prod ready” and will ship it, with 100% unit test coverage and zero understanding of what was implemented). Now this may sound like an imaginary scenario, but if it could happen, it will happen, and it probably already happens.

Cloud agents are nice enough to set the bot as the author and you as a co author, but still the GitHub MCP or CLI will use your OAuth identity.

I don’t have a clear answer to how to solve it, maybe force a shadow identity to each human so it’s clear the AI is the one who commented. But it’s easy to bypass. I’m worried not more people are worried about it.

Re: Three Inverse Laws of AI

#274

Post by Dylan16807 »

Eisenstein wrote: post #253
batshit_beaver wrote: post #153

Earlier quoted context omitted.

Right, and then look at any number of research papers showing that CoT output has limited impact on the end result. We've trained these models to pretend to reason.

> Right, and then look at any number of research papers showing that CoT output has limited impact on the end result. Which research papers? Do I have to find them? > We've trained these models to pretend to reason. I have no idea why that matters. Can you tell me what the difference is if it looks exactly the same and has the same result?

When they say "pretends to" here they're talking about something quantifiable, that the extra text it outputs for CoT barely feeds back into the decisionmaking at all. In other words it's about as useful as having the LLM make the decision and then "explain" how it got there; the extra output is confabulation.

Though I'm not sure how true that claim is...

Re: Three Inverse Laws of AI

#275

Post by nijave »

davebranton wrote: post #232

This phrase always fascinates me : "AI-generated content must not be treated as authoritative without independent verification appropriate to its context." I've heard the same thing expressed somewhat more concisely as "Never ask AI a question to which you don't already know the answer". Which raises the question, and I do think it's an important one. Given that this is true, what function does AI answering a questio…

>You could achieve precisely the same outcome by using search engines and normal research

When it comes to software engineering (as a software engineer myself), the AI is generally a lot quicker than me researching "the old fashion way"

I can fumble around and say "list free software that does X" without knowing I'm looking for, say, a CRM and then spend a couple minutes looking over the results when the "manual" method I would have spent 10-30 minutes just figuring out I was looking for "CRMs"

I like to think of these as sort of "psuedo NP hard" or questions that are slow to answer but quick to validate

Re: Three Inverse Laws of AI

#276

Post by applfanboysbgon »

canjobear wrote: post #209
dijit wrote: post #72

Earlier quoted context omitted.

> Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent. He would be the very first person to say that they are flawed, that is the intent of them. He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.

I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem. Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want…

> Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do

No, it is not "figuring out" anything, much less like a human might. Every time "I'm cold" appears in the training data, something else occurs after that. ChatGPT is a statistical model of what is most likely to follow "I'm cold" (and the other tokens preceding it) according to the data it has been trained on. It is not inferring anything, it is repeating the most common or one of the most common textual sequences that comes after another given textual sequence.

Re: Three Inverse Laws of AI

#277

Post by polynomial »

eranation wrote: post #273

Two of these laws I see being violated repeatedly, but it’s not always as obvious as one would hope. Claude Code, Cursor, Codex etc impersonate your GitHub user. Either via CLI or MCP or using your git credentials. It’s perfectly reasonable that a piece of code made it to production where not a single human actually looked at it (Alice wrote it with AI, Bob “reviewed it” with AI, including posting PR comments as Bob,…

Not everybody isn't worried: https://ctolunchnyc.substack.com/p/cto-lunch-nyc-spring-2026...

Re: Three Inverse Laws of AI

#278

Post by boredatoms »

protocolture wrote: post #259
protocolture wrote: post #254

>Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence. Impossible. I anthropomorphise my chair when it squeaks. Humans anthropomorphise everything. They gender their cars and boats. This tool can actually make readable sentences and play a…

>>Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence. Still angry about this. The reason humans ban animal cruelty is that animals look like they have emotions humans can relate to. LLMs are even better than animals at this. If you aren't…

/s ?

Re: Three Inverse Laws of AI

#280

Post by goatlover »

aranchelk wrote: post #92

Anthropomorphizing is likely a mistake, but Daniel Dennett’s idea that the most straightforward (possibly only practical) way to create the external appearance of consciousness is a real internal consciousness does float around in my thoughts. I haven’t yet seen any convincing appearance of one in an LLM, but I think if skeptical people don’t keep an eye out for the signs, we may be the last to see it. He also wrote…

> but Daniel Dennett’s idea that the most straightforward (possibly only practical) way to create the external appearance of consciousness is a real internal consciousness does float around in my thoughts.

Pretty sure Daniel Dennett has been adamantly opposed to any sort of theater in the mind when it comes to consciousness. He views it as biologically functional. For him, to make a conscious robot, you need to reproduce the functionality of humans and animals that are conscious, not just an appearance, such as outputting text. Although he's also suggested that consciousness might be a trick of language. In which case ... that might be an older view though. He used to argue that dreams were "seeming to come to remember" upon awakening, because again he his view is to reject any sort of homunculus inside the head.

You might be mixing up some of Dennett and David Chalmer's views. David Chalmers is a proponent of the hard problem, but he's fine with a kind of psycho-physical-functional connection for consciousness. Any informationally rich process might be conscious in some manner.

Post reply on HN