That post never existed. Stop listening to that thing • Hacker News Board

Re: That post never existed. Stop listening to that thing

#31

Post by fwlr » Mon, Jul 20, 2026, 6:51 PM UTC

benwr wrote: post #15

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? In case anyone is wondering: yes obviously even the dumbest current models correctly answer, given this prompt verbatim, that it's because of lunar new year.

No, the driving through Chinatown question is a question for self-driving cars. It is not a question for LLMs. There is some other question for LLMs, and the author is using the ancient technique of analogy to get you to think about that question.

Re: That post never existed. Stop listening to that thing

#32

Post by wilg » Mon, Jul 20, 2026, 6:52 PM UTC

SirFatty wrote: post #27

tptacek wrote: post #8

There's really no content in this post other than the claim that LLMs are stochastic parrots. That was a live debate two years ago. It's a very strange thing to write in 2026.

Why is it strange? It's still true.

It's an esoteric philosophical question that has no truth value either way.

Re: That post never existed. Stop listening to that thing

#33

Post by christina97 » Mon, Jul 20, 2026, 6:53 PM UTC

axus wrote: post #17

"Thus, anyone who wants to corral that kind of entity and make it do their bidding? Yeah, they want slaves." Always comes to mind when I see Elon and friends getting excited about AI robots. Slavery was more about economics than the role-playing.

I didn’t really get this point in the essay. What’s wrong with wanting servants as long as it’s done ethically etc and is not indentured servitude?

The whole gig/services economy is just building this up piece-by-piece: you can now pick the set of household needs you want taken care of for varying levels of money; and practically everyone participates in one form or another. This is exactly a disaggregated 21st century version of servants: paying for convenience. Of course with many issues in implementation, but I don’t see the ethical/moral issue with wanting this kind of thing?

Re: That post never existed. Stop listening to that thing

#34

Post by TZubiri » Mon, Jul 20, 2026, 6:53 PM UTC

SirFatty wrote: post #27

tptacek wrote: post #8

There's really no content in this post other than the claim that LLMs are stochastic parrots. That was a live debate two years ago. It's a very strange thing to write in 2026.

Why is it strange? It's still true.

If anything it's more important to hold. It's easy to hold one position and then falter, there's a pressure to always be with the times and not be 2 years demodé, but simple positions still hold true.

I wrote in the opencode thread that when it came out I put it behind a vm and its own user, and I never allowed it to run outside of it. But I know of people that as soon as they noticed that it worked well like 99% of the time, they let their guard down and give in to YOLO mode. And in orgs I've even seen CEOs treat their agents less like a user/employee/contractor, and try to 'empower' it by giving it ALL the data. Time bomb.

It's like fucking with condoms just the first couple of times. And then simultaneously ditching it and joining the free love movement.

Re: That post never existed. Stop listening to that thing

#35

Post by Terr_ » Mon, Jul 20, 2026, 6:56 PM UTC

I'd like to highlight that even if one added an URL-exists step [0], that doesn't do a dang thing for result-set problems of:

1. False-negatives, where relevant posts that do exist are not being shown (imagined or otherwise) to the user.

2. Posts which exist but don't fit the words the chaos-parrot uses to describe them.

3. "Relevance" being determined by unpredictable factors that aren't stable, predictable, or desirable.

In other words, it's just more whack-a-mole lipstick-on-a-pig third-animal-idiom-here.

[0] A bad idea on its own, since it creates a security vulnerability for data-exfiltration or indirect malicious attacks.

Re: That post never existed. Stop listening to that thing

#36

Post by close04 » Mon, Jul 20, 2026, 6:57 PM UTC

andrewflnr wrote: post #2

> I maintain that anything that is sufficiently aware to be able to actually understand things is also going to have enough of a sense of self that you won't just be able to tell it what to do. I don't see any reason for this to be true. "Actual understanding" (which I take to mean something like a predictive world model) and desire for self-determination coincide in humans because of our evolutionary history, becaus…

We have difficulty defining “actual understanding”. Humans get a free pass because we assume humans as a species possess this power but if we were to judge based on output alone maybe you couldn’t tell a human from a sufficiently advanced LLM/AI.

So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script, so there’s a mechanism there that doesn’t just follow hardcoded paths, it takes new data and processes it in novel but human like ways to come to a new conclusion if needed, then the author is right, in my opinion.

We don’t know what the reward function is for an AI but AI is trained on so much human work that it probably starts off with the same biases in its understanding and reactions. It feels like there’s something very basic in becoming more independent the more you understand of the world. Animals go through this as they grow too, not just humans (listening to parental authority until they eventually don’t anymore).

Since there is no established consensus on this one the burden is on either party to prove their own side. Just because the author said something first doesn’t mean they need to write a full proof while you get to say “nuh-uh” and that’s enough.

Re: That post never existed. Stop listening to that thing

#37

Post by hyperpape » Mon, Jul 20, 2026, 7:00 PM UTC

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea?

The premise seems to be that models aren't smart enough to understand this, and if they were, they'd be sentient and want autonomy.

For an article that's about making things up, and being too trusting, this seems bad. Maybe the author knows a lot about LLMs, but it doesn't seem like it.

Pasting the verbatim quote from the article into a free ChatGPT session: https://chatgpt.com/s/t_6a5e6f5e24f08191b6a482aad63cae63

Going to an incognito window and using a less leading question: https://chatgpt.com/s/t_6a5e6ee4a3508191bc1b352b41911b53.

If I go generic and just ask if there's anywhere I shouldn't drive, it doesn't get to Lunar New Year until I ask about "events" on the third question: https://chatgpt.com/s/t_6a5e6fb239208191b18cebcf7642c8b0. It's sort of a win for the article, if you think that people who run driverless car companies are all dumb, and won't create a prompt to tell their LLM to "consider events that might disrupt traffic."

Re: That post never existed. Stop listening to that thing

#38

Post by randusername » Mon, Jul 20, 2026, 7:00 PM UTC

grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

I bet there will be a very interesting generational divide between the kids that were born before or after about 2010; old enough to have some critical thinking facilities at the dawn of ChatGPT when it was still noticeably dumb.

Re: That post never existed. Stop listening to that thing

#39

Post by gnfargbl » Mon, Jul 20, 2026, 7:00 PM UTC

SirFatty wrote: post #27

tptacek wrote: post #8

There's really no content in this post other than the claim that LLMs are stochastic parrots. That was a live debate two years ago. It's a very strange thing to write in 2026.

Why is it strange? It's still true.

Because it's correct but irrelevant. It tells you about as much about the utility of LLMs as the statement "humans are just overpowered tree shrews" tells you about us.

Re: That post never existed. Stop listening to that thing

#40

Post by dreambuffer » Mon, Jul 20, 2026, 7:01 PM UTC

fwlr wrote: post #31

benwr wrote: post #15

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? In case anyone is wondering: yes obviously even the dumbest current models correctly answer, given this prompt verbatim, that it's because of lunar new year.

No, the driving through Chinatown question is a question for self-driving cars. It is not a question for LLMs. There is some other question for LLMs, and the author is using the ancient technique of analogy to get you to think about that question.

She's not wrong at all about her rhetorical implication (being, some information requires more than simple systems can provide), but she has concluded incorrectly that a car which does not know how to avoid a busy route is useless technology, or that the problem can only be solved by inventing life.