Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

That post never existed. Stop listening to that thing

rachelbythebay.com

Post reply on HN
Print view View original

51–60 of 184 posts

Re: That post never existed. Stop listening to that thing

#51

Post by axus »

wilg wrote: post #21
axus wrote: post #17

"Thus, anyone who wants to corral that kind of entity and make it do their bidding? Yeah, they want slaves." Always comes to mind when I see Elon and friends getting excited about AI robots. Slavery was more about economics than the role-playing.

Slavery was about racism and power and control, not economics. Slavery is bad for the economy! https://www.nber.org/papers/w31758 https://www.noahpinion.blog/p/nations-dont-get-rich-by-plund...

AI is bad for the economy too, but the AI-holders will make a lot of money. There was plenty of racism and control after slavery ended.

Re: That post never existed. Stop listening to that thing

#52

Post by grey-area »

wilg wrote: post #7
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

Making things up is only really a common issue on the non-thinking models which nobody should be using. The regular chatbots are Autogooglers and are very useful for research. This is just not a good argument anymore. Edit: Guys, why are we downvoting this? Does no one use like ChatGPT or Claude and understand how it works? Do you all think its regularly hallucinating links still? Is everyone on HN using like free si…

We know they are, and the author of the article cites the proof. LLMs do hallucinate, there is no way to make them not do it, because of the way they work.

Re: That post never existed. Stop listening to that thing

#53

Post by kaikai »

christina97 wrote: post #33
axus wrote: post #17

"Thus, anyone who wants to corral that kind of entity and make it do their bidding? Yeah, they want slaves." Always comes to mind when I see Elon and friends getting excited about AI robots. Slavery was more about economics than the role-playing.

I didn’t really get this point in the essay. What’s wrong with wanting servants as long as it’s done ethically etc and is not indentured servitude? The whole gig/services economy is just building this up piece-by-piece: you can now pick the set of household needs you want taken care of for varying levels of money; and practically everyone participates in one form or another. This is exactly a disaggregated 21st centu…

There’s a difference between servitude and slavery. Servitude is voluntary, slavery is not. There’s a gradation between them, and indentured servitude is somewhere in between the two. The “issues in implementation” are exactly where the ethical and moral issue lie.

Re: That post never existed. Stop listening to that thing

#54

Post by dpoloncsak »

Kerrick wrote: post #9
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

Yeah, it's like everyone was under the impression you could just trust the internet before LLMs.

It's a great tool, but verify the important things (or do them yourself)

Re: That post never existed. Stop listening to that thing

#55

Post by input_sh »

Kerrick wrote: post #9
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

One of the things I do semi-frequently is look for the evidence that some concert took place 15+ years ago. Or maybe I already definitively know it happened, but not exactly at which venue or the exact date of the concert. This I feel like is a non-trivial task, but one with a very definitive answer whose evidence more often than not still exists somewhere online.

In my experience every LLM out there is utterly useless and quickly defaults into "here are other concerts that took place around that time near that location". Google Search (ignoring the AI overview) is even more useless, as it refuses to show literally any webpage that's older than say 5 years. YouTube search is genuinely better than Google at surfacing old and grainy fan-made videos uploaded in like 2010, but also defaults into synonyms nonsense pretty quickly.

But, the search functionality of exactly one forum and three local news websites that I know have an archive that dates back long enough beats every single one of those abovementioned every single time. Three people are talking about their experience at a concert on a random 15+ year old forum thread? It happened. The tiny list of 5 or so (Google-hosted!) Blogspot blogs I have bookmarked? They usually have a photo of the ticket that Google Images refuses to show me.

Not only are search engines completely dead as a category, but LLMs are a shit replacement for them. "We" (okay, Google specifically) has truly committed a crime comparable to burning the Library of Alexandria. Everything older than a decade that wasn't properly documented on Wikipedia is just gone, never to be seen again.

Re: That post never existed. Stop listening to that thing

#56

Post by cma256 »

wilg wrote: post #21
axus wrote: post #17

"Thus, anyone who wants to corral that kind of entity and make it do their bidding? Yeah, they want slaves." Always comes to mind when I see Elon and friends getting excited about AI robots. Slavery was more about economics than the role-playing.

Slavery was about racism and power and control, not economics. Slavery is bad for the economy! https://www.nber.org/papers/w31758 https://www.noahpinion.blog/p/nations-dont-get-rich-by-plund...

True but I would also caveat that it may have been an open economic question back then (I don't know the state of the debate) and the personal-economics of slavery are unmistakable for the "lucky" few.

Re: That post never existed. Stop listening to that thing

#57

Post by Kerrick »

rogerrogerr wrote: post #16
Kerrick wrote: post #9

Earlier quoted context omitted.

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

A lot of that sense was probably based on side channels like proper grammar, writing style, etc. That’s all gone now :(

No, the sense had nothing to do with the content and everything to do with the context. Perfect grammar and writing style were never enough to get me to trust an anonymous forum post or unknown bloggers post for certain topics like health advice. Sloppy grammar and writing style were never a deterrent for me believing them for other kinds of topics like where to check on the HVAC system to find the sticker. I think the line could more accurately be described as the level of risk if it's wrong.

Re: That post never existed. Stop listening to that thing

#58

Post by skybrian »

The 404s mean that somebody or something checked if the post at the URL existed, and got a clear answer. Seems like it's good that they checked, at least. You won't see any evidence of the ones who don't check.

Also, I imagine keeping their cars out of Chinese New Year celebrations (and other big events) is something Waymo could figure out how to do if they put their minds to it.

Re: That post never existed. Stop listening to that thing

#59

Post by floren »

> Here's the example I throw out to people who have been in the Bay Area for a year or two. It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea?

Well the answer is actually that it's always a bad idea to drive straight through the middle of Chinatown at any time of the year, because the streets are narrow and full of tourists.

Re: That post never existed. Stop listening to that thing

#60

Post by tauroid »

SirFatty wrote: post #27
tptacek wrote: post #8

There's really no content in this post other than the claim that LLMs are stochastic parrots. That was a live debate two years ago. It's a very strange thing to write in 2026.

Why is it strange? It's still true.

It seems vanishingly rare that people acknowledge the true situation which is that, during training, it really does "think" in that it develops beliefs and marks out precisely chosen trails through its vast and expanding territory. Has a soul, attuned to God, blessed member of the flock, or may as well be.

And then during inference the light goes out and the "agent" staggers randomly like a zombie along those preset paths. Stochastic parrot.

So you and your AGENT.md and your skills files and your harnesses will never make your Claude perceive something that is not in its model checkpoint.

ML experts and neurobiologists free to correct me.

Post reply on HN