Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

That post never existed. Stop listening to that thing

rachelbythebay.com

Post reply on HN
Print view View original

61–70 of 184 posts

Re: That post never existed. Stop listening to that thing

#61

Post by uberstuber »

nightpool wrote: post #19
senkora wrote: post #3

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)

Unfortunately, I just had this experience with Google Maps last month—I wanted to go to downtown to buy cheese at Pike Place, but I didn't realize that 6th avenue was closed off for the annual Pride Parade. Traffic was horrible, and Google Maps kept telling me to turn right onto closed off streets until I just had to abandon my trip entirely. Definitely some lack of communication between Seattle city planning and the…

SPD close off certain directions of traffic after Mariner's games. It's always the same layout, always after Mariner's games, and Google still tries to send me through the area the wrong way every time

Re: That post never existed. Stop listening to that thing

#62

Post by alex0015 »

wilg wrote: post #7
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

Making things up is only really a common issue on the non-thinking models which nobody should be using. The regular chatbots are Autogooglers and are very useful for research. This is just not a good argument anymore. Edit: Guys, why are we downvoting this? Does no one use like ChatGPT or Claude and understand how it works? Do you all think its regularly hallucinating links still? Is everyone on HN using like free si…

I'm right there with you for a lot of stuff. I ask a question and can be very confident that ChatGPT is citing sources, then sometimes I go read the sources. The more critical the information I'm looking for is, the more careful I am about this.

The other day though I was seeing how well it could pull details of its own conversations with me. It often does this pretty well for broad strokes of things - it remembers, largely, what cameras I have and use when I ask photography questions. It's never made things up here, but it does forget details, such as whether I've bought something or am just considering it. However, when I asked it for a specific interaction I thought I remembered, it gladly went along with my false memory and provided an affirmative answer. It was the first time I'd been caught in a serious hallucination with a frontier model (Sol High on the web chat interface) in a long time.

Re: That post never existed. Stop listening to that thing

#63

Post by cwillu »

input_sh wrote: post #55
Kerrick wrote: post #9

Earlier quoted context omitted.

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

One of the things I do semi-frequently is look for the evidence that some concert took place 15+ years ago. Or maybe I already definitively know it happened, but not exactly at which venue or the exact date of the concert. This I feel like is a non-trivial task, but one with a very definitive answer whose evidence more often than not still exists somewhere online. In my experience every LLM out there is utterly usele…

It's vector search that's eaten everything that used to have at least a smidgen of parametric search.

Re: That post never existed. Stop listening to that thing

#64

Post by cwillu »

undersuit wrote: post #47
wilg wrote: post #7

Earlier quoted context omitted.

Making things up is only really a common issue on the non-thinking models which nobody should be using. The regular chatbots are Autogooglers and are very useful for research. This is just not a good argument anymore. Edit: Guys, why are we downvoting this? Does no one use like ChatGPT or Claude and understand how it works? Do you all think its regularly hallucinating links still? Is everyone on HN using like free si…

>Do you all think its regularly hallucinating links still? When did that stop? May 7th, 2026?

A tech blog is going to have more than it's fair share of enthusiasts using small self-hosted and similar models; it is entirely possible that that completely accounts for the behaviour described in the article.

Re: That post never existed. Stop listening to that thing

#65

Post by Buttons840 »

rogerrogerr wrote: post #16
Kerrick wrote: post #9

Earlier quoted context omitted.

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

A lot of that sense was probably based on side channels like proper grammar, writing style, etc. That’s all gone now :(

We have other signals now.

Before we would find an intriguing post on the internet from years ago, and you have to verify it with additional research--it's easy to skip that additional research.

With a LLM when you're skeptical you can interogate it. One thing we know for sure is LLMs are quick to admit mistakes were made when interrogated, comically so. A LLM might not always recognize its own mistake, but at least it is available for easy interogation, unlike the forum posts of old.

Manual research from reputable sources remains an option.

Re: That post never existed. Stop listening to that thing

#66

Post by cwillu »

close04 wrote: post #36
andrewflnr wrote: post #2

> I maintain that anything that is sufficiently aware to be able to actually understand things is also going to have enough of a sense of self that you won't just be able to tell it what to do. I don't see any reason for this to be true. "Actual understanding" (which I take to mean something like a predictive world model) and desire for self-determination coincide in humans because of our evolutionary history, becaus…

We have difficulty defining “actual understanding”. Humans get a free pass because we assume humans as a species possess this power but if we were to judge based on output alone maybe you couldn’t tell a human from a sufficiently advanced LLM/AI. So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script, so there’s a mechanism there that doesn’t just follow hardcode…

> Humans get a free pass

They really shouldn't.

Does nobody remember why fizzbuzz was a thing? People who talked a big game while having no actual competence or understanding of the subject matter?

Re: That post never existed. Stop listening to that thing

#67

Post by sudobash1 »

TZubiri wrote: post #25
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

Nowadays not just big orgs are falling into the delusion, but also big people. When Linus posted that AIs and vibecoding were here to stay and declared resistance to it as harmful, I stopped to consider whether I was wrong, but it has made me realize that in retrospect Linus Torvalds and Linux itself aren't actually the holy grail of computing. I didn't feel that way with Richard Dawkins, its not like falling for an…

When did he post that "vibecoding" was here to stay? He is allowing AI generated code, and AI linting tooling in kernel development, but importantly, the expectation of human responsibility and review remains. This seems a far cry from vibecoding.

Re: That post never existed. Stop listening to that thing

#68

Post by orev »

senkora wrote: post #3

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)

Maps apps see the streets that don’t have cars on them (the same ones closed for a festival) as good routes to send vehicles because there’s currently no vehicles using them. It keeps trying to route people there and doesn’t understand why.

I think it takes someone (at Google) manually marking those roads as unavailable before it will stop trying. I’ve seen it happen with other things too, like if a highway is closed because of a bad accident.

Re: That post never existed. Stop listening to that thing

#69

Post by jt2190 »

hugodan wrote: post #23
wilg wrote: post #10

Could the article "The Stack" the user was looking for have been 'What is "the stack"?' by Julia Evans? https://web.archive.org/web/20160305142512/https://jvns.ca/b...

No, its this: https://rachelbythebay.com/w/2013/05/20/stack/ it exists.

That’s “Sysadmin work teaches you the value of stacks” from May 20, 2013. AI hallucinated a post named “The Stack” from August 19, 2013.

Re: That post never existed. Stop listening to that thing

#70

Post by dcrazy »

nightpool wrote: post #19
senkora wrote: post #3

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)

Unfortunately, I just had this experience with Google Maps last month—I wanted to go to downtown to buy cheese at Pike Place, but I didn't realize that 6th avenue was closed off for the annual Pride Parade. Traffic was horrible, and Google Maps kept telling me to turn right onto closed off streets until I just had to abandon my trip entirely. Definitely some lack of communication between Seattle city planning and the…

A couple years ago Apple Maps did not know that roads in downtown SF were closed for Bay to Breakers. Thankfully the cops were still setting up barriers and I was able to weave around them to reach the Bay Bridge.

This kind of thing happens regularly.

Post reply on HN