Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

That post never existed. Stop listening to that thing

rachelbythebay.com

Post reply on HN
Print view View original

71–80 of 184 posts

Re: That post never existed. Stop listening to that thing

#71

Post by mananaysiempre »

orev wrote: post #68
senkora wrote: post #3

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)

Maps apps see the streets that don’t have cars on them (the same ones closed for a festival) as good routes to send vehicles because there’s currently no vehicles using them. It keeps trying to route people there and doesn’t understand why. I think it takes someone (at Google) manually marking those roads as unavailable before it will stop trying. I’ve seen it happen with other things too, like if a highway is closed…

I’m guessing GP’s point is that Google also has the locations of all the people not in cars, so Google Maps could take the hint if it wanted. But thus far nobody appears to care enough to fix this. (And apparently in e.g. India cars and pedestrians may routinely use the same roads at the same time, so perhaps such a system might be more fragile globally that our own experiences would lead us to believe.)

Re: That post never existed. Stop listening to that thing

#73

Post by dnemmers »

Kerrick wrote: post #9
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

I'm not sure if this is your take, but it feels an aweful lot like an AI "Good enough is good enough" handwave.

Re: That post never existed. Stop listening to that thing

#74

Post by andrewflnr »

close04 wrote: post #36
andrewflnr wrote: post #2

> I maintain that anything that is sufficiently aware to be able to actually understand things is also going to have enough of a sense of self that you won't just be able to tell it what to do. I don't see any reason for this to be true. "Actual understanding" (which I take to mean something like a predictive world model) and desire for self-determination coincide in humans because of our evolutionary history, becaus…

We have difficulty defining “actual understanding”. Humans get a free pass because we assume humans as a species possess this power but if we were to judge based on output alone maybe you couldn’t tell a human from a sufficiently advanced LLM/AI. So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script, so there’s a mechanism there that doesn’t just follow hardcode…

> Since there is no established consensus on this one the burden is on either party to prove their own side.

Nah. Separate things are separate until proven otherwise.

So much bullshit appears in the form "X is Y" where X and Y are related but distinguishable phenomena. If you really try to pretend that every such phrase deserves serious consideration just because it feels plausible to someone, you drown in nonsense immediately. (Of course what you actually do is only grant that consideration to ideas that feel plausible to you personally, but it's obvious why that's not a good principle, right?) So we have to make all of them justify their existence.

Anyway:

> So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script

No. None of this. Pretty sure this is just an incoherent analogy.

> so there’s a mechanism there that doesn’t just follow hardcoded paths, it takes new data and processes it in novel but human like ways...

You're putting your conclusion in the premise, right there in the open.

> ...then the author is right, in my opinion.

This still doesn't follow from your handwaved premises as far as I can tell.

Re: That post never existed. Stop listening to that thing

#75

Post by ButlerianJihad »

orev wrote: post #68
senkora wrote: post #3

> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)

Maps apps see the streets that don’t have cars on them (the same ones closed for a festival) as good routes to send vehicles because there’s currently no vehicles using them. It keeps trying to route people there and doesn’t understand why. I think it takes someone (at Google) manually marking those roads as unavailable before it will stop trying. I’ve seen it happen with other things too, like if a highway is closed…

Road closures and accidents are crowdsourced things, so ordinary users can contribute intel live from the scene.

Google is able to estimate crowd sizes in a business or on a public transit vehicle. This seems to be based on the number of Android devices reporting their location in a cluster. Maps could obviously make inferences if there were large crowds of people, not moving in vehicles.

Re: That post never existed. Stop listening to that thing

#76

Post by andrewflnr »

cwillu wrote: post #66
close04 wrote: post #36

Earlier quoted context omitted.

We have difficulty defining “actual understanding”. Humans get a free pass because we assume humans as a species possess this power but if we were to judge based on output alone maybe you couldn’t tell a human from a sufficiently advanced LLM/AI. So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script, so there’s a mechanism there that doesn’t just follow hardcode…

> Humans get a free pass They really shouldn't. Does nobody remember why fizzbuzz was a thing? People who talked a big game while having no actual competence or understanding of the subject matter?

This too, yes. There are many such cases of desire for self-determination separate from "understanding". The question is whether it also goes the other way.

Re: That post never existed. Stop listening to that thing

#77

Post by andrewflnr »

runarberg wrote: post #14
andrewflnr wrote: post #2

> I maintain that anything that is sufficiently aware to be able to actually understand things is also going to have enough of a sense of self that you won't just be able to tell it what to do. I don't see any reason for this to be true. "Actual understanding" (which I take to mean something like a predictive world model) and desire for self-determination coincide in humans because of our evolutionary history, becaus…

> because our reward function involves reproducing in a competitive environment This is an extremely reductive way to look at human existence. So much so that I read this with Richard Dawkins voice in my head. Our existence is far richer then just the capability to reproduce. We (as well as other animals and even plants) do far more things then multiply, and in fact we often do things which are detrimental towards th…

Humans do indeed have complex behavior and motivations, that sometimes relate only abstractly if at all to the obvious gene drives. But the link between intelligence and self-determination was basically set at the dawn of intelligence itself. Even an insect will struggle if it's confined, and you can believe it's using whatever intelligence it can to get free. If anything self-determination is prior.

Re: That post never existed. Stop listening to that thing

#78

Post by snailmailman »

Kerrick wrote: post #9
grey-area wrote: post #6

> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

An anonymous answer to a question is more trustworthy to me. They have no reason to lie. They are usually answering out of kindness. At least they used to be. Now it’s often actually bots advertising a product or pushing something, pretending to be a helpful user with an anecdote and a good experience using a niche product.

AI shouldn’t have any reason to lie. But its lies aren’t intentional. It’s just actually making things up and “hallucinating” when it pretends that an option or setting exists, or confidently claims something entirely untrue, and makes up a source to go with it. For something Google is willing to shove into the top of every search result it’s crazy the percentage of time the answer is blatantly incorrect.

Re: That post never existed. Stop listening to that thing

#79

Post by rogerrogerr »

Buttons840 wrote: post #65
rogerrogerr wrote: post #16

Earlier quoted context omitted.

A lot of that sense was probably based on side channels like proper grammar, writing style, etc. That’s all gone now :(

We have other signals now. Before we would find an intriguing post on the internet from years ago, and you have to verify it with additional research--it's easy to skip that additional research. With a LLM when you're skeptical you can interogate it. One thing we know for sure is LLMs are quick to admit mistakes were made when interrogated, comically so. A LLM might not always recognize its own mistake, but at least…

You can't interrogate the context of an LLM when you only have its output.

Re: That post never existed. Stop listening to that thing

#80

Post by eddd-ddde »

rogerrogerr wrote: post #16
Kerrick wrote: post #9

Earlier quoted context omitted.

I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.

A lot of that sense was probably based on side channels like proper grammar, writing style, etc. That’s all gone now :(

For what reason would grammar influence whether something is true or not...
Post reply on HN