Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does Alan Kay think about LLMs?

quora.com

Post reply on HN
Print view View original

81–90 of 155 posts

Re: What does Alan Kay think about LLMs?

#81

Post by brookst »

halayli wrote: post #73
kergonath wrote: post #62

Earlier quoted context omitted.

> "reasoning by correlation" as superstition is a brutal insight. I don’t think he is right on that one, though. Reasoning by correlation is a kind of empiricism, and it can be tested. “These things happen together” or “if I do this, then that happens” can be disproven with statistical analysis even without any understanding of the underlying mechanisms at play their causes. Superstition is beyond that; it is a belie…

Superstition is assuming a causation when you identify a correlation.

Often followed by applying confirmation bias to see correlations when there are none.

Re: What does Alan Kay think about LLMs?

#82

Post by alexvoda »

kergonath wrote: post #62
ozten wrote: post #29

> That humans also do this all the time is “interesting”, “dangerous” etc., but it is also why trying to move from superstition (this is actually what “reasoning by correlation” amounts to) to more scientific methods is critical for anything like civilization to be created. "reasoning by correlation" as superstition is a brutal insight.

> "reasoning by correlation" as superstition is a brutal insight. I don’t think he is right on that one, though. Reasoning by correlation is a kind of empiricism, and it can be tested. “These things happen together” or “if I do this, then that happens” can be disproven with statistical analysis even without any understanding of the underlying mechanisms at play their causes. Superstition is beyond that; it is a belie…

I think the author and you are using different meanings for "reasoning".

We do know that many people even in professional fields confuse correlation with causation. And even when that doesn't happen, when only doing measurments, correlation is often considered good enough without giving any thought to the underlying mechanism. This may be the root of Goodhart's law stating that when a measure becomes a target it stops being a good measure. The measure was only correlated with the behaviour the person doing the measuring thought was measuring.

Superstition is similar, but drops any shred of statistical rigour, relying on mere anecdotal evidence.

Re: What does Alan Kay think about LLMs?

#83

Post by pixl97 »

adamrezich wrote: post #51
RodgerTheGreat wrote: post #42

Earlier quoted context omitted.

I think it's under-appreciated how much LLMs harvest the natural, human tendency to generously ascribe meaning, subtext, and intent to text they read, glossing over flaws and small mistakes so long as the overall "thrust" seems reasonable enough. In a sense, LLMs have reinvented cold-reading from first-principles and created the cleverest Hans of them all.

We really should have figured out anthropomorphization and how to overcome it before we got to the "AI age".

Heh, I don't think you've had enough philosophy under your belt to realize how incredibly far we are from figuring that out...

Or to say, "It's Plato's cave all the way down"

Re: What does Alan Kay think about LLMs?

#84

Post by Me1000 »

ToucanLoucan wrote: post #21
falsaberN1 wrote: post #15

Earlier quoted context omitted.

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they just...recombine text by probabilities and aren't deterministic. I get incorrect information every time I ask them a thing, and it's incorrect in different ways every time. The only things they seem to get consistently correct are very widespread facts that are…

So much of the odd almost cultish community around LLMs seems to just be people who really want to be at the ground floor of the Next Big Thing who are so wildly biased into this being that next big thing that they will spend all their time, all their energy, not just on other people but on themselves, convincing themselves over and over that their LLM girlfriend really does love them, that their LLM assistant is goi…

You seem to be making a point in good faith so I would like to give you a slightly different perspective.

I'm not entirely sure what you mean by "cultish community", from my perspective there are a few distinct communities around LLMs, all focusing on different aspects, all excited about different things.

One common theme across all the groups though is that they used an LLM for the first time and their mind ran wild with the possibilities. That first moment when the LLM does something better than you expected, or even completely unexpected. I think most people understand that their imagination might be overactive in that moment. But it's a rare feeling to be surprised by a new technology (at least for me) these days.

On the other end, we have social media platforms where being a pessimistic curmudgeon ends up getting the likes and shares. And it's just easier to be a pessimistic curmudgeon; the vast majority of ideas never work as well in the real world as they do in your head. I'm just as guilty of this as anyone else. But the real problem is that it puts us into tribes. As someone who is very excited about what LLMs are going to bring to our futures, when I see someone post on Mastodon or HN, or wherever, I become defensive and my monkey brain feels the urge to push back. In particular because I think the criticisms generally voiced are not well reasoned or thought out. Your own post has a tone of dismissal, painting a lot of people, all of whom excited about different things, as a cult who is obsessed with their LLM girlfriend. I would agree that anyone today trying to draw some deeper meaning from the outputs of these systems are probably worthy of dismissal, but I don't think that's the vast vast majority of people who are excited about LLMs. And it makes _me_ sad that the extremists are the ones that get to suck all the oxygen out of the conversation.

We're in the beginning days of this new technology. LLMs are good at doing things traditional software isn't, and bad at doing a lot of things computers are traditionally good at. Natural language answer engines and sex bots might have been some of the first obvious applications of LLMs, but I'm willing to bet there are a lot more undiscovered use cases out there. Simon Willison has some great advice, which is for newcomers to try to break the LLM as quickly as they can, get it to lie to you, or do something wrong. Test its limits. That's part of the process! We're going to need some time to figure it all out and make these systems work well for us. I'm a technologist, and exploring this technology is exciting.

Re: What does Alan Kay think about LLMs?

#85

Post by bombcar »

chongli wrote: post #36
layer8 wrote: post #16

Earlier quoted context omitted.

Yes, that’s what I mean by vague suggestions. How do you actually build and maintain a system of any complexity with that? How do you ensure it will do what it is supposed to do? And Smalltalk wasn’t that, it is mostly just regular method calls and not “message passing”.

He’s not just a proponent of message passing, he’s a proponent of late binding [1]. The idea there is to have a running image of the environment and interact with it in real time, updating code while it’s running (no recompiling or anything like that). The idea is a high level of interactivity with very tight feedback loops. A whole operating system like this would allow you to hack on the user interface and change t…

The closest I’ve seen to this is programming in Excel - each step is visible to you and you can tweak it in the cell itself and see what it does.

Re: What does Alan Kay think about LLMs?

#86

Post by pixl97 »

thesz wrote: post #9
altruios wrote: post #7

Earlier quoted context omitted.

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

I think I have to know what chain of reasoning is behind this or that fact and/or deduction. i would like to be able to verify that. For example, the proof of the absence of solution in SAT should be accompanied with the easily verifiable chain of reasoning. This shows the absence of incorrect deductions and missing assignments. Another example is autovectorization in contemporary compilers, they can show you why par…

Even you can't do that yourself. You can only mad post ad-hoc justifications for the choices you made.

Re: What does Alan Kay think about LLMs?

#87

Post by andrewflnr »

pixl97 wrote: post #70
ianbicking wrote: post #59

This feels like a limited and perhaps naive perspective on LLMs. If you looked at computers as adding machines in the 60s/70s then you'd be missing most of what was interesting about computers. And if you look at LLMs as a question answering service now, you are also missing a lot. It's hard to compare trust of LLMs to other computing, because many of the things that LLMs get wrong and right were previously intractab…

“All models are wrong, some are useful.” What's kind of funny is almost all the people that complain about accuracy of LLMs would gladly answer the question of "What's the chance for rain today" without giving giving you a 5 minute lecture on what forecasts actually mean.

Yeah because most everyone you talk to knows how to treat weather forecasts, but LLMs are still widely misunderstood. The right idea seems to be getting out, but in no small part due to the "complaining" you mock.

Re: What does Alan Kay think about LLMs?

#88

Post by alexvoda »

ianbicking wrote: post #59

This feels like a limited and perhaps naive perspective on LLMs. If you looked at computers as adding machines in the 60s/70s then you'd be missing most of what was interesting about computers. And if you look at LLMs as a question answering service now, you are also missing a lot. It's hard to compare trust of LLMs to other computing, because many of the things that LLMs get wrong and right were previously intractab…

There was a time when a search engine would return no results if it didn't find anything instead of returning increasingly irrelevant results.

It was a conscious decision by corporations to implement the dropping of constraints and search terms when too few results would have been returned. Today the search operators are a joke.

Re: What does Alan Kay think about LLMs?

#89

Post by brookst »

soperj wrote: post #76
pixl97 wrote: post #70

Earlier quoted context omitted.

“All models are wrong, some are useful.” What's kind of funny is almost all the people that complain about accuracy of LLMs would gladly answer the question of "What's the chance for rain today" without giving giving you a 5 minute lecture on what forecasts actually mean.

"I'm unable to provide real-time weather updates as my data is not current. To find out the chances of rain today, you can check your local weather forecast through a reliable weather website or app, or tune in to a local news station for the most up-to-date information." - chatgtp

"Seattle is known for its rainy weather, especially during the spring months. According to historical data, March typically sees an average of 17 rainy days in Seattle. Given that we're in the middle of March, there's a fairly high likelihood of rain on any given day.

Without access to real-time weather data, I'd estimate that there's approximately a 50-60% chance of rain in Seattle on March 20th, based on historical averages. However, for the most accurate forecast, I'd recommend checking a reliable weather website or app closer to the date." -- chatgpt

(prompted with the current date, location, an admonition to estimate, and a promise that I understand it will likely be wrong)

Re: What does Alan Kay think about LLMs?

#90

Post by brookst »

falsaberN1 wrote: post #15
altruios wrote: post #7

Earlier quoted context omitted.

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they just...recombine text by probabilities and aren't deterministic. I get incorrect information every time I ask them a thing, and it's incorrect in different ways every time. The only things they seem to get consistently correct are very widespread facts that are…

s/LLMs/people
Post reply on HN