Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does Alan Kay think about LLMs?

quora.com

Post reply on HN
Print view View original

21–30 of 155 posts

Re: What does Alan Kay think about LLMs?

#21

Post by ToucanLoucan »

falsaberN1 wrote: post #15
altruios wrote: post #7

Earlier quoted context omitted.

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they just...recombine text by probabilities and aren't deterministic. I get incorrect information every time I ask them a thing, and it's incorrect in different ways every time. The only things they seem to get consistently correct are very widespread facts that are…

So much of the odd almost cultish community around LLMs seems to just be people who really want to be at the ground floor of the Next Big Thing who are so wildly biased into this being that next big thing that they will spend all their time, all their energy, not just on other people but on themselves, convincing themselves over and over that their LLM girlfriend really does love them, that their LLM assistant is going to be the next DaVinci, that their generated art is so much better than anything else.

More than anything it makes me sad. ANY amount of critical thinking would tell you all of this is not true, which isn't to say there's NO USE AT ALL for this technology, it certainly exists and has it's applications, and I also do more or less believe someday we'll create digital intelligence, but at the same time... ChatGPT is not that. DALL-E is not that. These systems are interesting and they have uses but they are not emergent intelligence, they don't know anything, they just assemble words from massive probability matrices and then the people who read those words ascribe meaning to them that is far, far beyond what originated them.

In this way it's not so dissimilar from any garden variety religion, it's just religion for people who think they're too smart to fall into the trap of motivated reasoning and magical thinking.

Re: What does Alan Kay think about LLMs?

#22

Post by markrages »

altruios wrote: post #7
dpflan wrote: post #4

There is a lot in here, various paths to venture off, but the bottom line seems to be trust is important when running commands on a machine, and LLMs are not trustable. What else?

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

Even with these in place, the output of the LLM is not trustworthy. In the sense that it doesn't care what it true, only what is plausible.

Re: What does Alan Kay think about LLMs?

#23

Post by layer8 »

theGnuMe wrote: post #8
layer8 wrote: post #5

Earlier quoted context omitted.

He always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.

True message passing is one of the key ideas behind smalltalk which would improve computer security considerably. This is analogous to the design of the internet that he references. The internet scales well. So imagine building entire computing systems this way.

How would it improve it, though? Dynamically-typed languages tend to introduce gradual typing after a while to increase reliability and predictability, and the end game of that (static typing) is the antithesis of late binding. Message passing only improves security by adding a validation layer, but managed languages (JVM, CLR, etc.) effectively do the same.

> The internet scales well. So imagine building entire computing systems this way.

That’s very similar to microservices, and look how messy these tend to get. The internet scales well because it’s a network with flexible capacity and attached processing power, and because its clients are either human users who can deal with failure modes, or software with stable and well-specified point-to-point protocols.

Re: What does Alan Kay think about LLMs?

#24

Post by Xelynega »

layer8 wrote: post #16
sebastianconcpt wrote: post #10

Earlier quoted context omitted.

He did tho. He mentioned more than once that OOP was more about message passing than behavior + state.

Yes, that’s what I mean by vague suggestions. How do you actually build and maintain a system of any complexity with that? How do you ensure it will do what it is supposed to do? And Smalltalk wasn’t that, it is mostly just regular method calls and not “message passing”.

Erlang(and more modernly elixir/Gleam) are an example of this in practice, and were created to make it easier to ensure the working and manage the complexity of large systems.

Re: What does Alan Kay think about LLMs?

#25

Post by mempko »

I see a lot of people here are missing his "big deal" which he talks about in the end where he references the "Spaceship Earth" problem.

What I believe he is getting at is people are going to use LLMs to build systems at scale to further strip mine society.

The "Spaceship Earth" problem is a reference to Limits to Growth. For those who haven't read "Limits to Growth", and the more recent Re-calibration of Limits to Growth, I implore you to do so.

https://onlinelibrary.wiley.com/doi/full/10.1111/jiec.13442

Re: What does Alan Kay think about LLMs?

#26

Post by bongodongobob »

devjab wrote: post #18

I use LLMs quite a lot to help me in my work, but they are wrong so often that it’s ridiculous. This isn’t a major issue when you’re an expert using the tools to be more efficient, because you’ll spot and laugh at the errors it makes. Sometimes it’ll be things that anyone would notice, like how a LLM will simply “invent” a library function that has never existed. Even if you’re not an expert, you’re not going to get…

I'm sorry, but it sounds like you're using ChatGPT3.5. I get very very few invented libraries unless it's a very specific version of an old or rare language. I tried your "ING" example and GPT4 got 100/100.

Idk, ymmv, but GPT4 beats the pants off of 3.5.

Re: What does Alan Kay think about LLMs?

#27

Post by layer8 »

Xelynega wrote: post #24
layer8 wrote: post #16

Earlier quoted context omitted.

Yes, that’s what I mean by vague suggestions. How do you actually build and maintain a system of any complexity with that? How do you ensure it will do what it is supposed to do? And Smalltalk wasn’t that, it is mostly just regular method calls and not “message passing”.

Erlang(and more modernly elixir/Gleam) are an example of this in practice, and were created to make it easier to ensure the working and manage the complexity of large systems.

Fair enough. But Alan Kay doesn’t say “use Erlang”, nor does he go into any depths about what the desirable feature set is and what would make it work or would fail to make it work. He’s always just waxing around nebulous desiderata and complaining about the status quo.

Re: What does Alan Kay think about LLMs?

#28

Post by sumtechguy »

devjab wrote: post #18

I use LLMs quite a lot to help me in my work, but they are wrong so often that it’s ridiculous. This isn’t a major issue when you’re an expert using the tools to be more efficient, because you’ll spot and laugh at the errors it makes. Sometimes it’ll be things that anyone would notice, like how a LLM will simply “invent” a library function that has never existed. Even if you’re not an expert, you’re not going to get…

Yesterday I asked it the date. Then asked it if it was possible to give me the date. It gave a very different answer that contradicted what it just did. It is kind of hard to trust when it can not even make itself consistent between 2 prompts. For creative prompts this thing is very cool. For logical prompts it is frustrating to no end how much detail it gets wrong.

Re: What does Alan Kay think about LLMs?

#29

Post by ozten »

> That humans also do this all the time is “interesting”, “dangerous” etc., but it is also why trying to move from superstition (this is actually what “reasoning by correlation” amounts to) to more scientific methods is critical for anything like civilization to be created.

"reasoning by correlation" as superstition is a brutal insight.

Re: What does Alan Kay think about LLMs?

#30

Post by samatman »

bongodongobob wrote: post #26
devjab wrote: post #18

I use LLMs quite a lot to help me in my work, but they are wrong so often that it’s ridiculous. This isn’t a major issue when you’re an expert using the tools to be more efficient, because you’ll spot and laugh at the errors it makes. Sometimes it’ll be things that anyone would notice, like how a LLM will simply “invent” a library function that has never existed. Even if you’re not an expert, you’re not going to get…

I'm sorry, but it sounds like you're using ChatGPT3.5. I get very very few invented libraries unless it's a very specific version of an old or rare language. I tried your "ING" example and GPT4 got 100/100. Idk, ymmv, but GPT4 beats the pants off of 3.5.

I use GPT4 and it confabulates constantly. Try it on a language in the 10th-20th range of popularity (let alone lower) and I expect you'll see it as well. I don't see that as often with Python, the only top-5 language I currently work with.
Post reply on HN