Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does Alan Kay think about LLMs?

quora.com

Post reply on HN
Print view View original

11–20 of 155 posts

Re: What does Alan Kay think about LLMs?

#11

Post by utopcell »

altruios wrote: post #7
dpflan wrote: post #4

There is a lot in here, various paths to venture off, but the bottom line seems to be trust is important when running commands on a machine, and LLMs are not trustable. What else?

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

Open sourcing would only guard against human programmers. The fact of the matter is, we don't _really_ know why LLMs work.

Re: What does Alan Kay think about LLMs?

#13

Post by vajrabum »

theGnuMe wrote: post #8
layer8 wrote: post #5

Earlier quoted context omitted.

He always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.

True message passing is one of the key ideas behind smalltalk which would improve computer security considerably. This is analogous to the design of the internet that he references. The internet scales well. So imagine building entire computing systems this way.

If smalltalk is late binding and message passing, I think it's interesting that modern large scale webapps (say Netflix or Google) do just that just not inside process and server boundaries. I'd guess that Alan Kay would think that's a bastardization, and there are loads of complications but it's a definite improvement over the large php ball of mud monolith you might have seen at say Box or Facebook 10-15 years ago. Because the 'object' boundaries are so coarse you can also better leverage the few people inside an org who know how to design, scale, improve robustness and secure systems. If architecture is the metaphor, it's still pretty much piling mud bricks but at least there are corbeled arches, streets, city walls, public buildings and a rudimentary notion of style.

As for LLMs I think that long form typeahead is the right way to think about it.

Re: What does Alan Kay think about LLMs?

#14

Post by mempko »

layer8 wrote: post #5
skadamat wrote: post #2

This answer is amazing and classically Alan Kay! There's so much here to unpack because of all the different areas Alan draws from in his work (he's like a computing philosopher ). All I will say is that for people who want to understand his perspective, there's a large epistemological load to overcome. Sampling his talks is a good starting point though: https://tinlizzie.org/IA/index.php/Talks_by_Alan_Kay

He always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.

What? He literally built systems that did things differently. Look at smalltalk.

Re: What does Alan Kay think about LLMs?

#15

Post by falsaberN1 »

altruios wrote: post #7
dpflan wrote: post #4

There is a lot in here, various paths to venture off, but the bottom line seems to be trust is important when running commands on a machine, and LLMs are not trustable. What else?

what would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they just...recombine text by probabilities and aren't deterministic. I get incorrect information every time I ask them a thing, and it's incorrect in different ways every time. The only things they seem to get consistently correct are very widespread facts that are much faster and trivial to google.

I still find it funny we managed to get image generation working so much better than text.

Re: What does Alan Kay think about LLMs?

#16

Post by layer8 »

sebastianconcpt wrote: post #10
layer8 wrote: post #5

Earlier quoted context omitted.

He always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.

He did tho. He mentioned more than once that OOP was more about message passing than behavior + state.

Yes, that’s what I mean by vague suggestions. How do you actually build and maintain a system of any complexity with that? How do you ensure it will do what it is supposed to do? And Smalltalk wasn’t that, it is mostly just regular method calls and not “message passing”.

Re: What does Alan Kay think about LLMs?

#17

Post by sebastianconcpt »

It is by definition that the silicon automatons cannot escape from their ontologically stochastic imitative hallucinations.

All they can do is be observed by us and with that expose us to get induced by association, ontologically human hallucinations and deal with its outcome of real consequences.

Re: What does Alan Kay think about LLMs?

#18

Post by devjab »

I use LLMs quite a lot to help me in my work, but they are wrong so often that it’s ridiculous. This isn’t a major issue when you’re an expert using the tools to be more efficient, because you’ll spot and laugh at the errors it makes. Sometimes it’ll be things that anyone would notice, like how a LLM will simply “invent” a library function that has never existed. Even if you’re not an expert, you’re not going to get that PNP-whatever function to work in Powershell if it never existed in the module to begin with.

Where it becomes more dangerous, at least in my opinion, is when the LLM only gets it sort of wrong. Maybe the answer it gives you is old, maybe it’s inefficient, maybe it’s insecure or a range of other things, and if you’re new to programming, you’re probably not going to notice. Hell, I’ve reviewed code from senior programmers that pulled in deprecated things with massive security vulnerabilities and never noticed because they were too focused on fast delivery and “it worked”. I can’t imagine how that would work out for people trying to actually learn things.

I’m not sure what we can really do about it though. I work a side gig as an external examiner for CS students. A lot of the curriculum being taught (at least here in Denmark) are things I’ve seen the industry move “beyond” in the previous 20 years. Some of it is so dated that it really makes no sense at all. Which isn’t exactly a great alternative to the LLMs, and it’s only natural that a lot of people simply turn to these powerful tools.

I tend to tell people to ask their favorite LLM to help them solve a crossword. When you ask it to give you words ending on “ing” it’ll give you words that don’t end on “ing” because of how the tokens used work. This tends to be an eye opener for people in regards to how much they trust their LLM. At least until they get refined enough that they can also do these things.

Anyway, it’s a good answer.

Re: What does Alan Kay think about LLMs?

#20

Post by layer8 »

detourdog wrote: post #6
layer8 wrote: post #5

Earlier quoted context omitted.

He always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.

I think trustability is pretty specific and without a foundation of trust the rest of the effort”s value is unknowable.

That’s “what we would like to have” (so mostly just a criticism that we don’t have it), but without much of a suggestion of how to actually achieve it in practice.
Post reply on HN