Jeff Bezos on AI (1998) [video] • Hacker News Board

Re: Jeff Bezos on AI (1998) [video]

#71

Post by jmopp » Tue, Aug 22, 2023, 7:57 AM UTC

BoiledCabbage wrote: post #14

mholt wrote: post #5

The common sentiment around AI in the 90s and early 2000s was that it didn't work; it had its hype, it had its heyday, but it seemed like a dead-end for the most part. The Perceptron was merely a linear function approximator. And the Multi-layer Perceptron was a little more capable, but the many orders of magnitude it would have to scale up in order to be convincing just wasn't feasible back then (it finally was in t…

> That's what ultimately depresses me about AI. It's still just a parlor trick. We haven't actually taught computers to think, to reason, to be innovative. And what do you feel when we make these parlor tricks more capable than us at the majority of tasks? And what do you feel when we understand it well enough to realize we're the same type of parlor tricks? To me it seems like you're most interested in a magic 'aha'…

Computers are already better than humans at a wide variety of tasks. Text generation just happens to now be one of those tasks. But if you look at the prompt -> output -> prompt feedback loop, it's clear that the human submitting the prompts is still doing all the thinking. We're not yet at the point where the AI can prompt itself and improve its output in a logical manner.

Re: Jeff Bezos on AI (1998) [video]

#72

Post by IanCal » Tue, Aug 22, 2023, 8:00 AM UTC

corethree wrote: post #64

oldandtired wrote: post #62

Earlier quoted context omitted.

You make a claim here with "Each answer displayed astonishing understanding of what occurs." and the question you fail to ask is: Whose understanding? The responses are based on the accumulated knowledge of humans and not machines. The systems have not thought through anything and understand nothing. A process of analysing or pattern matching the input question with the data stored retrieves an answer. But that data…

> You make a claim here with "Each answer displayed astonishing understanding of what occurs." and the question you fail to ask is: Whose understanding? The answer is obvious. The LLM is understanding the concepts. The last question was unique. The resulting answer was also unique. It was not a "retrieved" answer. It was a unique answer. A correct composition of several underlying concepts. A correct composition can…

It's generating code for a brand new library based on explanations from me, it can write poems about the current news headlines and it can answer hypotheticals with words I've made up. I agree it cannot be just looking up stored answers.

Gpt Othello is a good discussion about this that's more constrained too.

Re: Jeff Bezos on AI (1998) [video]

#73

Post by oldandtired » Tue, Aug 22, 2023, 8:20 AM UTC

vczf wrote: post #48

flatline wrote: post #26

Earlier quoted context omitted.

There is a famous Dijkstra quote, “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.” Do the intrinsic properties of the system really matter at the end of the day if it performs as well as we do at some task? Heck they’ve been doing many things better for decades, but those are the types of tasks we take it for granted that a machine should be able…

I wrote about my definition of intelligence earlier this month: https://tildes.net/~comp/194n/language_is_a_poor_heuristic_f... I have a definition of intelligence. [...] Intelligence is prediction. In the case of intelligent living processes ranging from single celled organisms to complex multicellular life, intelligence arises from the need to predict the future to survive and reproduce. More intelligent organisms…

Your definition of intelligence has been around for millennia and is part of the pantheism concept.

Re: Jeff Bezos on AI (1998) [video]

#74

Post by prmoustache » Tue, Aug 22, 2023, 8:24 AM UTC

mholt wrote: post #5

The common sentiment around AI in the 90s and early 2000s was that it didn't work; it had its hype, it had its heyday, but it seemed like a dead-end for the most part. The Perceptron was merely a linear function approximator. And the Multi-layer Perceptron was a little more capable, but the many orders of magnitude it would have to scale up in order to be convincing just wasn't feasible back then (it finally was in t…

> , but they're still just that: parlor tricks

It isn't that much different than human behaviors.

People tend to repeat stuff they have seen done by our parents, sibling, friends, medias. Listen to people smalltalking in the streets, repeating the same things all over again every day. The easy success of marketing, politicians, dictators. The power of marketing and success of consumption society. Racism, bigotry, religions, addictions. All these are easily explained because people barely think. They just respond to internal and external stimulus with recipes they have been taught to follow without giving a second thought.

Re: Jeff Bezos on AI (1998) [video]

#75

Post by RandomLensman » Tue, Aug 22, 2023, 8:24 AM UTC

corethree wrote: post #50

mholt wrote: post #5

The common sentiment around AI in the 90s and early 2000s was that it didn't work; it had its hype, it had its heyday, but it seemed like a dead-end for the most part. The Perceptron was merely a linear function approximator. And the Multi-layer Perceptron was a little more capable, but the many orders of magnitude it would have to scale up in order to be convincing just wasn't feasible back then (it finally was in t…

The parlor tricks are getting better. In terms of practicality you just need to look at AI art. That stuff is good enough to replace the real thing. In terms of actual sentience, understanding I had chatGPT answer questions in the following order: 1. Describe to me what happens when you throw sugar in coffee 2. Describe the same thing in terms of atoms. 3. Is this an example of entropy decreasing? 4. What if the arro…

It sounds like understanding, but not sure because it already doesn't consider if sugar (or water) molecules could form/exist in the alternate universe (also ignores trading off energy vs entropy changes) or should the water just crystalize in one half of the cup (why isng that considered)?. I don't think one can say it has really though about the problem. To be fair, language is probably not the tool to analyse the problem but mathematics are.

It is a bit like technobabble.

Re: Jeff Bezos on AI (1998) [video]

#76

Post by worksonmine » Tue, Aug 22, 2023, 8:44 AM UTC

mholt wrote: post #5

The common sentiment around AI in the 90s and early 2000s was that it didn't work; it had its hype, it had its heyday, but it seemed like a dead-end for the most part. The Perceptron was merely a linear function approximator. And the Multi-layer Perceptron was a little more capable, but the many orders of magnitude it would have to scale up in order to be convincing just wasn't feasible back then (it finally was in t…

> That's what ultimately depresses me about AI. It's still just a parlor trick. We haven't actually taught computers to think, to reason, to be innovative.

If you think about it we've gone the other way. We're teaching/conditioning humans to think less and react more. This has only gotten worse the last few years and I don't see any shift coming soon. Humanity unfortunately seems just as simple as the algorithm, use the right inputs, in the right context and you can make most of us act just like you want.

Re: Jeff Bezos on AI (1998) [video]

#77

Post by oldandtired » Tue, Aug 22, 2023, 8:48 AM UTC

ben_w wrote: post #65

oldandtired wrote: post #57

Earlier quoted context omitted.

It is impossible to enumerate all the things that we humans do. However, we can enumerate all the things that we create can do. Every system we create has its limitation due to the limitations that we create in them. All systems we create cannot exceed those limitations. We make machines that are stronger, faster, and can have much finer motor control than we have as individual abilities. No machine we have created h…

Mostly I agree with you, but > However, we can enumerate all the things that we create can do. Not really, no. Even before AI, "Turing Complete" makes things extremely hard to enumerate; see Busy Beaver numbers for how small a system can be and still outside our ability to fully comprehend — needing to use up-arrow notation because exponentials aren't big enough is always good for a laugh.

With your example of "Turing Complete", we know what cannot be done and in this way, we have enumerated the things that can be done, if you like. You appreciate the humour required for the up-arrow notation - a very human quality.

You example of the Busy Beaver numbers, which was a recent interesting read, is a good example of what I was trying to point out. We have a definition and even if we cannot enumerate each number, we discuss and think about these in a rational way. At the moment, I am quite interested in Computer Algebra Systems (of which there are a variety) and I find it interesting just how limited these systems are and just how difficult it is to program into them the capabilities that humans use to solves various problems. The various discussions have been quite enlightening.

Mathematics is an interesting subject and I think shows up the intractability of ever getting that highly feared singularity.

All artificial computing systems are limited in ways we are not. Your "Turing Machine" example is one such case. The Halting Problem being a class example.

I think that far too often, we fail to recognise that what we create is not that great. We often stand in awe of the things we make without comprehending that these things are a very poor reflection of what is around us and what we ourselves are.

Every time some hype comes about these artificial stupidity systems, I look at my youngest granddaughter and see in her, capabilities that far exceed anything that we have created. Even my old buck of a goat demonstrates capabilities far, far in excess of anything we have created in all of our computational systems.

As I have said elsewhere here, we have to be careful that we do not cede control of our lives to systems that we think are more than they really are - systems that are limited, fragile and prone to failure.

Re: Jeff Bezos on AI (1998) [video]

#78

Post by ben_w » Tue, Aug 22, 2023, 8:57 AM UTC

oldandtired wrote: post #77

ben_w wrote: post #65

Earlier quoted context omitted.

Mostly I agree with you, but > However, we can enumerate all the things that we create can do. Not really, no. Even before AI, "Turing Complete" makes things extremely hard to enumerate; see Busy Beaver numbers for how small a system can be and still outside our ability to fully comprehend — needing to use up-arrow notation because exponentials aren't big enough is always good for a laugh.

With your example of "Turing Complete", we know what cannot be done and in this way, we have enumerated the things that can be done, if you like. You appreciate the humour required for the up-arrow notation - a very human quality. You example of the Busy Beaver numbers, which was a recent interesting read, is a good example of what I was trying to point out. We have a definition and even if we cannot enumerate each n…

> All artificial computing systems are limited in ways we are not. Your "Turing Machine" example is one such case. The Halting Problem being a class example.

You appear to be asserting that humans can tell if a loop will end, when that loop is defined so that if it does it doesn't and if it doesn't it does.

> Even my old buck of a goat demonstrates capabilities far, far in excess of anything we have created in all of our computational systems.

How so?

Not saying this is necessarily false — GPT-3 is about as complex as the brain of a rodent, so it wouldn't exactly be surprising even though the LLM only does text and completely different AI do other things — but still, what exactly do goats do that's "far in excess"?

Re: Jeff Bezos on AI (1998) [video]

#79

Post by worksonmine » Tue, Aug 22, 2023, 9:25 AM UTC

IanCal wrote: post #67

oldandtired wrote: post #62

Earlier quoted context omitted.

You make a claim here with "Each answer displayed astonishing understanding of what occurs." and the question you fail to ask is: Whose understanding? The responses are based on the accumulated knowledge of humans and not machines. The systems have not thought through anything and understand nothing. A process of analysing or pattern matching the input question with the data stored retrieves an answer. But that data…

> A process of analysing or pattern matching the input question with the data stored retrieves an answer. They're not just retrieving stored text like pulling the most relevant passage from a database. If they were they'd not be able to deal with things outside the training set. They couldn't write code for a custom library that was created after the cutoff (they can with a description), and they couldn't write about…

I don't see why not. It's not taking a single answer from a database no, it's taking several based on probability and merging them into what it thinks we're looking for. If you learn to multiply with code to perform one task, you can then apply that knowledge for a completely different task. It may look like solving a completely new problem but the LLM doesn't even see the difference.

When you use the term "custom library" that might be you over-complicating the task. It's still just looking up function to do x, function to do y and applying it to the output. Don't get me wrong it's impressive where we're at but there's no need to exaggerate it as magic.

Re: Jeff Bezos on AI (1998) [video]

#80

Post by worksonmine » Tue, Aug 22, 2023, 9:43 AM UTC

tsunamifury wrote: post #30

KirillPanov wrote: post #16

Earlier quoted context omitted.

This is why they've called it AI winter the past three times. It's a season. Like the seasons, the cycle repeats.

What a tired and lame take. 100m people used this latest iteration. It’s hardly a winter.

If there's anything the last few years has taught me is that hype is rarely any indication of common sense. I'm always surprised what reaches success and what doesn't. Truly revolutionary ideas are ignored and not understood while stupid but polished ideas are booming.

One example of that is the crypto space. So many actual good for humanity implications, but the "killer app" that made the news was fucking NFTs. I'm trying really hard not to come to the conclusion that humans are just mindless zombies but it's getting harder every year.

Once we reach a billion DAU you can be sure it will just be "Original Memes Tailored Just For YOU"™ instead of figuring out the logistics for solving world hunger. Mark my words.