Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Life is more than an engineering problem

lareviewofbooks.org

Post reply on HN
Print view View original

171–180 of 257 posts

Re: Life is more than an engineering problem

#171

Post by acchow »

weego wrote: post #125
abecedarius wrote: post #68

Chiang makes some insightful points, e.g. about what we mean by magic. Then I come to > [LLMs] can get better at reproducing patterns found online, but they don’t become capable of actual reasoning; it seems that the problem is fundamental to their architecture. and wonder how an intelligent person can still think this, can be so absolute about it. What is "actual" reasoning here? If an AI proves a theorem is it only…

Counter point: what is it about scraping the Internet and indexing it cleverly that makes you believe that would lead to the the creation of the ability to reason above it's programming? No one in neuroscience, psychology or any related field can point to reasoning or 'consciousness' or whatever you wish to call it and say it appeared from X. Yet we have this West Coast IT cultish thinking that if we throw money at i…

The assumption is that since there is already a neural network that “got there” (our brains), we should be able to achieve the same thing synthetically.

We just need to figure out how to train that network.

Re: Life is more than an engineering problem

#172

Post by fatbird »

rcxdude wrote: post #136
weego wrote: post #125

Earlier quoted context omitted.

Counter point: what is it about scraping the Internet and indexing it cleverly that makes you believe that would lead to the the creation of the ability to reason above it's programming? No one in neuroscience, psychology or any related field can point to reasoning or 'consciousness' or whatever you wish to call it and say it appeared from X. Yet we have this West Coast IT cultish thinking that if we throw money at i…

It's more that if you actually work with LLMs they will display reasoning. It's not particularly good or deep reasoning (I would generally say they have a superhuman amount of knowledge but are really quite unintelligent), but it is more than simply recall.

Are they displaying reasoning, or the outcome of reasoning, leading you to a false conclusion?

Personally, I see ChatGPT say "water doesn't freeze at 27 degrees F" and think "how can it possibly do advanced reasoning when it can't do basic reasoning?"

Re: Life is more than an engineering problem

#173

Post by TeMPOraL »

corimaith wrote: post #163
TeMPOraL wrote: post #99

Earlier quoted context omitted.

> If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value? No, but you described a `cp` command, not an LLM. "Creativity" in the sense of coming up with something new is trivial to implement in comp…

>Creativity" in the sense of coming up with something new is trivial to implement in computers, and has long been solved. Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity. Formal Proof Systems aren't even nearly close to completion, and for patterns we don't have a strong enough formal system to fully represent the problem space. If we take the P=NP problem, that likely…

Whatever the underlying "real" pattern is, doesn't really matter. We don't need to represent it. People learn to understand it implicitly, without ever seeing some formal definition spelled out - and learn it well enough that if you take M works to classify as "creative" or "not", then pick N people at random and ask each of them to classify each of the works, you can expect high degree of agreement.

LLMs aren't leaning what "creativity" is from first principles. They're learning it indirectly, by being trained to reply like a person would, literally, in the fully general meaning of that phrase. The better they get at that in general, the better they get at the (strict) subtask of "judging whether a work is creative the same way a human would" - and also "producing creative output like a human would".

Will that be enough to fully nail down what creativity is formally? Maybe, maybe not. On the one hand, LLMs don't "know" any more than we do, because whatever the pattern they learn, it's as implicit in their weights as it is for us. On the other hand, we can observe the models as they learn and infer, and poke at their weights, and do all kinds of other things that we can't do to ourselves, in order to find and understand how the "deeper superstructure behind these problems" gets translated into abstract structures within the model. This stands a chance to teach us a lot about both "these problems" and ourselves.

EDIT:

One could say there's no a priori reason why those ML models should have any structural similarity to how human brains work. But I'd say there is a reason - we're training them on inputs highly correlated with our own thoughts, and continuously optimizing them not just to mimic people, but to be bug for bug compatible with them. In the limit, the result of this pressure has to be equivalent to our own minds, even if not structurally equivalent. Of course the open question is, how far can we continue this process :).

Re: Life is more than an engineering problem

#174

Post by [deleted] »

kmmlng wrote: post #169
d4mi3n wrote: post #142

Earlier quoted context omitted.

Waters are often muddied here by our own psychology. We (as a species) tend to ascribe intelligence to things that can speak. Even more so when someone (or thing in this case) can not just speak, but articulate well. We know these are algorithms, but how many people fall in love or make friends over nothing but a letter or text message? Capabilities for reasoning aside, we should all be very careful of our perception…

This seems like the classic shifting of goalposts to determine when AI has actually become intelligent. Is the ability to communicate not a form of intelligence? We don't have to pretend like these models are super intelligent, but to deny them any intelligence seems too far for me.

[deleted]

Re: Life is more than an engineering problem

#175

Post by vhantz »

suddenexample wrote: post #131
abecedarius wrote: post #68

Chiang makes some insightful points, e.g. about what we mean by magic. Then I come to > [LLMs] can get better at reproducing patterns found online, but they don’t become capable of actual reasoning; it seems that the problem is fundamental to their architecture. and wonder how an intelligent person can still think this, can be so absolute about it. What is "actual" reasoning here? If an AI proves a theorem is it only…

I think the "LLM is intelligence" crowd has a very simplistic view of people. If you feel that natural language and the systems responsible it are pretty much the only things that human intelligence produces, then I can see the argument. But I don't believe that. That a machine that can produce convincing human-language chains of thought says nothing about its "intelligence". Back when basic RNNs/LSTMs were at the fo…

> It's surprising to me that the people most knowledgeable about the models often appear to be the biggest believers - perhaps they're self-interestedly pumping a valuation or are simply obsessed with the idea of building something straight from the science fiction stories they grew up with.

"Believer" really is the most appropriate label here. Altman or Musk lying and pretending they "AGI" right around the corner to pump their stocks is to be expected. The actual knowledgeable making completely irrational claims is simply incomprehensible beyond narcissism and obscurantism.

Interestingly, those who argue against the fiction that current models are reasoning, are using reason to make their points. A non-reasoning system generating plausible text is not at all a mystery can be explained, therefore, it's not sufficient for a system to generate plausible text to qualify as reasoning.

Those who are hyping the emergence of intelligence out of statistical models of written language on the other hand rely strictly on the basest empiricism, e.g. "I have an interaction with ChatGPT that proves it's intelligent" or "I put your argument into ChatGPT and here's what it said, isn't that interestingly insightful". But I don't see anyone coming out with any reasoning on how ability to reason could emerge out of a system predicting text.

There's also a tacit connection made between those language models being large and complex and their supposed intelligence. The human brain is large and complex, and it's the material basis of human intelligence, "therefore expensive large language models with internal behavior completely unexplainable to us, must be intelligent".

I don't think it will, but if the release of the deepseek models effectively shifts the main focus towards efficiency as opposed to "throwing more GPUs at it", that will also force the field to produce models with the current behavior using only the bare minimum, both in terms of architecture and resources. That would help against some aspects of the mysticism.

The biggest believers are not the best placed to drive the research forward. They are not looking at it critically and trying to understand it. They are using every generated sentence as a confirmation of their preconceptions. If the most knowledgeable are indeed the biggest believers, we are in for a long dark (mystic) AI winter.

Re: Life is more than an engineering problem

#176

Post by layer8 »

asdf6969 wrote: post #155
ChrisKnott wrote: post #14

> "It’s like imagining that a printer could actually feel pain because it can print bumper stickers with the words ‘Baby don’t hurt me’ on them. It doesn’t matter if the next version of the printer can print out those stickers faster, or if it can format the text in bold red capital letters instead of small black ones. Those are indicators that you have a more capable printer but not indicators that it is any closer…

> You need to really get into the weeds of what "actually feeling" means. We don’t even know what this means when it’s applied to humans. We could explain what it looks like in the brain but we don’t know what causes the perception itself. Unless you think a perfect digital replica of a brain could have an inner sense of existence Since we don’t know what “feeling” actually is there’s no evidence either way that a co…

> I will never believe it’s possible for an LLM to feel.

Why is that, given that, as you state, we don’t know what “feeling” actually is?

Re: Life is more than an engineering problem

#177

Post by daveguy »

kmmlng wrote: post #169
d4mi3n wrote: post #142

Earlier quoted context omitted.

Waters are often muddied here by our own psychology. We (as a species) tend to ascribe intelligence to things that can speak. Even more so when someone (or thing in this case) can not just speak, but articulate well. We know these are algorithms, but how many people fall in love or make friends over nothing but a letter or text message? Capabilities for reasoning aside, we should all be very careful of our perception…

This seems like the classic shifting of goalposts to determine when AI has actually become intelligent. Is the ability to communicate not a form of intelligence? We don't have to pretend like these models are super intelligent, but to deny them any intelligence seems too far for me.

AI certainly won't be intelligent while it has episodic responses to queries with no ability to learn from or even remember the conversation without it being fed back through as context. This is the current case for LLM models. Token prediction != Intelligence no matter how intelligent it may seem. I would say adaptability is a fundamental requirement of intelligence.

Re: Life is more than an engineering problem

#178

Post by hollerith »

daveguy wrote: post #177
kmmlng wrote: post #169

Earlier quoted context omitted.

This seems like the classic shifting of goalposts to determine when AI has actually become intelligent. Is the ability to communicate not a form of intelligence? We don't have to pretend like these models are super intelligent, but to deny them any intelligence seems too far for me.

AI certainly won't be intelligent while it has episodic responses to queries with no ability to learn from or even remember the conversation without it being fed back through as context. This is the current case for LLM models. Token prediction != Intelligence no matter how intelligent it may seem. I would say adaptability is a fundamental requirement of intelligence.

>AI certainly won't be intelligent while it has episodic responses to queries with no ability to learn from or even remember the conversation without it being fed back through as context.

Thank God no one at the AI labs is working to remove that limitation!

Re: Life is more than an engineering problem

#179

Post by daveguy »

TeMPOraL wrote: post #99
sleepytree wrote: post #85

Earlier quoted context omitted.

If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value? I think what he is trying to say is that LLMs current architecture seems to mainly work by understanding patterns in the existing body of kno…

> If I write a book that contains Einstein's theory of relativity by virtue of me copying it, did I create the theory? Did my copying of it indicate anything about my understanding of it? Would you be justified to think the next book I write would have anything of original value? No, but you described a `cp` command, not an LLM. "Creativity" in the sense of coming up with something new is trivial to implement in comp…

> Take some pattern - of words, of data, of thought. Perturb it randomly. Done. That's creativity.

This seems a miopic view of creativity. I think leaving out the pursuit of the implications of that perturbation is leaving out the majority of creativity. A random number generator is not creative without some way to explore the impact of the random number. This is something that LLM inference models just don't do. Feeding previous output into the context of a next "reasoning" step still depends on a static model at the core.

Re: Life is more than an engineering problem

#180

Post by layer8 »

emtel wrote: post #109
emtel wrote: post #86

Earlier quoted context omitted.

I’ll try to dig some up soon (I’m on my phone now). But of course the output contains errors sometimes. So do search engine results. The important thing for difficult questions is whether the right answer (or something pointing toward it) is available _at all_. Of course this assumes you can verify the answers somehow (usually easy with programming questions), but again, search engines have the same limitation.

Ok, the first example I found was when I was trying to find a way to write a rust proc macro that recursively processes functions or modules and re-writes arithmetic expressions. The best way to do this, it turns out, is with `VisitMut` or `fold`. I cannot find any results discussing these approaches with google, but ChatGPT (4) suggested it within the first couple refinements of a query. Another recent example from…

I found the following as the first Google result for “rust proc macro AST rewrite” (and I don’t know much about Rust”): https://users.rust-lang.org/t/using-macros-to-modify-ast-to-...

And I found the following for “different future same rust struct” (first search attempt): https://stackoverflow.com/questions/65587187/how-to-instanti...

I’m not saying that LLMs can’t be useful for stuff like that, but they haven’t been that much of an improvement over Google search so far. And I always google about what an LLM suggests in any case, to verify and to get a better feeling about the real-world state of the topic in question.

Post reply on HN