Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talking About Large Language Models

arxiv.org

Post reply on HN
Print view View original

101–110 of 158 posts

Re: Talking About Large Language Models

#101

Post by xg15 »

wellpast wrote: post #99
xg15 wrote: post #98

I mean, if you accept the assumption that consciousness is biological (so there is no soul or other spiritual or metaphysical entity), then there is some algorithm or processing model that produces genuine consciousness: The one that takes place in our brains. The question remains if this processing model would be in any way similar to the processing model that LLMs use - and yes, we can probably rule that out pretty…

You're making an assumption out of thin air here: that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. What's your basis for that assumption other than just asserting it?

> that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something.

Yeah, I genuinely only see those two possibilities: Either consciousness is somehow the result of the interactions of (some of) the billions of neurons in our brain - or it isn't. If it isn't, either consciousness is a product of some other biological or physical* process that so far isn't identified - or it isn't. And if it's not the result of any physical process then I don't see what would be left except metaphysics.

We don't know as of now what consciousness is or how it is produced - but I think there are some strong hints that in fact it is produced by the interactions of our neurons. Mainly that we can directly influence consciousness with psychoactive drugs - which don't do anything interesting physics-wise except affecting how neurons exchange signals - and that we can also observe certain patterns of physical activity in the brain using EEG, MRI and other technologies and map this activity to mental tasks like concentrating or relaxing.

Edit: * I used "psychological" there but that was a typo. I meant "physical".

Re: Talking About Large Language Models

#102

Post by hnnotreddit »

oneoneonetwo wrote: post #100
antonvs wrote: post #88

Earlier quoted context omitted.

You’re looking for someone to argue with, but your arguments are trivial and meritless. You’re not a scientist and never have been. The fact that you’re posting with a throwaway account created an hour ago says it all. HN should not allow people like you to post.

Quoted post unavailable.

You fundamentally misunderstand the scientific method which includes falsifiability but is not limited to it, moved the goalpost away from peer review for journals into general bar stool philosophy all while continuously insulting the GP.

What is ironic is you hiding behind throwaway accounts and asserting superior knowledge while fundamentally misunderstanding the issue of positive claims in science.

Re: Talking About Large Language Models

#103

Post by xg15 »

fourfivefour wrote: post #72
Jack000 wrote: post #37

Earlier quoted context omitted.

LLMs may be overhyped, but transformers in general are under hyped. LLMs make a lot of mistakes because they don't actually know what words mean. The key thing is though - it's much harder to generate coherent text when you don't know what the words mean. In a similar vein it's completely unreasonable to expect an LLM to perform visual tasks when it literally has no sense of sight. The fact that it can kind of sort o…

How can these things not know what words mean? Did you not see how they created a virtual machine under chatGPT? They told it to imitate bash and they typed ls, and cat jokes.txt and it outputted things completely identical to what you'd expect. Look it up. https://www.engraved.blog/building-a-virtual-machine-inside/ I don't see how you can explain this as not knowing what words mean. It KNOWS.

Yeah, that's the actual bit that baffles me about ChatGPT still. Producing coherent, fluent text is alright, but we could already sort of do that 20 years ago with markov models or even just grammars (see Chomsky).

Understanding text in the depth that ChatGPT (and GPT-3) appear to understand the prompts is something entirely different and has to my knowledge never been archieved before the current architectures.

Re: Talking About Large Language Models

#104

Post by wellpast »

xg15 wrote: post #101
wellpast wrote: post #99

Earlier quoted context omitted.

You're making an assumption out of thin air here: that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. What's your basis for that assumption other than just asserting it?

> that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. Yeah, I genuinely only see those two possibilities: Either consciousness is somehow the result of the interactions of (some of) the billions of neurons in our brain - or it isn't. If it isn't, either consciousness is a product of some other biological or physical* process that so far isn't identifi…

> that so far isn't identified ... then I don't see what would be left except metaphysic

So out of all "that so far isn't identified", there are just two classes of things: (i) biological and psychological processes; and (ii) metaphysics. This is some magic voodoo hand-waving here. Metaphysics seems to be just your way of saying "everything we don't know or understand".

Re: Talking About Large Language Models

#105

Post by xg15 »

wellpast wrote: post #104
xg15 wrote: post #101

Earlier quoted context omitted.

> that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. Yeah, I genuinely only see those two possibilities: Either consciousness is somehow the result of the interactions of (some of) the billions of neurons in our brain - or it isn't. If it isn't, either consciousness is a product of some other biological or physical* process that so far isn't identifi…

> that so far isn't identified ... then I don't see what would be left except metaphysic So out of all "that so far isn't identified", there are just two classes of things: (i) biological and psychological processes; and (ii) metaphysics. This is some magic voodoo hand-waving here. Metaphysics seems to be just your way of saying "everything we don't know or understand".

I think "physical process" already covers a lot of ground besides metaphysics. My main distinction was that anything physical is governed by the laws of physics that we know and that are generally deterministic.

In contrast, metaphysics would be something entirely out of our realm of reality - which was how consciousness has been generally seen for a long time.

But what other possibility would you see?

Edit:

> (i) biological and psychological processes

Sorry, I had mixed up the word there. I was meaning "physical process", not "psychological". I had edited the other post to fix the word.

Re: Talking About Large Language Models

#106

Post by jungturk »

wellpast wrote: post #99
xg15 wrote: post #98

I mean, if you accept the assumption that consciousness is biological (so there is no soul or other spiritual or metaphysical entity), then there is some algorithm or processing model that produces genuine consciousness: The one that takes place in our brains. The question remains if this processing model would be in any way similar to the processing model that LLMs use - and yes, we can probably rule that out pretty…

You're making an assumption out of thin air here: that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. What's your basis for that assumption other than just asserting it?

> if you accept the assumption that consciousness is biological

The poster wasn't making that assumption - they're stating the implications of those who do.

Re: Talking About Large Language Models

#107

Post by xg15 »

jungturk wrote: post #106
wellpast wrote: post #99

Earlier quoted context omitted.

You're making an assumption out of thin air here: that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. What's your basis for that assumption other than just asserting it?

> if you accept the assumption that consciousness is biological The poster wasn't making that assumption - they're stating the implications of those who do.

Yeah, but if isn't biological, then what exactly is it?

Re: Talking About Large Language Models

#108

Post by doliveira »

wellpast wrote: post #104
xg15 wrote: post #101

Earlier quoted context omitted.

> that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. Yeah, I genuinely only see those two possibilities: Either consciousness is somehow the result of the interactions of (some of) the billions of neurons in our brain - or it isn't. If it isn't, either consciousness is a product of some other biological or physical* process that so far isn't identifi…

> that so far isn't identified ... then I don't see what would be left except metaphysic So out of all "that so far isn't identified", there are just two classes of things: (i) biological and psychological processes; and (ii) metaphysics. This is some magic voodoo hand-waving here. Metaphysics seems to be just your way of saying "everything we don't know or understand".

Something that happens in our universe that Physics somehow has nothing to do with? How is that not magical/spiritual/metaphysical voodoo handwaving?

Re: Talking About Large Language Models

#109

Post by xg15 »

gillesjacobs wrote: post #80
oneoneonetwo wrote: post #75

Earlier quoted context omitted.

> In science, if you don't know, you don't make the claim, that is basic positivism and the scientific method. Yes you're correct. So you can't make the claim that it's NOT cognition. That is my point. You also can't make the claim that it is cognition which was the OTHER point. Completely agree with your statement here. But it goes further then this, and your statement shows YOU don't understand science. >So basic i…

Let me falsify your claim immediately: the inputs of these models are nothing like the inputs a human receives, subword tokens do not even match up with lexical items (visually, textually and semantically). You seem to agree with me even though your interpretation of falsifiability is inverted: I am not asking that authors make a claim that their models do not mimick human intelligence. Like OP, I ask them that they…

The inputs and outputs for humans are synaptic action potentials, neurotransmitters and hormones. Is that so different?

Also agreeing that current LLMs are probably not sentient in any meaningful way. But what I don't like about the discussion is is that it steers in a direction where it would fundamentally be seen as bad science to claim that any AI model could be conscious - and I don't see that covered by the scientific method either, for the reasons that onetwoonetwo explained.

This reminds me a bit of the discussion whether or not animals can be conscious/experience emotions/feel pain, etc.

Re: Talking About Large Language Models

#110

Post by godelski »

wellpast wrote: post #99
xg15 wrote: post #98

I mean, if you accept the assumption that consciousness is biological (so there is no soul or other spiritual or metaphysical entity), then there is some algorithm or processing model that produces genuine consciousness: The one that takes place in our brains. The question remains if this processing model would be in any way similar to the processing model that LLMs use - and yes, we can probably rule that out pretty…

You're making an assumption out of thin air here: that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something. What's your basis for that assumption other than just asserting it?

> that consciousness is somehow produced or emergent, or that if it isn't you're in the realm of spiritual or something.

I'm reading the parent's comment as "Either consciousness is super natural or is created from nature." I don't think this is an unreasonable assumption (literally "all things are either hot dogs, or not hot dogs", it is simple but not unreasonable). If you have a third possibility I'd like to know what that is.

Supposing that consciousness (whatever it is) isn't super natural it follows that we can create it. It does not follow that it will be easy or done anytime soon (or even in the distant future), but possible nonetheless. Unless you have reason to believe that a physical process cannot be reproduced.

Post reply on HN