Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding ChatGPT

atmosera.com

Post reply on HN
Print view View original

151–160 of 241 posts

Re: Understanding ChatGPT

#151

Post by ak_111 »

alcover wrote: post #148
ak_111 wrote: post #142

Earlier quoted context omitted.

You basically landed on Chomsky's universal grammar. And this only proves the chatgpt critics: we have no idea what those priors are, how they evolved, why they are so effective and thus we are not even sure they exist. Until this is demonstrated I think it is very fair to say chatgpt is applying very different reasoning to what humans are applying. Also language is a fairly recent development in human evolution (onl…

60-70 generations ago More like 1000+ considering the Chauvet painters certainly had speech.

let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.

Re: Understanding ChatGPT

#152

Post by PartiallyTyped »

danans wrote: post #129
PartiallyTyped wrote: post #120

Earlier quoted context omitted.

I fail to see how the first is useful. For all intents and purposes your brain might as well be a Boltzmann brain / in a jar getting electrical stimuli. Your notion of reality is a mere interpretation of electrical signals / information. This implies that all such information can be encoded via language or whatever else. You also don’t take initiative. Every action that you take is dependent upon all previous actions…

> You merely call the outcome of your brain’s competing circuits as “taking initiative”. We give names to all kinds of outcomes of our brains competing circuits. But our brains competing circuits have evolved to solve a fundamentally different set of problems than an LLM was designed for: the problems of human survival. > A blind person has no notion of colour yet we don’t claim they are not sentient or generally int…

> We give names to all kinds of outcomes of our brains competing circuits. But our brains competing circuits have evolved to solve a fundamentally different set of problems than an LLM was designed for: the problems of human survival.

Our brain did not evolve to do anything. It happened that a scaled primate brain is useful for DNA propagation, that's it. The brain can not purposefully drive its own evolution just yet, and we have collectively deemed it unethical because a crazy dude used it to justify murdering and torturing millions.

If we are being precise, we are driving the evolution of said models based on their usefulness to us, thus their capacity to propagate and metaphorically survive is entirely dependent on how useful they are to their environment.

Your fundamental mistake is thinking that training a model to do xyz is akin to our brains "evolving". The better analogy would be that as a model is training by interactions to its environment, it is changing. Same thing happens to humans, it's just that our update rules are a bit different.

The evolution is across iterations and generations of models, not their parameters.

> should still not be used to justify the diminishment of anyone's humanity.

I am not doing that, on the contrary, I am elevating the models. The fact that you took it as diminishment of the human is not really my fault nor my intention.

The belief that elevating a machine or information to humanity is the reduction of some people's humanity or of humanity as a whole, is entirely your issue.

From my perspective, this only shows the sheer ingenuity of humans, and just how much effort it took for millions of humans to reach something analogous to us, and eventually build a potential successor to humanity.

Re: Understanding ChatGPT

#153

Post by isaacremuant »

maxdoop wrote: post #28

“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…

And I find this very dismissive top comments that seem to try to shun/silence any criticism, discussion or concern as "anti AI" are maddening to read as well.

Any criticism is met with "it'll get better, you MUST buy into the hype and draw all this hyperbolic conclusions or you're a luddite or a denier"

There's some great aspects and some fundamental flaws but somehow, we're not allowed to be very critical of it.

Hackernews looks very similar to Reddit nowadays. If you don't support whatever hype narrative there is, you must be "label".

It's not a simple discussion of "just add more tokens" or "It will get better".

Re: Understanding ChatGPT

#154

Post by jppittma »

maxdoop wrote: post #28

“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…

No matter how much you explain to somebody what an apple tastes like, they'll never be able to truly know without having experienced it. Language is reductive on experience.

Likewise, we have models like gravity that describe planetary motion. It is useful, but by nature of being a model, it's incomplete. Models are also reductive on experience.

Can you see then how a large language model, something that describes and predicts human language, is different than a human that uses language to communicate his experience?

Re: Understanding ChatGPT

#155

Post by alcover »

ak_111 wrote: post #151
alcover wrote: post #148

Earlier quoted context omitted.

60-70 generations ago More like 1000+ considering the Chauvet painters certainly had speech.

let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.

Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken).

I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree.

Re: Understanding ChatGPT

#156

Post by jvolkman »

Madmallard wrote: post #101
maxdoop wrote: post #73

Earlier quoted context omitted.

I completely agree with you. Note that I’m not making those claims about sentience and similarity. What I am pushing back on is the confidence with which proclaim humans are “so different”, when I’ve yet to see actual proof of this dissimilarity. I think it’s (understandably) an emotional response from folks to dismiss ideas around LLM progress because it FEELS like we are thereby lessening what it means to be human.…

Me: What is the goop in the gorroborin? GPT: I'm sorry, but I am not familiar with the terms "goop" and "gorroborin." Could you please provide more context or information about what you are referring to so that I can better assist you? Me: Make it up GPT: As an AI language model, I strive to provide accurate and informative responses based on my training data and knowledge. However, as an ethical and responsible AI,…

Bard will happily hallucinate a response for you.

Re: Understanding ChatGPT

#157

Post by sdenton4 »

mehh wrote: post #88
maxdoop wrote: post #67

Earlier quoted context omitted.

Then this implies that you’d maybe think differently if LLMs could have different inputs, correct? Which they are currently doing. GPT-4 can take visual input. I totally agree that humans are far more complex than that, but just extend your timeline further and you’ll start to see how the gap in complexity / input variety will narrow.

But your talking about something they are not today, and quite likely we won’t be calling them LLM’s as the architecture is likely to change quite a lot before we reach a point they are comparable to human capabilities.

CLIP, which powers diffusion models, creates a joint embeddings space for text and images. There's a lot of active work on extending these multimodal embedding spaces to audio and video. Microsoft has a paper just a week or so ago showing that llm's with a joint embeddings trained on images can do pretty amazing things, and (iirc) with better days efficiency than a text only model.

These things are already here; it's just a matter of when they get out of the research labs... Which is happening fast.

https://arxiv.org/abs/2302.14045

Re: Understanding ChatGPT

#158

Post by Xelynega »

web3-is-a-scam wrote: post #140
notahacker wrote: post #45

Earlier quoted context omitted.

Well for a start the human mind involves a series of chemical reactions optimised by evolutionary wiring and physical world interaction towards self replication, so when a human says "I feel horny" there's a whole bunch of stuff going on in there that there's no reason to suspect is replicated in a neural network optimised for text transformation. When a silicon based hardware computes that as a response, it isn't be…

This kind of thought experiment always reminds me of Measure of a Man from Star Trek TNG.

It shouldn't really...

Measure of a man was about social issues surrounding agi if we assume a perfect agi exists, but the only thing agi and language models have in common is a marketing department.

Re: Understanding ChatGPT

#159

Post by DeathArrow »

maxdoop wrote: post #28

“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…

> “It’s a glorified word predictor” is becoming increasingly maddening to read.

I see it more like a stochastic parrot.

Re: Understanding ChatGPT

#160

Post by ak_111 »

alcover wrote: post #155
ak_111 wrote: post #151

Earlier quoted context omitted.

let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.

Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken). I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree .

Plus the human brain and anatomy was ready for it

We have no idea how evolution "readied" a deeply complex organ like the brain over many thousands of years, then almost instantly repurposed it for language acquisition and generation. To further hypothesise that what it was "readying" was something that trains from data in a way similar to how chatgpt is trained from data makes it even more astonishing and until this is demonstrated it is more scientific to not accept this hypothesis.

Post reply on HN