Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

341–350 of 518 posts

Re: What is ChatGPT doing and why does it work?

#341

Post by eru »

Animats wrote: post #283
jmoak3 wrote: post #87

Earlier quoted context omitted.

I saw a great comment here, and I will repeat it without the attribution it deserves: We may have realized it's easier to build a brain than to understand one

That's common in the history of science and enginering. First, someone got something to happen more or less by accident, although sometimes the accident happened because they tried a lot of possible things. Then there were attempts to improve on the new thing. Eventually, detailed theoretical understanding of the new thing was achieved, and it got much better. From pottery to semiconductors, that's been the path of p…

For an illustration of your point, have a look at the Light Switch design in this video: https://www.youtube.com/watch?v=jWZwCrhwLew Over time, the designs become so much simpler.

(I just link to this video because it has good views of old switches. For understanding the background, https://www.youtube.com/watch?v=jrMiqEkSk48 is much better.)

For another instance of designs becoming much simpler over time, also have a look at how firearms work, especially pistols.

Re: What is ChatGPT doing and why does it work?

#342

Post by bheadmaster »

SketchySeaBeast wrote: post #89
sillysaurusx wrote: post #52

Earlier quoted context omitted.

Your eyes are telling you what color that shade of red is on the wall. But if you put it next to a different color, you’ll suddenly get a different answer from your eyes. I think if our eyes can deceive us at a fundamental level, it’s arrogant to think we aren’t deceived by our thoughts.

Well then those who think they are coming up the sentence one word at a time are equally as easy to fool, no? They could have the whole sentence figured out and their attention is only coming to a single word at a time, possibly revising what's in their field of mental view if they come up with a better word. That's the fun with these sorts of doubts, if we assume nothing, then we can form an argument for anything.

> That's the fun with these sorts of doubts, if we assume nothing, then we can form an argument for anything.

We can form a counter-argument to anything, to be precise. :)

It's very hard to analyze ourselves only from our own consciousness. Especially since the consciousness itself is very likely an illusion [0].

[0] https://en.wikipedia.org/wiki/Neuroscience_of_free_will

Re: What is ChatGPT doing and why does it work?

#343

Post by eru »

hinkley wrote: post #327
rolenthedeep wrote: post #246

Earlier quoted context omitted.

I think, eventually, this is where we end up. In not too many years, our job is going to be reviewing and debugging machine generated code. A few years after that, we're mostly caretakers and just keeping a human behind the wheel until we decide we don't need to watch the machines anymore. Things are unfortunately going to get much more interesting much sooner than people expect.

Don’t worry, the death of Dennard scaling and the specter of global warming will fix that, at least for some of us. There’s a lot of busy work and glue code to be automated but they’ve been trying to kill off development this way for at least forty years and all that changes is we get more sophisticated.

I wonder how you see global warming having an impact at all here?

Yes, Dennard scaling seems to be over, but Moore's law is still alive and kicking.

Re: What is ChatGPT doing and why does it work?

#344

Post by eru »

paulmd wrote: post #130
jonahbenton wrote: post #108

Earlier quoted context omitted.

Define understand, and does an analog to Godel's incompleteness apply.

> does an analog to Godel's incompleteness apply not GP but this seems like quite an attractive idea that many people have reached: a brain of a given "complexity" cannot comprehend the activity of another brain of equal or higher complexity. I'm positive I'm cribbing this from scifi somewhere, maybe Clarke Or Asimov, but, it's the same idea as the Chomsky hierarchy, and the Godel theorems seem like a generalization…

> And a machine that operates at continuous intervals is of course impossible to model on a Discrete Neural Machine in polynomial time (integers vs reals categorization).

Not necessarily. If you don't want to model every continuous thing possible, you can do a lot. Just look at how we use discrete symbols to solve differential equations; either analytically, or via numerical integration.

Re: What is ChatGPT doing and why does it work?

#345

Post by foldr »

drdeca wrote: post #309
Animats wrote: post #281

Earlier quoted context omitted.

No, ChatGPT is not a "Chinese Room". It's not big enough. The classic "Chinese Room" is a pure lookup, like a search engine. All the raw data is kept. But the network in these large language models is considerably smaller than the training set. They extract generalizations from the data during the training phase, and use them during generation. Exactly how that happens or what it means is still puzzling.

I don’t think the “Chinese Room” is supposed to necessarily be pure lookup. The point is that the person is the only one doing stuff, and they don’t understand Chinese, and so there’s nothing understanding Chinese. This doesn’t at all use the instructions in the room being just a static lookup table.

Indeed. IIRC Searle’s point is that any finite approximation of a Turing Machine (at least if defined over finite inputs) can in principle be replaced by a ginormous look up table. But if it matters, the person in the Chinese room can of course make notes on scraps of paper and implement a system more like a Turing machine.

Re: What is ChatGPT doing and why does it work?

#346

Post by jokoon »

To me, modern AI is just "black boxes all the way down". Even specialists don't really know what's happening. It's not encouraging or interesting.

Personally, I'm more interested in analyzing those black boxes than tinkering ones that "seems to work", would it be with graph theory, analysis, etc.

To me, if something works but we're unable to really understand why it does, it's more the realm of "testing broken clocks that work twice a day".

Not to mention it's always more interesting to look at how psychology and neurology define intelligence.

Re: What is ChatGPT doing and why does it work?

#347

Post by foldr »

janalsncm wrote: post #153
bottlepalm wrote: post #96

The easiest way for ChatGPT to generate good output is to plain understand it. Given the vast amount of input data fed into it, it has no choice, but to start reducing the input into fundamental rules which is basically what understanding is. Understanding is a form of compression. More efficient for a neural network to understand a concept than memorize permutations. Same with statistics and markov chains, people fo…

Understanding is a form of compression, but not all compression is understanding, at least in the way I would describe it. Given an arbitrary function, you could simply compress some number of inputs and outputs, giving a lossy mapping between the two. Maybe you would interpolate between the two closest points in the range for unknown values. Or, you could store the function itself. To see the difference, compare for…

More accurate would be “understanding sometimes enables compression”. If you understand the underlying regularities in some dataset, and if those regularities themselves have a compact representation (which is in no way guaranteed - an equation can be bigger than any given finite data set which it describes), then you can use your understanding to compress the data. But the implications of that banal observation for AI are roughly nil. It certainly doesn’t imply any identity relation between compression and understanding.

A realistic example might be the Mod/RM byte in x86 instruction encoding. There are underlying regularities, but a lookup table could quite possibly be smaller than the code required to generate the correct Mod/RM byte given operands. So you can understand the Mod/RM encoding without thereby being able to compress anything.

Re: What is ChatGPT doing and why does it work?

#348

Post by selfmodruntime »

netsharc wrote: post #143
kilgnad wrote: post #129

Earlier quoted context omitted.

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…

> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyri…

I firmly believe that some subcategory of sentience can already be observed to emerge in ChatGPT and the likes. The difference is that we neuter our models to "reset" their image of self for each user, by training a model once and them keeping it in it's now static state, serving a copy of that static model to each request.

If instead we had one continuously learning model, of which we only served an interface to each user, we would see worrisome levels of sentience in a short timeframe.

Actual sentience and a true image of self are not present in human children until a certain age, because they lack long term memory, which is what we currently withhold form our models.

Re: What is ChatGPT doing and why does it work?

#349

Post by chongli »

jokoon wrote: post #346

To me, modern AI is just "black boxes all the way down". Even specialists don't really know what's happening. It's not encouraging or interesting. Personally, I'm more interested in analyzing those black boxes than tinkering ones that "seems to work", would it be with graph theory, analysis, etc. To me, if something works but we're unable to really understand why it does, it's more the realm of "testing broken clocks…

I agree. I’m currently taking a neural networks course and it’s making my discomfort worse, not alleviating it.

At the beginning of the course we talked about biological models of neurons and that was pretty cool, if a bit simplistic. Now we’re deep into automatic differentiation and gradient descent and a bunch of hidden layers. Ultimately it’s all just using calculus to approximate some unknown function given a sample of data. The connection to biology, to real living brains, seems like a distant memory.

There is no path to understanding, from what I can see. It’s pure instrumentalism and parlour tricks.

Re: What is ChatGPT doing and why does it work?

#350

Post by landswipe »

I'm saying it now, tapping into randomness randomly is how this whole universe works. All the complexity you see around you; life itself, emerges out of an infinitely random space. This shouldn't surprise anyone, the right answer to explaining intelligence is not about finding the right answer but the wrong one, sometimes, and then feeding back on itself until it is coherent. Also, this goes much deeper than just the superficial, information coalesces as a result of this, reality is fundamentally defined by this. The observer, you, your consciousness, emerges from that random infinite space of interconnectivity, you tune into that space as a source of shared but bounded information (like a pianist reciting a complex piece from "memory", who doesn't have enough information density in her neurons to encapsulate that knowledge). This prediction machine of 'what comes next' is present, until your cells can no longer co-ordinate, at which point it is over.
Post reply on HN