Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

201–210 of 518 posts

Re: What is ChatGPT doing and why does it work?

#201

Post by pedrosorio »

moomoo11 wrote: post #197
gtoubassi wrote: post #159

Earlier quoted context omitted.

I agree this is an incredibly interesting paper. I am not a practitioner but I interpreted the gradient article differently. They didn’t directly find 64 nodes (activations) that represented the board state as I think you imply. They trained “64 independent two-layer MLP classifiers to classify each of the 64 tiles”. I interpret this to mean all activations are fed into a 2 layer MLP with the goal of predicting a sin…

What is MLP?

https://en.m.wikipedia.org/wiki/Multilayer_perceptron

A “Classic” neural network, where every node from layer i is connected to every node on layer i+1

Re: What is ChatGPT doing and why does it work?

#202

Post by tkgally »

britzkopf wrote: post #132
codeulike wrote: post #32

Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…

The thing I'm sort of confused about, but maybe someone can explain why I shouldn't be, is, why does there seem to be no implication for language translation? Or is there but coverage is overwhelmed by the fascination with chatGPT? In short, is machine language translation now a fully solved problem? A couple years ago when I tested Google translate in a non-esoteric conversation with my Russian speaking girlfriend a…

I worked as a translator for many years and have been following developments in machine translation closely. In my opinion, ChatGPT does represent a significant advance for machine translation. If you have the time to watch it, I made a video about the topic last week:

https://youtu.be/najKN2bXqCo

Re: What is ChatGPT doing and why does it work?

#204

Post by abudabi123 »

yen223 wrote: post #27
krackers wrote: post #11

Earlier quoted context omitted.

If this idea space is linear, it gives a whole new meaning to an idea being orthogonal to another.

I've always thought "orthogonal" as in ideas came from "orthogonal" as in vectors - meaning to be independent of each other.

To sort the mess in psychology a statistical technique separates the variables as dependent and independent.

Re: What is ChatGPT doing and why does it work?

#205

Post by WillDaSilva »

netsharc wrote: post #143
kilgnad wrote: post #129

Earlier quoted context omitted.

Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…

> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyri…

> [...] it's still a Chinese Room.

So are you, and so am I.

That's not to say that ChatGPT is sentient or has a significant amount of personhood, but we shouldn't wholly dismiss its significance in this regard, particularly not using that faulty argument.

Re: What is ChatGPT doing and why does it work?

#206

Post by bottlepalm »

janalsncm wrote: post #153
bottlepalm wrote: post #96

The easiest way for ChatGPT to generate good output is to plain understand it. Given the vast amount of input data fed into it, it has no choice, but to start reducing the input into fundamental rules which is basically what understanding is. Understanding is a form of compression. More efficient for a neural network to understand a concept than memorize permutations. Same with statistics and markov chains, people fo…

Understanding is a form of compression, but not all compression is understanding, at least in the way I would describe it. Given an arbitrary function, you could simply compress some number of inputs and outputs, giving a lossy mapping between the two. Maybe you would interpolate between the two closest points in the range for unknown values. Or, you could store the function itself. To see the difference, compare for…

I agree understanding would be a subset of compression, reducing massive amounts of data into ‘meaning’ that can be applied effectively to new arbitrary sets of data.

Re: What is ChatGPT doing and why does it work?

#207

Post by yCombLinks »

jeremyjh wrote: post #194
cm2012 wrote: post #188

Earlier quoted context omitted.

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it a…

Ahh, so being as intelligent as an average intelligence is no longer sufficient to declare it intelligent. Now it must surpass all of our achievements. 99.9% of people will never "create some substantiative achievement".

Re: What is ChatGPT doing and why does it work?

#209

Post by hanniabu »

jeremyjh wrote: post #194
cm2012 wrote: post #188

Earlier quoted context omitted.

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it a…

Considering all the info comes from human generated content, I think a better term would be Collective Intelligence rather than artificial intelligence

Re: What is ChatGPT doing and why does it work?

#210

Post by wankle »

thfuran wrote: post #78
calf wrote: post #51

I think what is interesting is even in the 1st sentence of Wolfram's essay, he acknowledges what ChatGPT is doing could merely be superficial. So we can't even tell if ChatGPT actually learned "language" in a superficial versus nontrivial/substantial sense. Lots of people say, I asked ChatGPT to write me a poem/essay, and it did! But was it really a poem/essay, or did it just look like one and on closer examination i…

>But was it really a poem/essay, or did it just look like one and on closer examination it is more like a fake out of a poem/essay? What does that even mean?

Perhaps they were referring to the nature of machine learning to be akin to the Ronco slicer/dicer from the 1970's but with data.
Post reply on HN