Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

191–200 of 518 posts

Re: What is ChatGPT doing and why does it work?

#191
post #126

Earlier quoted context omitted.

Could you elaborate please? For those of us not familiar with Wittgenstein's work, could you link to sources depicting before and after his views changed, preferably with summaries.

Summary of before: https://en.wikipedia.org/wiki/Tractatus_Logico-Philosophicus Summary of after: https://en.wikipedia.org/wiki/Philosophical_Investigations

thank you

Re: What is ChatGPT doing and why does it work?

#192

Earlier quoted context omitted.

Take a screen shot. On iPad/iPhone you can save the screenshot of a webpage as a full PDF of the page. Make sure you are in Safari.

Awesome tip. Thank you.

You can also print the page to a PDF. In the share menu, pick Print. In the print options dialog do the pinch to zoom motion on one of the thumbnails and this will give you a big preview that you can save as PDF (again through the share menu).

Re: What is ChatGPT doing and why does it work?

#193

I spent about an hour reading this. I know most people probably won't be able to do the same because of time constraints in their day, but seriously this is the most awesome post about large language models I've read. Take a second to think about how much time it must have taken to write this, to prepare all of these examples. This post gave me another way to look at something I already partially understood pieces of…

I was estimating more than 1 hour to read it properly. Since you did it in only 1 hour that’s encouraging me to try tomorrow or another day.

I can’t comprehend how ChatGPT can keep such incredible internal state and I’m hoping Wolfram might have some ideas in this post. For instance try the prompt that turns it into a linux terminal and it can closely “hallucinate” approximately the correct changes in state to the host after the commands but long long after the command occurred in the dialogue. To me this shows it extends beyond human language.

Re: What is ChatGPT doing and why does it work?

#194
post #188

Earlier quoted context omitted.

But if you ever a have a conversation with it you know it isn't a brain. I'm not talking about detection here; its whole point is to generate credible text so it is going to evade detection well. But can't you just tell from talking to it that there is nothing there?

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it actually creates some substantiative achievement, such as designing a workable cold fusion setup I'm not going to recognize it as a super-intelligence.

Re: What is ChatGPT doing and why does it work?

#195
post #130

Earlier quoted context omitted.

Define understand, and does an analog to Godel's incompleteness apply.

> does an analog to Godel's incompleteness apply not GP but this seems like quite an attractive idea that many people have reached: a brain of a given "complexity" cannot comprehend the activity of another brain of equal or higher complexity. I'm positive I'm cribbing this from scifi somewhere, maybe Clarke Or Asimov, but, it's the same idea as the Chomsky hierarchy, and the Godel theorems seem like a generalization…

Nice! I like the way you put this.

Re: What is ChatGPT doing and why does it work?

#196
post #185

Earlier quoted context omitted.

> To me, it sounds like you're the fearful one. What do you think that fear may be?

IDK you tell me? All I can say is your OP, said "I think part of it is a subconscious fear ... I understand what I'm saying is dramatic". Why do you think it is a fear (you explained your thoughts so no need to re-explain), and why do you think what you say is dramatic? It appears to me you are projecting your thoughts and fears. I do though, find your last post dramatic, as you have capital words "ARE MORE" and "OBV…

You asking me or him? I can't speak for the person you responded to but my own answer is here: https://news.ycombinator.com/item?id=34800063

To be honest I'm sort of in denial too. My actions contradict my beliefs. I'm not searching for occupations and paths that are separate from AI, I'm still programming as if I could do this forever.

Also I capitalize words for emphasis. It doesn't represent emotion. Though I do have emotions and I do have bias, but not on the topics I am describing here.

Re: What is ChatGPT doing and why does it work?

#197

Earlier quoted context omitted.

That Kenneth Li Othello paper is great. The accompanying blog post https://thegradient.pub/othello/ was discussed on HN here https://news.ycombinator.com/item?id=34474043 A lot of people didn't seem to get it when it was discussed on HN. A GPT had _only_ ever seen Othello transripts like: "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. It learned Othe…

I agree this is an incredibly interesting paper. I am not a practitioner but I interpreted the gradient article differently. They didn’t directly find 64 nodes (activations) that represented the board state as I think you imply. They trained “64 independent two-layer MLP classifiers to classify each of the 64 tiles”. I interpret this to mean all activations are fed into a 2 layer MLP with the goal of predicting a sin…

What is MLP?

Re: What is ChatGPT doing and why does it work?

#198
post #185

Earlier quoted context omitted.

> To me, it sounds like you're the fearful one. What do you think that fear may be?

IDK you tell me? All I can say is your OP, said "I think part of it is a subconscious fear ... I understand what I'm saying is dramatic". Why do you think it is a fear (you explained your thoughts so no need to re-explain), and why do you think what you say is dramatic? It appears to me you are projecting your thoughts and fears. I do though, find your last post dramatic, as you have capital words "ARE MORE" and "OBV…

Im not the OP, i just asked the question. And imagined the answer would involve something about existential threat to human intelligence or something like that.. I simply asked out of curiosity

Re: What is ChatGPT doing and why does it work?

#199

Earlier quoted context omitted.

I agree this is an incredibly interesting paper. I am not a practitioner but I interpreted the gradient article differently. They didn’t directly find 64 nodes (activations) that represented the board state as I think you imply. They trained “64 independent two-layer MLP classifiers to classify each of the 64 tiles”. I interpret this to mean all activations are fed into a 2 layer MLP with the goal of predicting a sin…

What is MLP?

Multilayer Perceptron

Re: What is ChatGPT doing and why does it work?

#200

Earlier quoted context omitted.

I agree this is an incredibly interesting paper. I am not a practitioner but I interpreted the gradient article differently. They didn’t directly find 64 nodes (activations) that represented the board state as I think you imply. They trained “64 independent two-layer MLP classifiers to classify each of the 64 tiles”. I interpret this to mean all activations are fed into a 2 layer MLP with the goal of predicting a sin…

What is MLP?

Multi-layer perception, synonym of neural network but perhaps with the additional implication that it is fully connected.
Post reply on HN