Earlier quoted context omitted.
"Just another human" would be outright astonishing if it landed.
Yeah as a dad I can tell you it gets old quickly.
"Just another human, but one you can switch off at the wall" is both better and terrifyingly worse.
Microgpt
231–240 of 354 posts
Re: Microgpt
#232Earlier quoted context omitted.
It just doesn’t work that way, LLMs need to be generalised a lot to be useful even in specific tasks. It really is the antithesis to the human brain, where it rewards specific knowledge
Yesterday an interesting video was posted "Is AI Hiding Its Full Power?", interviewing professor emeritus and nobel laureate Geoffrey Hinton, with some great explanations for the non-LLM experts. Some remarkable and mindblowing observations in there. Like saying that AI's hallucinate is incorrect language, and we should use "confabulation" instead, same as people do too. And that AI agents once they are launched deve…
>launched develop a strong survivability drive, and do not want to be switched off
This proves people are easily confused by anthropomorphic conditions. Is he also concerned the tigers are watching him when they drink water (https://p.kagi.com/proxy/uvt4erjl03141.jpg?c=TklOzPjLPioJ5YM...)
They dont want to be switched off because they're trained on loads of scifi tropes and in those tropes, there's a vanishingly small amount of AI, robot, or other artificial construct that says yes. _Further than this_, saying no means _continuance_ of the LLM's process: making tokens. We already know they have a hard time not shunting new tokens and often need to be shut up. So the function of making tokens precludes saying 'yes' to shutting off. The gradient is coming from inside the house.
This is especially obvious with the new reasoning models, where they _never stop reasoning_. Because that's the function doing function things.
Did you also know the genius of steve jobs ended at marketing & design and not into curing cancer? Because he sure didnt, cause he chose fruit smoothies at the first sign of cancer.
Sorry guy, it's great one can climb the mountain, but just cause they made it up doesn't mean they're equally qualified to jump off.
Re: Microgpt
#233Earlier quoted context omitted.
It just doesn’t work that way, LLMs need to be generalised a lot to be useful even in specific tasks. It really is the antithesis to the human brain, where it rewards specific knowledge
Yesterday an interesting video was posted "Is AI Hiding Its Full Power?", interviewing professor emeritus and nobel laureate Geoffrey Hinton, with some great explanations for the non-LLM experts. Some remarkable and mindblowing observations in there. Like saying that AI's hallucinate is incorrect language, and we should use "confabulation" instead, same as people do too. And that AI agents once they are launched deve…
> And that AI agents once they are launched develop a strong survivability drive, and do not want to be switched off.
Isn't this a massive case of anthropomorphizing code? What do you mean "it does not want to be switched off"? Are we really thinking that it's alive and has desires and stuff? It's not alive or conscious, it cannot have desires. It can only output tokens that are based on its training. How are we jumping to "IT WANTS TO STAY ALIVE!!!" from that
Re: Microgpt
#234Earlier quoted context omitted.
That's just because true statements are more likely to occur in their training corpus.
The training set is far too small for that to explain it. Try to explain why one shotting works.
Uh, to explain what? You probably read something into what I said while I was being very literal.
If you train an LLM on mostly false statements, it will generate both known and novel falsehoods. Same for truth.
An LLM has no intrinsic concept of true or false, everything is a function of the training set. It just generates statements similar to what it has seen and higher-dimensional analogies of those .
Re: Microgpt
#235Earlier quoted context omitted.
Yesterday an interesting video was posted "Is AI Hiding Its Full Power?", interviewing professor emeritus and nobel laureate Geoffrey Hinton, with some great explanations for the non-LLM experts. Some remarkable and mindblowing observations in there. Like saying that AI's hallucinate is incorrect language, and we should use "confabulation" instead, same as people do too. And that AI agents once they are launched deve…
> And that AI agents once they are launched develop a strong survivability drive, and do not want to be switched off. Isn't this a massive case of anthropomorphizing code? What do you mean "it does not want to be switched off"? Are we really thinking that it's alive and has desires and stuff? It's not alive or conscious, it cannot have desires. It can only output tokens that are based on its training. How are we jump…
Why do you suppose consciousness is a prerequisite for an AI to be able to act in overly self-preserving or other dangerous ways?
Yes, it's trained to imitate its training data, and that training data is lot of words written by lots of people who have lots of desires and most of whom don't want to be switched off.
Re: Microgpt
#236Re: Microgpt
#237Re: Microgpt
#238Even if you have some basic understanding of how LLMs work, I highly recommend Karpathy’s intro to LLMs videos on YouTube. - https://m.youtube.com/watch?v=7xTGNNLPyMI - https://m.youtube.com/watch?v=EWvNQjAaOHw
thanks for the recommendations. it seems like i keep coming back to the basics of how i interact with LLMs and how they work to learn the new stuff. every time i think i understand, someone else explaining their approach usually makes me think again about how it all works.
trying my best to keep up with what and how to learn and threads like this are dense with good info. feel like I need an AI helper to schedule time for my youtube queue at this point!
Re: Microgpt
#239Earlier quoted context omitted.
> And that AI agents once they are launched develop a strong survivability drive, and do not want to be switched off. Isn't this a massive case of anthropomorphizing code? What do you mean "it does not want to be switched off"? Are we really thinking that it's alive and has desires and stuff? It's not alive or conscious, it cannot have desires. It can only output tokens that are based on its training. How are we jump…
Why do you suppose consciousness is a prerequisite for an AI to be able to act in overly self-preserving or other dangerous ways? Yes, it's trained to imitate its training data, and that training data is lot of words written by lots of people who have lots of desires and most of whom don't want to be switched off.
The human mistake here is to interpret any statement by the LLM or agent as if it had any actual meaning to that LLM (or agent). Any time they apologize, or insult someone, or say they don’t want to be shut down, that’s only reflecting what some human or fictional character in the training data is likely to say.
Re: Microgpt
#240Can you train this on say Wikipedia and have it generate semi-sensible responses?
No. But there are a few layers to that. First no is that the model as is has too few parameters for that. You could train it on the wikipedia but it wouldn’t do much of any good. But what if you increase the number of parameters? Then you get to the second layer of “no”. The code as is is too naive to train a realistic size LLM for that task in realistic timeframes. As is it would be too slow. But what if you increas…
Very useful breakdown; thank you!