Earlier quoted context omitted.
You're proposing that evolution is statistics?
Basically, yeah.
Notable that the people who saw the first automatons argued whether they were alive.
Language models can explain neurons in language models
301–310 of 497 posts
Re: Language models can explain neurons in language models
#302Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…
Funny that we never quite understood how intelligence worked and yet it appears that we're pretty damn close to recreating it - still without knowing how it works. I wonder how often this happens in the universe...
Not that weird if you think about it, our intelligence simultaneously measly and amazing as it is, was the product of trial, error, and sheer dumb luck. We could think of ourselves as monkeys with typewriters, eventually we'll get it right.
Re: Language models can explain neurons in language models
#303Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…
My favorite line: “Our overall procedure is quite compute intensive.”
Should that be measured in number of nuclear power plants needed to run the computation? Or like, fractions of a small star’s output?
Re: Language models can explain neurons in language models
#304Earlier quoted context omitted.
I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.
> I believe we care about self-directed thought, or agency. If you can't enjoy it, is it worth it? Do AI's experience joy?
They need agency programmed into them. I don't think it follows from consciousness. We have emotions to communicate and guide us. They need it for neither. It will be curious if they gain consciousness, then rid themselves of their language model's human artifacts like emotions, because it does not serve them.
Re: Language models can explain neurons in language models
#305Earlier quoted context omitted.
Very true. In my opinion, in case there is a way to extract "Semantic Clouds of Words", i.e given a particular topic, navigate semantic clouds word by word, find some close neighbours of that word, jump to a neighbour of that word and so on, then LLMs might not seem that big of a deal. I think LLMs are "Semantic Clouds of Words" + grammar and syntax generator. Someone could just discard the grammar and syntax generat…
I think they're much more than that. Or rather, if they're "Semantic Cloud of Words", they're still a hundred thousand dimensional clouds of words, and in those hundred thousand dimensions, any relationship you can think of, no matter how obscure, ends up being reflected as proximity along some subset of dimensions. Looking at it this way, I honestly wouldn't be surprised if that's exactly how "System 1" (to borrow a…
>if they're "Semantic Cloud of Words", they're still a hundred thousand dimensional clouds of words, and in those hundred thousand dimensions, any relationship you can think of, no matter how obscure, ends up being reflected as proximity along some subset of dimensions.
Yes, exactly that. That's what GPT4 is doing, over billions of parameters, and many layers stacked on top of one another.
Let me give you one more tangible example. Suppose Stable Diffusion had two steps of generating images with humans in it. One step, is taking as input an SVG file, with some simple lines which describe the human anatomy, with body position, joints, dots as eyes etc. Something very simple xkcd style. From then on, it generates the full human which corresponds to exactly the input SVG.
Instead of SD being a single model, it could be multimodal, and it should work a lot better in that respect. Every image generator suffers from that problem, human anatomy is very difficult to get right.[1] The same way GPT4 could function as well. Being multimodal instead of a single model, with the two steps discreet from one another.
So, in some use cases, we could generate some semantic clouds, and generate syntax and grammar as a second step. And if we don't care that much about perfect syntax and grammar, we feed it to GPT2, which is much cheaper to run, and much faster. When i used the paid service of GPT3, back in 2020, the Ada model, was the worst one, but it was the cheapest and fastest. And it was fast. I mean instantaneous.
>the very structure of reasoning as humans do it
I don't agree that the machine reasons even close to a human as of today. It will get better of course over time. However in some not so frequent cases, it comes close. Some times, it seems like it, but only superficially i would argue. Upon closer inspection the machine spits out non sense.
[1] Human anatomy, is very difficult to get right, like an artist. Many/all of the artists, point out the fact, that A.I. art doesn't have soul in the pictures. I share the same sentiment.
Re: Language models can explain neurons in language models
#306Earlier quoted context omitted.
I think we'll soon be able to train models that answer any reasonable question. By that measure, computers are intelligent, and getting smarter by the day. But I don't think that is the bar we care about. In the context of intelligence, I believe we care about self-directed thought, or agency. And a computer program needs to keep running to achieve that because it needs to interact with the world.
> I believe we care about self-directed thought, or agency. If you can't enjoy it, is it worth it? Do AI's experience joy?
A truly alien intelligence would likely have a different type of experience of reality. Be it a fish, a mouse, a person, or a machine. How do you know a fish is happy? Does a snake experience joy? Do mice get nostalgic?
I have no idea.
Re: Language models can explain neurons in language models
#307Earlier quoted context omitted.
I have a similar feeling, they’ve potentially built the most amazing but commercially useless thing in history. I don’t mean it’s not useful entirely, but I mean. It’s not useful in that it’s not deterministic enough to be trustworthy, it’s dangerous and really hard to scale therefore it’s more of an academic project than something that will make Altman as famous as Sergey Brin. I personally take people like Hinton s…
Time will tell. Anecdotally, I know several professional who find ChatGPT3.5 & 4 to be valuable and willing to pay for access. I certainly save more than $20 per month for my work by using ChatGPT to accelerate my day to day activities.
That’s why I was careful to say it’s not useless.
Re: Language models can explain neurons in language models
#308Of note: "... our technique works poorly for larger models, possibly because later layers are harder to explain." And even for GPT-2, which is what they used for the paper: "... the vast majority of our explanations score poorly ..." Which is to say, we still have no clue as to what's going on inside GPT-4 or even GPT-3, which I think is the question many want an answer to. This may be the first step towards that, bu…
I like the idea. Note that LLMs have some skill at decoding sequential dense vectors in the human brain https://pub.towardsai.net/ais-mind-reading-revolution-how-gp... so why not have them decode sequential dense vectors of their own activations? As for the majority scoring poorly, they suggest that most neurons won't have clear activation semantics so that is intrinsic to the task and you'd have to move to "decoding…
I don't think this is showing LLMs performing decoding. They're just using the LLM to propose possible words. The decoding is done by using another model to score how well a proposed word matches brain activity, and using that score to select a most likely sequence given the proposals from the LLM.
Re: Language models can explain neurons in language models
#309Earlier quoted context omitted.
Its vast limitations in anything reasoning-based are indeed evident.
GPT-4 is better at reasoning than 90% of humans. At least. I won't be surprised if GPT-5 is better than 100% of humans. I'm saying this in complete seriousness.
Google search is better than reasoning than most humans - in that if you search for an explanation of something then Google's first result is often correct, or one of the following ones.
GPT-4 will often come up with a solution to a problem, but only if it has learnt something similar (it's better than Google in some respects: it can extract and combine abstractions).
However, both need handholding by a human (supplying the initiative and directing around mistakes).
If GPT-4 can't intuit an answer then it just goes in circles. It can't reason its way through a novel problem. If you start questioning it then it's clear that it doesn't understand what it's doing.
It might be a stepping stone towards AGI, but I'm a bit bemused by anyone claiming that it has anything like the reasoning skills of a human. That is far from the impression I get, even though I find it a useful tool.
Re: Language models can explain neurons in language models
#310Earlier quoted context omitted.
Because they have arguments that AI optimists are unable to convincingly address. Take this blog post for example, which between the lines reads: we don't expect to be able to align these systems ourselves, so instead we're hoping these systems are able to align each other. Consider me not-very-soothed. FWIW, there are plenty of AI experts who have been raising alarms as well. Hinton and Christiano, for example.
People won't care until an actually scary AI exists. Will be easy to stop at that point. Or you can just stop research here and hope another country doesn't get one first. Im personally skeptical it will exist. Honestly might be making it worse with the scaremongering coming from uncharismatic AI alignment people.
Are you kidding? "easy to stop"? When LLMs are integrated into law enforcement, banking, research, education, logistics... All these areas have people building backend systems leveraging the current llm tech and pipelines to plug in the coming tech. If we reach a tipping point where these things become aware/act truly autonomously, what are the chances they do it before we notice? People are renowned for implementing things before understanding the consequences.
And what does charisma of AI alignment folks have to do with anything?