The Yudkowskian 'Bayesians' are a scifi cult. Intelligence as it exists in nature is neither arteficial nor general nor 'super.' All human technology is natural. It certainly is not supernatural.
The sigmoids won't save you
281–290 of 297 posts
Re: The sigmoids won't save you
#282Earlier quoted context omitted.
> basically how well a universal function approximator can fit to a function we define That's what you've got wrong. We don't define functions that an LLM approximates. Autoregressive pretraining approximates an unknown function that produces text (that is what the brain does). RL doesn't approximate functions, it optimizes objective by finding an unknown function that performs better.
What an interesting perspective! There is just a small problem. How does the human brain work in the phase between being born and learning written language? Seems like a significant bootstrapping problem for your theory.
LLMs capture a snapshot of the "collective brain" functionality as represented by literate people. I don't speculate how our brains get to this point.
Re: The sigmoids won't save you
#283Earlier quoted context omitted.
This is all speculative. We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation", especially once you add feedbacks loops.
We understand it enough to see the obvious massive deficiencies in LLMs. They can predict likely sentences but not evaluate truth or logic. They can fairly reliably record facts about the world but not construct internal models of the world.
> They can predict likely sentences but not evaluate truth or logic.
They do probabilistically. So do humans as a matter of fact. The best of us are better at it than LLMs, but that's not persuasive evidence of anything meaningful really.
> They can fairly reliably record facts about the world but not construct internal models of the world.
You don't know that, unless your presuppose a very specific definition of world model that necessarily precludes emergent ones.
Re: The sigmoids won't save you
#284Earlier quoted context omitted.
I think this is a bit too pessimistic. Progress in algorithms has matched or exceeded progress in hardware, so the same number of FLOPS spent training GPT-3 years ago would produce a much better model today. Ditto for energy use, and hardware is more efficient at delivering FLOPS. > the widening wealth gaps creating a new underclass not seen since before WW2 I go back and forth on this. I think the reality is that "u…
> AI and automation makes things so cheap that today's underclass lives better than kings ever did I suggest you go share this opinion with the people living on the street because they can't afford housing.
That's a tiny minority. The inconvenient truth is that the vast majority of those living on the street are mentally ill or drug addicts.
Re: The sigmoids won't save you
#285Earlier quoted context omitted.
AI images are all unique and new.
Not nearly as new as Shakespeare was when he single handedly reinvented English vocabulary, for example.
He accidentally formalized spelling. You can't invent vocabulary when you're writing plays for a mass audience and want the audience to actually understand what is being said. That makes no sense.
Re: The sigmoids won't save you
#286There are practitioners and pontificaters, and never the twain shall meet.
Re: The sigmoids won't save you
#287Earlier quoted context omitted.
Look at the tasks in the benchmark (see §2 https://arxiv.org/html/2503.14499v3 )
Yeah, what about them? As far as I read it the tasks are fixed. The AI companies should know the tasks by now, and have overfitted their models on the tests by now, in the same way I'm implying I overfitted my model to reproduce Harry Potter.
You can choose to believe that.
Re: The sigmoids won't save you
#288Earlier quoted context omitted.
LLMs also don't run on tokens internally, they're just the inputs and outputs. The reasoning models do operate (partially) in the token space, but then so do I.
LLM's generate their output words sequentially based on probability (from learned stats). Human's don't operate the same way, the thought happens and then the words are generated to reasonably describe that thought.
> the thought happens and then the words are generated to reasonably describe that thought.
Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts.
Claims like "humans don't operate the same way" has no basis. Not only do we literally not know how humans operate mechanistically, and so we literally don't know the logical structure of human thought, but any system that is Turing complete is so easy to create that many wildly different mechanistic systems are fundamentally equivalent/interconvertible.
Re: The sigmoids won't save you
#289Earlier quoted context omitted.
LLM's generate their output words sequentially based on probability (from learned stats). Human's don't operate the same way, the thought happens and then the words are generated to reasonably describe that thought.
> the thought happens and then the words are generated to reasonably describe that thought. Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts. Claims like "humans don't operate the same way" has no basis. Not only do we li…
> Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts.
Yes, possible, that's why I asked you above if that's what you meant by "token". Someone else responded and I didn't notice it wasn't you.
> Claims like "humans don't operate the same way" has no basis. Not only do we literally not know how humans operate mechanistically, and so we literally don't know the logical structure of human thought, but any system that is Turing complete is so easy to create that many wildly different mechanistic systems are fundamentally equivalent/interconvertible.
I think this position is too extreme, we do have some information.
We know how LLM's work when generating a sequence of words and I know that my brain does not work the same way for word generation because I am fully aware of the complete thought in advance of any words getting generated by me externally or internally.
I know prior to generating words that my thought is X and the words I'm about to produce need to express that thought.
But with LLM's we know that the essence of what they produce is not known in advance, that it must complete the word generation process to fully realize the end result and that multiple different end results are possible.
Re: The sigmoids won't save you
#290Earlier quoted context omitted.
LLMs also don't run on tokens internally, they're just the inputs and outputs. The reasoning models do operate (partially) in the token space, but then so do I.
LLM's generate their output words sequentially based on probability (from learned stats). Human's don't operate the same way, the thought happens and then the words are generated to reasonably describe that thought.
What I'm saying is that this is incorrect. An "idea" exists within a model before it generates tokens. This property does not distinguish humans from LLMs.
Additionally "from learned stats" doesn't disambiguate between a wider variety of things. I'm not aware of any other way to acquire knowledge from measurements. I'd bet that humans do this differently, based on the fact the humans can get further with less training data and that they learn actively during operation, but not so differently that 'learning stats' would be an inaccurate description.