Earlier quoted context omitted.
Imagine thinking someone donating money is evidence of something bad
If the cause they donate to is bad, it is
What is bad about "effective charities"? It's probably mostly for malaria nets!
AI 2040: Plan A
531–540 of 567 posts
Re: AI 2040: Plan A
#532Earlier quoted context omitted.
Yeah I think its possible that for many folks its the first time they're coming up on these concepts, and it troubles them in the same way that the concept of death troubles them (and me, to be clear!). For me its as simple as watching how people talk, and seeing how in every single case whatever the next thing is, if you believe It, there is only ever justification of doubling down, doing more, going deeper, reducin…
> whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. As usual, AI skeptics are moving goal posts. Modern LLMs are on a completely different level in terms of how GENERAL they are vs anything pre-LLM. You can give it a completely novel puzzle and it will solve it. 5+ years ago you had to tr…
[deleted]
Re: AI 2040: Plan A
#533Earlier quoted context omitted.
Yeah I think its possible that for many folks its the first time they're coming up on these concepts, and it troubles them in the same way that the concept of death troubles them (and me, to be clear!). For me its as simple as watching how people talk, and seeing how in every single case whatever the next thing is, if you believe It, there is only ever justification of doubling down, doing more, going deeper, reducin…
Did you actually read the text? OPs are calling that Plan D. They're proposing an alternative, which is a global brake on frontier AI research to keep the basilisk in its jar until we work out what we're dealing with and how to handle it.
No, they're proposing a spying panopticon and state control of global resource distribution - specifically general purpose compute - including seizing and destroying GPUs. They're proposing a totalitarian global dictatorship controlling computing hardware and software.
Lest you think I'm being hyperbolic: https://ai-2040.com/supplements/covert-ai-projects
This is arsonists selling fire insurance.
Re: AI 2040: Plan A
#534Earlier quoted context omitted.
The AI 2040 writing and the AI 2027 before it both sprang from within the rationalist communities. Scott Alexander (of the popular rationalist Slate Star Codex / Astral Codex Ten blog) played a role in both articles, though he says he chose to leave his name off of this one. AGI and AI doom scenarios have been evergreen topics in rationalism for years. Long before we had enough compute power to do anything at all use…
First I was intrigues by the rationalist movement, but I was deeply disappointed. To put it bluntly, seems to me like geeks LARPing as philosophers, and not with rational argument as they promise, but rationalizing their sci fi fanfictions.
Yes, calling themselves "rationalists" seems to be the biggest tell. That they wish they were that. Yet their behavior reflects anything but.
Re: AI 2040: Plan A
#535Earlier quoted context omitted.
It's a form of denial. We're getting another "de-thronement of man" on the order of Copernicus and Darwin. Some get excited, others turn away in horror. Negation is the outward expression of the desire to keep human intelligence wrapped in its mystical veil. One popular idea is that these systems will asymptotically approximate human intelligence because they're trained on mostly human-written texts. Not only is that…
> with previous RL-systems Most human systems are much, much, much more complicated than most closed world games (which is where RL approaches have seen massive success, mostly through self-play). Like LLMs are great, but I honestly can't see us getting actual general intelligence out of them.
They already possess general intelligence by many metrics. Sure they miss a few, but that's nitpicky and goalpost shifty - lots of humans make errors of all sorts as well, or incur brain damage limiting them in one or a few areas of intelligence - we don't then say they are not general intelligence anymore.
I think maybe you mean superintelligence, which is a more fair critique.
Re: AI 2040: Plan A
#536Earlier quoted context omitted.
What's strange to me about these comments is they're timeless. They could have been written in 2026 or 2016 or 1966. Like, afaict, for many on HN going from ELIZA->Fable 5 just didn't cause any update to priors regarding this whole philosophical question. The argument against has remained unchanged. I don't see any point in arguing about it, I just find it very strange.
Fable 5 doesn't represent anything new, other than scale and some refinement techniques, over the original LLMs. In the chase for AGI specifically, LLMs are a dead end, just like all the other AI technologies that died in the AI winter. What priors should be updated?
Human brain looks a lot like scale and minor refinement above dog brain. Or mouse brain.
Re: AI 2040: Plan A
#537Earlier quoted context omitted.
No one imagined LLMs in their current format, it was simply a result of discovering that scaling compute and tokens produced better and better results with the Transformer architecture. The inventors of the Transformer architecture were working on better translation, and probably did not imagine that their architecture would lead to modern LLMs. Imagining something in advance is not necessary at all for scientific ad…
> Imagining something in advance is not necessary at all for scientific advancement. This is particularily true in AI, and no one expects to imagine what superintelligence is until after it is created. Then why does anyone expect to create it? I'll take a stab at an answer: they think an LLM is some kind of "incremental improvement" and therefore a step along the inevitable path to discovering AI. But that seems delu…
I guess you don't really know how an LLM works then..?
Re: AI 2040: Plan A
#538Earlier quoted context omitted.
I used the example of 1G constant acceleration space flight in another thread which got downvoted to oblivion, but I think it's a good one. That's a technology we know how to build. We just need superconducting electronics and miniaturized fusion reactors, or a ship which is built like Project Orion to use nuclear bombs for propulsion. Now write down a blueprint for superintelligence. So I've given you two impossible…
In this very thread I am being told that Fable is nothing but a bit of scale and refinement on well-known neural network techniques. And next I am told that we can't even imagine how to build superintelligence. Which is it folks?
Indeed, the real fun signal here is the conflict between how intelligent humans think they are and the different story told by much of human behavior, including within this very thread!
When an LLM makes a cognitive mistake, humans jeer "dumb stochastic parrot!" - when a human does the same, many are blind to it. Like a weird auto-Gell-Mann Amnesia - relative to reflecting on our own perceived cognitive "strengths" versus others weaknesses.
Re: AI 2040: Plan A
#539Earlier quoted context omitted.
> I think you've already conceded that LLMs demonstrate emergent behaviour No. Please don't put words in my mouth. What I said is that an LLM compresses a bunch of information into a semantic embedding space and then does sort of a stochastic search in that embedding space. Any similarity to "intelligence" is accidental. You may look at the results of that process and "see" thinking or reasoning or something, but it…
> but it ain't there On what grounds? I don't think you've provided any evidence other than LLMs can't "adapt" or "learn" to show that LLMs do not show intelligence in any way. I think it's clear that there must be some emergent form of intelligence over words from just the agentic coding ability alone. I am not claiming that LLMs are intelligent, only that they display aspects of what we understand as intelligence.…
@sudb, I salute your effort. It is unfortunate how little this person took from your attempts at a genuine exchange.
Without moving goal posts: @27183: the word intelligence is defined as "noun. The ability to acquire, understand, and use knowledge."
LLMs can absolutely do this via in-context learning. They can even persist this as mementos on disk and re-attend to it later if its been purged from their context window.
You might try to make some comment like "they're not learning in the way humans do, they're not changing their weights, and they predict token by token!" Who cares? That's semantics - LLM attention is able to accumulate contributions of tokens and weights mixed together head by head and layer by layer with non-linear activations which functionally quack like intelligence.
They may be far different from us, but they're functionally intelligent (at least in several subsets of intelligence. And frankly, in some cases, they're more functionally capable than many humans).
Birds, planes, helicopters, rockets and gliders don't achieve flight in the same way, or have remotely the same capabilities in the air - but all are nonetheless forms of flight.
Re: AI 2040: Plan A
#540[flagged]