Never thought China would be the leader in open source AI. OpenAI and Anthropic are making fools out of themselves. The reason they want this is likely because they don't own the compute and their models get distilled shortly after releasing them
We must pace the frontier
481–490 of 935 posts
Re: We must pace the frontier
#482I also like the idea of pacing the frontier and slowing down development so we all have a chance to stop and catch our breath. However, I don't think regulating AI is the way to do it. I would tackle the problem by constraining the resources used to run AI models; the "simplest" way to do that is increasing the costs of running a data center. The two easiest places to raise data center costs are electricity and water…
Yes, water not used for human consumption should be more expensive (we should stop subsidies for that). Of course it won’t affect datacenters because they don’t use that much water, but well at least get rid of the most inefficient agricultural practices.
Re: We must pace the frontier
#483Earlier quoted context omitted.
I dunno, do you really want to bet that if you took a deliberately unaligned frontier model and asked it to permanently disable the US power grid, it would do a poor job of it? What odds would you take on this bet? I’m disturbingly close to 50/50.
If we focus too much on the world domination fiction, we really might, in a moment of lapsed attention, end up plugging essential infrastructure into the public internet, hoping some dream of global alignment might save us. That's like giving a gun to a monkey and hoping the monkey is trained well. ..a frontier monkey though.
You believe essential infrastructure is air-gapped from the public internet today?
Re: We must pace the frontier
#484Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…
[deleted]
Re: We must pace the frontier
#485Earlier quoted context omitted.
There are very lightweight models out there. Not everyone in an army is a general. The swarm could be 100,000s of thousands of tiny models, and they could manage conventional botnet computers, and they could all take direction and guidance from a handful of frontier generals
Maybe, but I don't think tiny models can be harnessed as intelligence. As in, if you have one rogue Mythos overseeing the swarm, it only produces 1 Mythos's worth of useful thoughts no matter how many gemma4:e4bs it consists of. And that removes the most dangerous part of AI-controlled botnets, which is a blowup in available inference compute - the Mythos general might as well replace the tiny models with ordinary wo…
I mean it is a very loose analogy, but pretty dumb things in the world can cause big problems, like real mice and rats and mosquitoes. I think it all depends on the rules and the 'alignment', but a not-so-smart model could still be narrowly focused to do a few things well? So you can have an army of lightweight specialists?
The other thing is: look at all the fab capacity that is being brought to bear on this. I guess within a couple of years we are going to have 5x the total online HBM that existed a year ago? As consumer machines and phones get far more powerful, they will start to be able to host models that could meaningfully participate, even if they will be a long way from mythos.
Re: We must pace the frontier
#486[In 2013], "Agents" caused Knight Capital to lose $450M in 45 minutes [1]. Implemented by humans and effected by computers, in the end it was really because of two reasons: * multiple levels of inappropriate controls and unintended consequences in several complex systems * the inability, both politically and technically, to turn it off [1] https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf EDIT: Comments in…
This confused me at first, so adding a tiny bit of context: The agents referred to here have nothing to do with AI Agents, and the linked report is from 2013.
Not directly relevant to the post being discussed, except as an example of how runaway automation can lead to unintended and large harmful consequences.
Re: We must pace the frontier
#487> Therefore any agreement must either have ironclad verifiability, or must be limited enough that defection would not be militarily existential. How does he propose that such a thing will work? Are we going to have US AI inspectors in China and vice versa? Why would China agree to such thing in the first place since this is a winner takes all situation. If the US slows down or stops altogether, China can continue to…
What sandbox? As soon as the weights are published anyone can run it anywhere.
Re: We must pace the frontier
#488Earlier quoted context omitted.
Sorry boss but this ain’t it. I don’t think the answer to nuclear proliferation is to let everybody have all the nukes they want. You’re framing this like it’s 1917, or 1945. But it is not - and likely has very little to do with capital at all. Any “revolution” here (assuming there is some truth in the doomerism which I believe there is given the state of AI) will really look more like an extinction or a genocide, re…
Sorry boss we live in a physical world, unplug it. Don’t grant information monopolies.
Ok - but the post I was responding to was suggesting we do the opposite and provide the intelligence to the people so I’m not sure how your reply rebuts my suggestions.
Re: We must pace the frontier
#489Don't you think that AI hate will limit the general use and then revenues so it will slow down by itself while the niche (AlphaFold for instance) will remains ?
Origins of AI hate:
* "my boss wants me to use AI but he does not understand my job nor how AI works"
* "AI will kill all of us"
* "AI will destroy my job (or my colleague's job if I use AI better than him)".
* I cannot pay my electricity bills because of AI labs.
* ...
I cannot remember a technology that arose so much hate, and for good reasons given how it is presented. I am tempted to think that AI hate or reasonable skepticism (vs unreasonable propaganda) can, maybe, reduce funding and will keep specialized AI for real problem solving (producing tons a LOC per day is not one of them, I think).
Re: We must pace the frontier
#490Earlier quoted context omitted.
This makes no sense. The hardware is only going to get cheaper. In a decade, everybody could be running an ASI on their phones. What we need to do is to make unaligned AIs illegal and monitor for them, like we do with nuclear weapons. And make aligned AI strong enough to counter it, for deterrence and defense.
> The hardware is only going to get cheaper. Is that so? Last time I checked, RAM was getting a lot more expensive...
How long do you think that's going to last? A few more years, at most. Wail 'til the Chinese ramp up production.