291–300 of 942 posts
> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…
It can use the compute of the computers it hacks.
I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…
Earlier quoted context omitted.
You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely un safe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
Can someone clarify this for me? How far along would the open weight models be without the frenzied pace of the frontier labs?
As a non-US citizen, which authoritarian powers is he referring to? As far as I'm aware, the US is the only country using frontier models to conduct air strikes on civilians and orchestrate assassinations of foreign governments.
Earlier quoted context omitted.
Who would you pick for "embedded evaluator"? "deep ties to everyone in the doomer media campaign" Are you suggesting these external NGOs were spun up as part of a gigantic pre-IPO hype stunt? METR was founded multiple years ago. This "stunt" is getting quite elaborate. https://substackcdn.com/image/fetch/$s_!O0R5!,f_auto,q_auto:... At a certain point, Occam's Razor says: These engineers are legitimately worried. They…
Founded several years after Anthropic. Like I said, look into where their funding is coming from. I can promise you it all leads back to Anthropic through their chain of NGOs. Are you aware that Dario's sister, president of Anthropic, is married to the co-founder of Open Philanthropy? The two largest AI doomer NGOs, Center for AI Safety (CAIS) and the Future of Life Institute (FLI), have both received many millions o…
Suppose a big foundation both funded clean energy technology, and also NGOs encouraging people to decarbonize. Is this just a cynical attempt to rig the game in their favor?
This is the most advanced technology ever developed because it can build all other technologies. It has enormous potential but poses proportionally serious risks, so what Dario is suggesting is very reasonable.
It's clear that this refers to China. With all due respect, in a call to slow down AI progress and a de facto arms race, can we stop throwing terminology like this around and just say that the goal is to work with all countries? Contrasting "authoritarian" countries with "democratic" countries and creating a two-tiered system seems like it is inevitably going to cause strife.
I fear for the tone of something like this coming off as overly combative without any gain.
Remember when we stopped developing nuclear power for no reason?
I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…
There's also huge market pressures and probably large negative economic consequences of a slowdown--probably better than what would happen without it, but it helps keep the pressure just as much as fear of competition
I’m disappointed in the level of groupthink reflexive cynicism I see from commenters any time prominent AI leaders talk about AI risks and the need for regulation or pacing. Yes, regulatory capture is a risk, but this is also a profoundly unusual, fast moving, and potentially extraordinarily dangerous technology. There are strict regulations around nuclear weapons, as well as around US financial, energy, and other in…