When has giving power to politicians have had a good result? Please don’t
Policy on the AI Exponential
251–260 of 280 posts
Re: Policy on the AI Exponential
#252Earlier quoted context omitted.
> So it is absolutely strange and contrasting to see you believe that LLMs are so weak as to create negative value while the CEO is asking about regulations because AI is too powerful. You wouldn't ask a chemistry professor to write code. So just because LLMs create negative value for software development doesn't mean that they can't be helpful for bioweapons synthesis, especially considering the range of chemistry a…
This is a ridiculous stance to take. That LLMs are simultaneously negative value but can also help synthesise bioweapons. It’s the sort of stance you take when you already feel ideologically against AI. I don’t think it’s coherent.
It's more about information availability rather than intelligence. An LLM has had access to more information during its training period than you'd ever even come across over a hundred lifetimes. It has been trained on billions of books and articles across every single subject that exists on the planet. Can you imagine what real intelligence could do with all that information?
Re: Policy on the AI Exponential
#253Oh yes, let's induce regulatory capture and ban open-weight models in the name of alignment.
Re: Policy on the AI Exponential
#254I like that he comes up with new laws and regulations for AI companies. Can I suggest some more? - You shall not embed copyrighted material in your models. - You shall not bombard every little website in existence with 1 million scraping queries per day. - You shall not use your political influence to pump and dump your AI (or rocket?) company. - You shall not imperill the whole IT sector by buying all CPU and memory…
> Can I suggest some more?
Dunno if you can, but your examples here are the legal equivalent of that time someone asked me about making "Uber for airplanes" without any elaboration on their part when I asked for it:
Far more vague than I think you realise.
You could probably write a book on each of those topics and a hundred others besides.
Re: Policy on the AI Exponential
#255Earlier quoted context omitted.
"We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction." "Yeah, and quotas on web scrapers!"
> We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction. That's easy. Stop training your AIs on cheesy old sci-fi that talks about robot uprisings. In fact, maybe y'all should just stop talking about robot uprisings altogether. Putting a stochastic parrot in charge of an agentic function-calling REPL doesn't somehow make it super-dangerous, except to the extent that dumb mistak…
> except to the extent that dumb mistakes might result in danger
That "except" goes all the way up to starting WW3. Or a leak from a viral research lab, and by "leak" I mean "mail order" and by "research lab" I mean "the companies who already ship custom DNA and RNA retroviruses": https://duckduckgo.com/?q=companies+who+already+ship+custom+...
If you can prove that simply not training on horror stories would work, it would make a lot of people very happy.
Unfortunately, I don't think it does a single thing to solve, for example, Elon Musk just plain asking some future version of Grok to take over the world for him.
Nor would merely failing to include them in traing data stop certain entire fictional scenarios such as that Doctor Who episode where the android repair bots weren't told that the crew were off-limits as spare parts, or the other Doctor Who episode where the utilitarian robots started killing everyone who was upset because they calculated net positive utility from upset people ceasing to exist. Well, except for the bit where the Doctor saves the day, because they are not real.
Re: Policy on the AI Exponential
#256Earlier quoted context omitted.
Yes, we get that if you assume there is zero existential risk from AI, then there is zero existential risk from AI.
The biggest existential risk from AI is its contribution to global climate change. The second biggest risk from AI is the potential for AI-generated disinformation and propaganda to spark, or to manufacture consent for, a world war. The risk of superintelligent paperclip maximizers is so low as to be negligible.
> The risk of superintelligent paperclip maximizers is so low as to be negligible.
Literal paperclips, sure.
But the point of the example was never literal paperclips.
The point is that maximising *any* goal, if it doesn't include what you care about, will annihilate what you care about.
If you don't believe me, consider what you yourself just said about climate change, and why this is a consequence from maximising money spent on data centres.
Re: Policy on the AI Exponential
#257Earlier quoted context omitted.
Respectfully, your link is not very convincing. But what’s interesting is that you are commenting on a post where Dario is suggesting that LLMs are so extremely powerful that they can take over, help synthesise bioweapons, help in warfare, help in drug discovery — the whole post here is to try and regulate this. If you believe AI can’t even create positive value let alone discover new things then your problem is some…
>Respectfully, your link is not very convincing. I'd love to understand why. This would be valuable feedback for me as I try to make my writing and exposition better. Also, if you have other data, that also would be valuable for me to know. >if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts.…
You have engaged in good faith discourse, thanks! I'll reply in a bit.
Re: Policy on the AI Exponential
#258Its hard to read the first half of this as anything other than regulatory capture propaganda. It really all ties together as: > AI has become a major commercial technology >Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety > AI companies that deve…
> Its hard to read the first half of this as anything other than regulatory capture propaganda
I wonder if any tech company managed to thrive long in history by betting so violently on fear mongering and regulatory capture.
Re: Policy on the AI Exponential
#259Earlier quoted context omitted.
I appreciate the data here but I don't think the read is quite right; Saying we have linear capability for super-linear cost compares an unbounded variable (dollars) to bounded instruments (because benchmarks saturate). On unbounded measures, growth is exponential; you can see METR time horizons double every ~4-7 months ( https://metr.org/blog/2026-1-29-time-horizon-1-1/ ). And capability being proportional to log(co…
METR's time horizon is not a reliable metric of LLM capability growth: https://www.transformernews.ai/p/against-the-metr-graph-codi...
Yes I've seen this before, and while the critiques are fair and high quality (and unfortunately not unique to METR) we're missing the forest for the trees here.
First of all, if you take the articles critiques and work out the implications on the METR graph, all you're doing is shifting the curve up or down, it doesn't change the fact that progress is scaling exponentially. While it is technically possible the universe could be throwing a massive pathological curveball to change the conclusion from METR data (which is we've been seeing exponential growth over the last 6 years), I think that seems very far from likely. The fact that we see the same behavior from a variety of sources over a wide variety of tasks and domains is a pretty clear indication that METR while certainly far from perfect is actually painting a consistent picture at least in terms of the rate of progress.
You can look at ECI for a summary benchmark statistic, which does NOT use METR's benchmark, and you see a similar trend. Same with SWE-bench where the task distribution is far more in domain for real world problems. It is a bummer that this METR data can't be better funded. It would probably take $1M or so to really beef it up properly which any of these labs probably have in their couch cushions.
Re: Policy on the AI Exponential
#260Earlier quoted context omitted.
Respectfully, your link is not very convincing. But what’s interesting is that you are commenting on a post where Dario is suggesting that LLMs are so extremely powerful that they can take over, help synthesise bioweapons, help in warfare, help in drug discovery — the whole post here is to try and regulate this. If you believe AI can’t even create positive value let alone discover new things then your problem is some…
>Respectfully, your link is not very convincing. I'd love to understand why. This would be valuable feedback for me as I try to make my writing and exposition better. Also, if you have other data, that also would be valuable for me to know. >if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts.…
I went through your post in substack (I think that's what you were referring to).
> I'd love to understand why. This would be valuable feedback for me as I try to make my writing and exposition better. Also, if you have other data, that also would be valuable for me to know.
I think it comes down to few things
- you took a single report that agreed with your statistics, for the sake or argument lets say I buy it completely
- you suggest that net value is lost simply because there are more incidents. this is a big jump
- you say that historically different technological improvements may have had similar patterns but this specific one is different because AI is stochastic
So it all really rests on you finding one distinction with AI and then disagreeing with the past trends.
I agree AI is stochastic and I'll put it this way: it is a high variance bet but it pays off. This is a bit hard for people to understand -- its a tool that works sometimes really nicely and fails other times. Overall you are better off using it but you need to use it enough to reduce variance.
Let me ask this: if you are so sure this won't lead to enterprise level productivity, how do you think this will show in macro trends? Surely you must believe that the valuations must drop wouldn't you? Can you come up with a concrete future scenario that would vindicate your opinion that AI doesn't make enterprises more productive?
> My question would be - why would Anthropic build something they so clearly think is dangerous? If they were really building something deserving of the valuation they have, why build applications like this?
I think this is fair and interesting question. Here is what I think they think: If they don't build it, someone else might do it. And they think they are more moral than others. If they have a head start they can set the political and regulatory landscape.