Live data from Hacker News

Policy on the AI Exponential

darioamodei.com

241–250 of 280 posts

Re: Policy on the AI Exponential

#241
post #151

Earlier quoted context omitted.

>This is basically a conspiracy theory I think this is pretty uncharitable, especially when I've provided you with a dataset you can evaluate yourself and an argument you can review for logical inconsistency. I have worked quite hard to locate data that supports your thesis, I can't find it. I've at least gone to the effort of documenting that search. Before you throw around such strong convictions, I suggest you act…

Respectfully, your link is not very convincing. But what’s interesting is that you are commenting on a post where Dario is suggesting that LLMs are so extremely powerful that they can take over, help synthesise bioweapons, help in warfare, help in drug discovery — the whole post here is to try and regulate this. If you believe AI can’t even create positive value let alone discover new things then your problem is some…

>Respectfully, your link is not very convincing.

I'd love to understand why. This would be valuable feedback for me as I try to make my writing and exposition better. Also, if you have other data, that also would be valuable for me to know.

>if you believe what you believe, you should also acknowledge that AI doesn’t need regulations in the context Dario is proposing since obviously AI can’t do anything he predicts. Do you agree?

I think you misunderstand my beliefs. On net I think how we're using LLMs destroys value. That doesn't mean no one ever gets value from LLM use.

My particular point about trillion dollars is - the main place Anthropic, OpenAI, and - hilariously - SpaceX think they will drive value creation is in enterprise applications. In that domain I think the evidence is very convincingly negative. I'm certainly not the only person who thinks this. It's pretty well accepted in economics right now that there is no observed organizational level productivity improvement. Lines break down on whether it will show up eventually or whether we will wait forever.

My belief about LLM value is that it's most useful for individuals and small teams. Places where coordination and trust are easily established and feedback loops to value creation are tight. They are "short range" as it were.

Their value starts to erode as soon as a user becomes disconnected from the point of direct value creation. Which is pretty much everyone who works inside of a large organization. It becomes negative at pretty small scale, IMO. I do think there are patterns of use that could drive value at these scales. I talk about that in my post.

On Bioweapons in particular, I could see small teams of people working to build something very dangerous. Having spent my formative academic years in a biochemistry and microbiology lab though, I do think the danger is overstated. Papers are not know-how or equipment. There's a lot of tacit knowledge that can't get written down that is super hard to acquire.

But, I'd be happy for us to regulate AI for dangerous applications.

My question would be - why would Anthropic build something they so clearly think is dangerous? If they were really building something deserving of the valuation they have, why build applications like this?

To my eyes - it's super weird that a company would build something they think is dangerous and turn around and beg the governments of the world to stop them. That's really strange behavior from my perspective.

Re: Policy on the AI Exponential

#242
post #215

Earlier quoted context omitted.

> So it is absolutely strange and contrasting to see you believe that LLMs are so weak as to create negative value while the CEO is asking about regulations because AI is too powerful. You wouldn't ask a chemistry professor to write code. So just because LLMs create negative value for software development doesn't mean that they can't be helpful for bioweapons synthesis, especially considering the range of chemistry a…

This is a ridiculous stance to take. That LLMs are simultaneously negative value but can also help synthesise bioweapons. It’s the sort of stance you take when you already feel ideologically against AI. I don’t think it’s coherent.

>This is a ridiculous stance to take.

I encourage you to be more curious. We're just talking. We would learn more from each other without these strong statements.

Re: Policy on the AI Exponential

#243
post #152

Earlier quoted context omitted.

>"On unbounded measures, growth is exponential" Maybe. There was a great comment in the thread on Fable 5 yesterday about benchmark comparisons between Fable and the latest opus models. here it is: https://news.ycombinator.com/item?id=48464600 . You could be right, but this is the most direct benchmark comparison I could find and it's not that strong. >the "destroying value" conclusion flips sign on an assumed 15% ba…

> But even if it were not true, a 16% throughput improvement is pretty weak given the investment - especially given the direct evidence of quality degradation. IMO. n=1 but at $JOB we have throughput quotas now, and what is happening is that teams are just finding lots of busywork (renaming things, gardening of ai .md files, rewriting uis etc) and also dividing prs into smaller chunks to match the quotas... so even "…

Productivity != value.

Thanks for the story.

Re: Policy on the AI Exponential

#244
post #215

Earlier quoted context omitted.

> So it is absolutely strange and contrasting to see you believe that LLMs are so weak as to create negative value while the CEO is asking about regulations because AI is too powerful. You wouldn't ask a chemistry professor to write code. So just because LLMs create negative value for software development doesn't mean that they can't be helpful for bioweapons synthesis, especially considering the range of chemistry a…

This is a ridiculous stance to take. That LLMs are simultaneously negative value but can also help synthesise bioweapons. It’s the sort of stance you take when you already feel ideologically against AI. I don’t think it’s coherent.

Is synthesizing bioweapons positive or negative value ?

Re: Policy on the AI Exponential

#245

Earlier quoted context omitted.

>why is all I hear these industry leaders talking about is the threat of extinction? . . . I would love to see a leading lab release research on the health effects of subaudible sound in datacenters It is straightforward for industry leaders to avoid living near data centers, but there's no way for them to insulate themselves from the extinction threat -- no way short of somehow eliminating the danger for everybody,…

I see. I wonder how this works out in terms of risk/reward. I suppose if you take extinction as -infinite cost than it would be the only issue worth thinking about. Where I think this line of thinking gets challenging is when you need to take in terms of a counter factual. A lot of these were already risks prior to AI (bioweapons, nukes, etc) so what’s the marginal increase in probability as a result of AI I guess is…

I'm doubtful of this idea that the reason the CEO of Anthropic says that AI could end the world is because he "stands to gain huge" by saying it.

If he is willing to lie to give his corporation some advantage, he couldn't come up with a lie that would sound less absurd and outlandish to the average decision-maker, who doesn't have much time to learn about this particular technology?

It is more likely in my eyes that he says it because he genuinely worries about AI extinction risk -- like many people do who've studied the technology for a long time.

Re: Policy on the AI Exponential

#246

Earlier quoted context omitted.

> after we have AIs that are better at evrything than humans. That this is worded so definitively is a testament to the success of the AI industry. The idea that LLMs will be "better at evrything than humans"[sic] is far from certain. I suspect that if someone does invent a machine like this, it won't look like a 2026 LLM, and it will be far far in the future. everybody relax.

“Hey everyone, no need to worry, bigfishrunning suspects it’ll be fine!”

I'm glad someone agrees! in all seriousness, there's really nothing you can do about it, so why worry? If the AI monster is gonna get us, then you might as well enjoy your life until that point.

Re: Policy on the AI Exponential

#247
post #227

Earlier quoted context omitted.

It is simply because you have spent all your life being told what to do with your working hours, that you cannot self-direct and find a productive use for your time other than lazying around. The fact that you call it ‘funemployment’ is proof of this, which is perfectly fine if the goal is relaxing between jobs. Plenty of people that work for themselves have no such notion.

It takes time to re-ground yourself, to renew your ambitions to new circumstances. It usually takes more than a year even for employees.

Oh yes, it's been 3 years for me of being 'untethered' and it's still so easy to just waste my day if I am not careful with distractions. And the feeling that I should just stop being silly and I should find a normal job like a normal person will often find me and ruin my days.

Re: Policy on the AI Exponential

#248

Earlier quoted context omitted.

> We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction. That's easy. Stop training your AIs on cheesy old sci-fi that talks about robot uprisings. In fact, maybe y'all should just stop talking about robot uprisings altogether. Putting a stochastic parrot in charge of an agentic function-calling REPL doesn't somehow make it super-dangerous, except to the extent that dumb mistak…

Yes, we get that if you assume there is zero existential risk from AI, then there is zero existential risk from AI.

The biggest existential risk from AI is its contribution to global climate change. The second biggest risk from AI is the potential for AI-generated disinformation and propaganda to spark, or to manufacture consent for, a world war. The risk of superintelligent paperclip maximizers is so low as to be negligible.

Re: Policy on the AI Exponential

#249

Earlier quoted context omitted.

Sorry, audibly chuckling about the idea of software developers who consider themselves "everyday working class people". Keep it coming, Tom Morello.

Believe it or not, but not every software developer is making 7 figures annually with a few millions in vested shares. Even within the US.

I see, the line between "working class" and "upper class" is $999,999 in total comp.

Re: Policy on the AI Exponential

#250

Earlier quoted context omitted.

I appreciate the data here but I don't think the read is quite right; Saying we have linear capability for super-linear cost compares an unbounded variable (dollars) to bounded instruments (because benchmarks saturate). On unbounded measures, growth is exponential; you can see METR time horizons double every ~4-7 months ( https://metr.org/blog/2026-1-29-time-horizon-1-1/ ). And capability being proportional to log(co…

METR's time horizon is not a reliable metric of LLM capability growth: https://www.transformernews.ai/p/against-the-metr-graph-codi...

Wow. This deserves to be much more widely read. Thank you for this.
Post reply on HN