Live data from Hacker News

Policy on the AI Exponential

darioamodei.com

111–120 of 280 posts

Re: Policy on the AI Exponential

#111

I know this is likely just for IPO hype but when I read things like this I sometimes wonder if I must be missing something. I use agents everyday and find them really useful and they save me a lot of headache. At the same time I find that if I let it self-direct at a high level at all it generally makes bad choices that cause me headaches later so I can’t really give them autonomy. Enough people seem to believe this…

What did your AI-assisted workflow look like 1 year ago? I can only speak for myself, but I would carefully specify a class or module in great detail and then hand it off to the model to implement, then carefully review the result.

How about 2 years ago? Back then, I wouldn't even trust it to write a 5-line function without making some sort of silly mistake.

Today, I can leave an agent running by itself for 20 or 30 minutes and most of the time, it comes back with a result that's either flawless or can be refined to be good with a few back and forth messages. Maybe I still have to make some high-level decisions ahead of time, but all of the details, including exploring the codebase and figuring out what to do based on that, can be left to the agent. The amount of improvement just in the last 2 years has been staggering.

Now extrapolate how things will look if the trend continues for another 2 or 3 years.

Is this guaranteed to happen? No. But people have been predicting that we're going to hit a wall for a long time now, and we haven't yet. Maybe there's a wall just ahead of us. But maybe there's not -- and the "not" case seems likely enough that we should at least be planning for it.

Re: Policy on the AI Exponential

#112

Earlier quoted context omitted.

This is a very tech-focused message board, populated by mostly tech-insiders, so perhaps a little outside perspective will help people understand. Tech people are following a religious belief system whose utopian promise is the all-powerful computer that will end all suffering. I once read an article in reason magazine from over 30 years ago about how an advanced computer in the future will bring everyone who has eve…

> Tech people are following a religious belief system whose utopian promise is the all-powerful computer that will end all suffering. Uh, I don't really think that's anywhere close to an accurate characterization of most people here. Everyone, including Dario and any researcher at any frontier lab, knows the situation is quite scary and unprecedented. There are problems that will be solved and diseases that will be c…

I dunno, I don't think an outside observer would be too hard-pressed to find the fervor with which some talk about the endless possibilities and miraculous works of AI to border on the cusp of religious.

Re: Policy on the AI Exponential

#113

I feel completely baffled by the other responses on this thread. People viewing this purely as a marketing stunt, regulatory capture or attack on their freedoms, with seemingly no appreciation of the real threat that AI could pose to society and even humanity given its current rate of progress. I'm not going to claim that the CEO of pre-IPO company has no incentive to bolster the claims of his tech, but to completely…

[deleted]

Re: Policy on the AI Exponential

#114

"AI is advancing at a lightning pace—in only four years, AI models have gone from barely being able to write a coherent line of code to writing most of the code at major AI companies. Similar gains have been made in biology, physics, math, finance, law, translation, and many other fields." This is a massive exaggeration. The advancement in the automation of computer code writing has been impressive and is obviously,…

Also I’m not denying it would have a major impact either way. But c suite have basically demanded people use ai to code. It’s not all organic

Re: Policy on the AI Exponential

#115

I know this is likely just for IPO hype but when I read things like this I sometimes wonder if I must be missing something. I use agents everyday and find them really useful and they save me a lot of headache. At the same time I find that if I let it self-direct at a high level at all it generally makes bad choices that cause me headaches later so I can’t really give them autonomy. Enough people seem to believe this…

> this exponential line of thinking

It's a clever argument because if you question it, you're reminded of the entire history of technological development which is, guess what, exponential.

You're sometimes also dismissed as not understanding the concept of exponentials. This again is clever, as it's baked into the definition that if you don't see it happening, or can't imagine it happening, well that's precisely a tell you're living through an exponential!

All the reasons you might give can be countered with, essentially, "that problem that seems clear today will go away sooner than you can imagine and when it does you'll be on the back foot, so you'd better just assume it will go away and project/plan accordingly".

The trick is entirely that one cannot possibly deny the general power of exponential progress across all of technology, it's almost a law, but it doesn't work in the other direction - no particular local technology is owed exponential growth because of this general pattern. Sometimes things just cap out at merely 'useful' and don't improve much further, no matter how much you want to believe they won't, no matter how steep the progress curve (or, indeed, line) has been up to that point.

To this point the narrative of what these tools can do over these last 3 or 4 years has always been way ahead of the reality. Everyone who works with the tools knows this.

Not everyone wants it to be true, so some will not acknowledge it and will just keep pushing this year-ahead projection as ground truth today. Many (not all) of those people aren't builders, so they don't have to deal with present reality jarring up against this projection of what ought to be possible, they're safe just talking about what should hypothetically be possible and making plans around that that won't be executed for months to years anyway. This keeps the flywheel going, and in fairness, some of the reality has actually caught up in certain ways, so some of those plans will have to some degree worked out which spins the flywheel faster still.

In the end though I just keep thinking: it's been 4 years (as referenced in the post). A lot has happened, the tools are very cool and very useful for certain things. But when I put my head up and look around in the world, even just the software world, nothing's really changed in terms of actual outcomes, in terms of new things appearing or being built that didn't exist 4 years ago. Certainly nothing feels instinctively like it's improved much, subjectively.

Maybe this is what it feels like to be in the knee of a curve of an exponential, but it seems equally reasonable this is just a breakthrough that's kind of improving at a clip you'd expect it to for all the investment put in, but fundamentally is just a new tool that needs to be slowly commercialised in an economically rational way, as we gear up for the next breakthrough which may or may not be related. Who says it must just keep improving forever? This argument never made much sense to me.

Re: Policy on the AI Exponential

#116

I like that he comes up with new laws and regulations for AI companies. Can I suggest some more? - You shall not embed copyrighted material in your models. - You shall not bombard every little website in existence with 1 million scraping queries per day. - You shall not use your political influence to pump and dump your AI (or rocket?) company. - You shall not imperill the whole IT sector by buying all CPU and memory…

"We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction." "Yeah, and quotas on web scrapers!"

The Debts and Engagements Clause of article VI of the US constitution was kind of a weird little thing to stick in there, but like, it was important to a lot of people at the time and probably helped move the needle to get the thing ratified.

Re: Policy on the AI Exponential

#117

I know this is likely just for IPO hype but when I read things like this I sometimes wonder if I must be missing something. I use agents everyday and find them really useful and they save me a lot of headache. At the same time I find that if I let it self-direct at a high level at all it generally makes bad choices that cause me headaches later so I can’t really give them autonomy. Enough people seem to believe this…

It's nice that people are genuinely curious about this. - All of your observations are absolutely dead on - Yet, we have very very very robust scaling laws that as Dario points out we've had and validated for over a decade. This extends to downstream measures like METR time horizon and compsosite benchmarks like the epoch capability index. - If you look at where you're at now, which is again dead on, you're looking a…

Are we plotting against cost? How is the capability advancement vs dollars paid for development?

By my read of the (very sparse) data, we're getting linear improvements in capability for super-linear increases in costs. [1] Indicates that by 2027 models will cost $1 billon to train. Dario estimates that model runs will cost $10 billion in 2026 [2]. That to me indicates costs are potentially growing faster than capability. Maybe by quite a bit.

If the value prop of LLMs doesn't prove out, that won't last. I'm of the opinion there is no data that shows actual economic value being delivered by models. The best data shows that LLM use might be destroying value [3].

[1] https://epoch.ai/publications/how-much-does-it-cost-to-train... [2] https://lexfridman.com/dario-amodei-transcript/ [3] https://unessays.substack.com/p/talk-is-cheap

Re: Policy on the AI Exponential

#118

Its hard to read the first half of this as anything other than regulatory capture propaganda. It really all ties together as: > AI has become a major commercial technology >Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety > AI companies that deve…

Can you explain how this is attempted regulatory capture? To start a lab now which can actually compete for the frontier (i.e. and pass the "Threshold of compute" needed to get regulated) a lab / company would need a ton of money. Surely a well funded operation of that kind can deal with the regulations.

You'd need a ton of money if you followed the unoptimized, cash burning mindset of existing AI labs. There's probably a ton of optimization that is just sitting on the table. Chinese labs have proven it can be done for way less money.

Then there's running inference service of open weights, which doesn't necessarily require opening a lab. You can grab Chinese model weights and sell inference.

Anthropic wants to make sure nobody can open a new domestic lab, or provide inference services of unauthorized open weight models, or release open weights if model is good. It is regulatory capture - it covers all areas that are dangers to Anthropic's bottom line.

Re: Policy on the AI Exponential

#119

Earlier quoted context omitted.

> Tech people are following a religious belief system whose utopian promise is the all-powerful computer that will end all suffering. Uh, I don't really think that's anywhere close to an accurate characterization of most people here. Everyone, including Dario and any researcher at any frontier lab, knows the situation is quite scary and unprecedented. There are problems that will be solved and diseases that will be c…

I dunno, I don't think an outside observer would be too hard-pressed to find the fervor with which some talk about the endless possibilities and miraculous works of AI to border on the cusp of religious.

Some people sure, but some people believe the earth is flat. What’s without a doubt is the impact AI has already had, something maybe 2 years ago was dismissed as religious fervor maybe. Statistics and robust measurements plus stable scaling laws make this pretty far from religious I would say.

Re: Policy on the AI Exponential

#120
post #5

> AI models have gone from barely being able to write a coherent line of code to writing most of the code at major AI companies. Gasoline has gone from barely being able to power stationary farm machines to now being the fuel that underpins our entire economy. So, great news all around, right? > which predict an exponential increase And was that actually delivered? Real question: If a model goes from 80% accurate to…

Yeah I’m lost by the constant use of “exponential”. What is the x and y axis when the talking heads say this?

I think people are typically referring to the task-completion time horizon at a fixed success rate [1]. That has had pretty robust exponential scaling for many years now.

[1] https://metr.org/time-horizons/

Post reply on HN