Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

371–380 of 935 posts

Re: We must pace the frontier

#371

Post by figassis »

> To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models

I have seen this before, first hand, on lower stakes for the world, but high stakes for the org. Self imposed safety goals do not last a quarter, but the clawback is not transparent. little things start snapping back internally, new, seemingly unlrelated initiatives are funded that take resources away from this, people are let go for different reasons, until 1y later you are right back where you started. I am hopeful though, that this becomes systemic and not something any one company can easily reverse course on.

Re: We must pace the frontier

#372

Post by rickydroll »

I also like the idea of pacing the frontier and slowing down development so we all have a chance to stop and catch our breath. However, I don't think regulating AI is the way to do it. I would tackle the problem by constraining the resources used to run AI models; the "simplest" way to do that is increasing the costs of running a data center. The two easiest places to raise data center costs are electricity and water tariffs.

I think those two tariffs are the best place to raise costs because they can be increased incrementally, state by state or country by country. The main problem with the regulatory approach is that it's a "big bang" post-board approach. Nothing happens until the regulation is defined and deployed, and corporations are experts at delaying implementation.

Yes, increasing tariffs means touching many individual regulatory domains at the state and municipal levels, but like deploying solar energy or wind power, you can do it incrementally.

Re: We must pace the frontier

#374

Post by wmf »

credit_guy wrote: post #217
Jcampuzano2 wrote: post #63

Earlier quoted context omitted.

If someone is genuinely afraid of this, they wouldn't IPO in the first place. All the talk in the article about commercial incentives means nothing when the company plans to IPO and become beholden to investors.

I don't think Anthropic and OpenAI will IPO at all. Word is that Anthropic will have $100 BN in revenues this year, and very likely OpenAI will get some similar amount. You IPO when you need money, and I think Anthropic and OpenAI are past that point.

They want to spend over $200B/year on $100B of revenue so they still need investment.

Re: We must pace the frontier

#375

Post by neomantra »

[In 2013], "Agents" caused Knight Capital to lose $450M in 45 minutes [1]. Implemented by humans and effected by computers, in the end it was really because of two reasons:

* multiple levels of inappropriate controls and unintended consequences in several complex systems

* the inability, both politically and technically, to turn it off

[1] https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf

EDIT: Comments indicated I was confusing, so I added a date to make clear that this is pre-LLM agents. My apologies, I intended to illustrate parallels and the post-mortem so we can learn from it.

Re: We must pace the frontier

#376

Post by dickersnoodle »

Razengan wrote: post #148
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

> genuinely afraid of the inevitability of AI turning into I'm going to get pitchforked on this bandwagon, but is really no one here genuinely -excited- about AI? of it turning into an evolution of sentience, or it turning out to be our first contact with alien intelligence? "durrr it's just matrix multiplications" mfer so is your brain. What humans should be doing is overhauling archaic social institutions to keep u…

>"durrr it's just matrix multiplications" mfer so is your brain.

Tell me you don't know anything about neuroscience without telling me you don't know anything about neuroscience.

Re: We must pace the frontier

#377

Post by polytely »

nickysielicki wrote: post #157
angusturner wrote: post #115

Earlier quoted context omitted.

This seems highly optimistic to me. What's to say current AI won't be highly power-concentrating by default? The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class. On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers. No…

> The best models are owned by a few companies In terms of cost per task, the open weight Chinese models are winning by a long shot. So it depends on what you mean by the best model.

Don't you think open weight Chinese models will be the first thing they start banning once the regulation of this industry starts to kick in?

Re: We must pace the frontier

#378

Post by CoolestBeans »

TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

I am willing to give Mr. Amodei the benefit of the doubt in the sincerity of his beliefs. Everyone assumes his motivations have to be perfectly rational and can't contradict but that's not how people act in practice.

The real problem is the net effect of his actions. He is very responsible for the expanding frontier of AI. Without his company, there would not be the competition necessary to push everyone else forward nor the source models for fast followers to release open weight models in his company's wake. Furthermore, it isn't like Anthropic is substantially different in safety than everyone else, their difference is on the margins.

So you have this guy who is building something he claims will hurt us all, but he's also saying "if you don't trust me to build it you might get hurt". And like I think most people understand that this is the sort of behavior Tony Soprano would understand. More bluntly, this is a kind of extortion.

Again, I don't doubt Amodei's motivations are sincere but the actual effects of his actions paint a completely different picture at which point, how can you trust him or his company?

Re: We must pace the frontier

#379

Post by cpeterso »

kennywinker wrote: post #323
zinodaur wrote: post #294

> Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers Can someone clarify this for me? How far along would the open weight models be without the frenzied pace of the frontier labs? As a non-US citizen, which authoritarian powers is he referring to? As far as I'm aware, the US is the only country using frontier models to conduct air strikes on civilians an…

> As a non-US citizen, which authoritarian powers is he referring to? As far as I'm aware, the US is the only country using frontier models to conduct air strikes on civilians and orchestrate assassinations of foreign governments. Israel is doing that too. I believe Russia and Ukraine have both also used AI powered autonomous drones in warfare at this point - tho probably not frontier ones, since they need to run on…

> Iran and Houthi rebels used Anthropic's Claude AI to target US warships and build hypersonic missiles — Houthi rebels also used the bot to code ballistic missile guidance systems

https://www.tomshardware.com/tech-industry/artificial-intell...

Re: We must pace the frontier

#380

Post by nullbio »

0xDEAFBEAD wrote: post #295
nullbio wrote: post #256

Earlier quoted context omitted.

Founded several years after Anthropic. Like I said, look into where their funding is coming from. I can promise you it all leads back to Anthropic through their chain of NGOs. Are you aware that Dario's sister, president of Anthropic, is married to the co-founder of Open Philanthropy? The two largest AI doomer NGOs, Center for AI Safety (CAIS) and the Future of Life Institute (FLI), have both received many millions o…

Suppose I showed you a number of climate change NGOs which shared staff and funding sources. Could we therefore conclude that their motives aren't genuine? Suppose a big foundation both funded clean energy technology, and also NGOs encouraging people to decarbonize. Is this just a cynical attempt to rig the game in their favor?

Pure cope.

> Suppose a big foundation both funded clean energy technology, and also NGOs encouraging people to decarbonize. Is this just a cynical attempt to rig the game in their favor?

Not comparable. The clean energy technology company wouldn't be trying to create an environment where no one else can create clean energy technology, or where only they are the ones who can decide how clean energy technology is created or used.

Post reply on HN