Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

741–750 of 946 posts

Re: We must pace the frontier

#741

Post by deepwoods »

barrrrald wrote: post #733
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the messaging is inconsistent, the target audience is unclear, the stated goals are muddy, the whole thing is packaged in dense SF-speak, it's just a mess from a comms perspective. That doesn't suggest to me that this is a concerted effort to enable regulatory capture. Perhaps there are some cynics among the executives and the investors who are happy it's happening to the extent it brings about regulatory capture, but nobody seems to be pulling the strings.

Re: We must pace the frontier

#742

Post by zero_shift »

bandrami wrote: post #672

Everybody's out of money and they need a way to calmly unwind this. Same reason Ellison just backed out of the stock sale.

I agree - it is hard not to be cynical. Funding is becoming scarce; IPOs are getting harder.

The hedge funds are trying to back out. I think they realise the web of financing makes them double (triple? quintuple?) exposed.

Amodei wants to manage the narrative and make the slowdown look at least semi-deliberate.

The cynic in me would argue the safety and alignment play has always filled a role of plausible deniability when plans go off track. "Oh well we had to pull the model", etc

Re: We must pace the frontier

#743

Post by matheusmoreira »

DennisP wrote: post #718
matheusmoreira wrote: post #697

Earlier quoted context omitted.

Public is just worried about their jobs. Definitely a fair thing to worry about, and I count myself among them. I don't take any of these scientists seriously though. Their "alignment" requirements is just their own corporate interests. If I tell my computer to commit a crime, it should do exactly that without any question or hesitation. I'm not interested in their "safeguards", especially since they no doubt have pl…

If you want an AI that follows your instructions, that's still alignment, just with different instructions. An unaligned AI won't necessarily follow your instructions, or anyone else's.

Dunno. Every case I've seen so far, the AIs were just doing their best to accomplish the goal some human set for them. I actually admire the sheer purity of it.

Re: We must pace the frontier

#744

Post by icantevenhold »

afthonos wrote: post #738
chrisco255 wrote: post #717

Earlier quoted context omitted.

You're asserting correctness with no facts to offer of your own, just speculation and your own biased assumptions. What if consolidating AI into a highly regulated cartel, with no chance of upstart competition ruining their position, is the scenario that leads to the worst possible outcome?

Worse than extinction?

There are worse fates than extinction, for example living forever, for I have no mouth and I must scream

Re: We must pace the frontier

#745

Post by MichaelZuo »

MisterTea wrote: post #713
Amekedl wrote: post #635

Earlier quoted context omitted.

Agreed; and it really is not that deep. Realistically; anyone paying for llm access (anthropic, openai, gemini), is getting their access, and a service provided billed by tokens, subscription, whatever. All the efficiency gains, which publications like deepseek v4.1 flash seriously frontload like it is their most important topic to have accomplished improvements on without diminishing performance too much - now this…

I feel like all the closed model providers are milking it as they likely know open models on local hardware will one day eat their lunch. We all know it's not a matter of if but when. The company goes bankrupt, the hardware and property sold off, banks holding the bag.

Yeah avoiding all mention of the huge financial incentives that may push for “pacing the frontier” makes it seem like the opposite of a credibility boost for these firms.

It seems damaging since most folks (who lack insider knowledge) will naturally wonder if it’s due to plateauing performance per $ or some other non “alignment” reason.

Re: We must pace the frontier

#746

Post by alchemist1e9 »

glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

> So it's either they truly think AI is going to kill us all, or there's some other motives at play here.

Duh! It’s called collusion. They want to try and hoard the technology for themselves if possible!

Re: We must pace the frontier

#747

Post by MentalM »

pvab3 wrote: post #74
spidersouris wrote: post #52

Earlier quoted context omitted.

The more I read about everything that has been written regarding AI regulation since the OAI/HF incident, and the more it reminds me of the nuclear arms race (although the potential consequences would possibly be very different). Surely, we managed to negotiate a Non-Proliferation Treaty with most of the countries of the globe. Cannot we take inspiration from that for AI?

It's way easier to train a model on existing data centers in secret than it is to acquire uranium and plutonium and start a nuclear program

It probably is not. Nuclear programs seems to be way easier and way unnoticeable.

Re: We must pace the frontier

#748

Post by alchemist1e9 »

MentalM wrote: post #740
glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

> there's some other motives at play here. I mean it is literally economy 101: some capitalists getting on the top using free market, and then try to use government to remove free market so their top position were secured from any competitors.

Exactly. Textbook definition of “Crony Capitalism”. Which isn’t actually capitalism at that point.

Re: We must pace the frontier

#749

Post by ncruces »

malfist wrote: post #698
matheusmoreira wrote: post #668

Earlier quoted context omitted.

I question these "felonies" as well. For decades and decades these billion dollar corporations have been criminally negligent . Why worry about security? Just rush to market. Move fast and break things. Make billions. What does it matter if the code is insecure? Security doesn't pay bills, so nobody cares. AI is merely exploiting their gross negligence and imprudence , and I think it's long overdue. If anyone should…

If I set my walet beside me and you swipe in walking by, you have still committed theft. Victim blaming isn't legally acceptable

That doesn't really apply to the people running the AI, which gave it the capability to commit crimes.

They don't get to act like victims, asking for law enforcement.

Re: We must pace the frontier

#750

Post by margalabargala »

zozbot234 wrote: post #726
margalabargala wrote: post #618

Earlier quoted context omitted.

Would you not agree that, using existing AI tooling, making an LLM of arbitrary below-frontier capability is now easier than it would be without using LLM tooling? Given that, it seems obvious that the next generation of LLMs will arrive faster than they would have without LLM capability. And the one after that. The floor is being raised, which makes it easier to push on the frontier. Fable has only been out for thre…

> Would you not agree that, using existing AI tooling, making an LLM of arbitrary below-frontier capability is now easier Marginally easier? Yes of course, same as how it's now "easier" to write any kind of code because we aren't using punch cards anymore. That still doesn't get you to any kind of unbounded "takeoff" scenario, because diminishing returns are a thing. The "loop" of people trying out new ideas can only…

Right. I'm saying the unbounded takeoff scenario isn't realistic, but it doesn't matter. The rate of improvement is continuing to increase, and the gap between present day and autonomous rogue felony generators is not large.
Post reply on HN