Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

951–960 of 1001 posts

Re: We must pace the frontier

#951

Post by Dlemlo »

cuuupid wrote: post #433

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

In good faith? In what context? When? Who?

Its a company, it sells a product, people use it.

Re: We must pace the frontier

#952

Post by nsdkdkdkfk »

boshalfoshal wrote: post #856
eknkc wrote: post #847

Earlier quoted context omitted.

> which they clearly think is bad These are companies with huge amounts of money invested in them. Their main responsibility is to their investors. If building better models (or making more money from the models they already have) would increase returns, they would do it. So they clearly can’t do it right now. Climate change hasn’t stopped the oil industry, and concerns about forever chemicals haven’t stopped the che…

I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance. Re the point on returning shareholder value, Anthropic is a PBC: > Public Benefit…

[dead]

Re: We must pace the frontier

#953

Post by toss1 »

cuuupid wrote: post #433

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

On the topic of regulatory capture, there is this rumor that Google has achieved Recursive Self Improvement, and the other three (OpenAI/Anthropic/X) are just trying to get the regulators to slow down Google so they can catch up.

Yes, what is in the labs is almost by definition ahead of what we can see, but I've come to take all of these announcements of impending doom as perverse marketing stunts — it is so powerful Disaster-XYZ is coming — so if it's that power you must invest...

[0] https://wccftech.com/a-wild-rumor-says-google-is-on-the-verg...

Re: We must pace the frontier

#954

Post by bwfan123 »

braydenm wrote: post #258

Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…

> to pose existential risk to all of humanity

The defense againt rogue-AI is very simple. Go to the nearest data-center with a fire-truck, and hose it down. So, I am in the least bit worried about this.

Try this thought experiment: If someone took away AI/LLMs for good tomorrow would you miss it ? I can bet that the average person cares more for their smartphone/amazon/netflix than chatgpt. I consider LLM to be a useful tool but I would not miss it if you took it away from me. This should tell you its value to society and the economy.

I think engineers at the AI labs are too far into the reality-as-a-simulation rabbit hole. This can lead to these sorts of collective delusions. When you are in the real-world smelling the roses and grounded by your own qualia, such fears dont arise. Therapy helps here - to get out of the simulation and into the real world.

Re: We must pace the frontier

#955

Post by techblueberry »

jrs100000 wrote: post #870
aarondong wrote: post #868

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

This seems like one of those things that prior to AI companies convincing us otherwise would have been obvious.

Least privilege and say only opening ports or installing applications an application needs to operate are extremely standard security practices.

We talk about a firewall blocking exultation of data, why not blocking exfiltration of your agent ?

Re: We must pace the frontier

#956

Post by RGS1811 »

bennydog224 wrote: post #531
RGS1811 wrote: post #493

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

I agree it’s not all altrusim. It’s a little less clear what you mean at the end though. For these companies, is your argument that “pacing the frontier” is their attempt to be nationalized and protect their investments?

No, my argument is that the need to pace the frontier means they cannot safely advance in capability due to liability concerns. The competition is already almost caught up. If OpenAI/Anthropic have hit an upper bound on safe capability improvement, the gap will close all the way, and we will have reached the full commoditization of LLM tokens very soon.

Re: We must pace the frontier

#957

Post by pastel8739 »

Razengan wrote: post #864
pastel8739 wrote: post #245

Earlier quoted context omitted.

Why? I am only excited about progress that improves life for humans. It seems unlikely that AI will do that, and so far I think it has made life worse for humans. So no, I am not excited about it.

Because the people whose lives are being improved* by AI aren't coming here to post about it, only to be downvoted by dumbasses who can't handle anyone saying anything except their opinion. And no, you don't get to decide what counts as "improvement"

Yes I do, when I’m forming my own opinion. You do too.

Re: We must pace the frontier

#958

Post by ethagnawl »

gverrilla wrote: post #648
ethagnawl wrote: post #589

> I believe that AI could cure most major diseases in the next 5–10 years This is Theranos-level bullshit. Why would you ever put such a thing in writing? (Aside from pumping the IPO, of course.)

Why would you expect anything different from the CEO of a company that's going to IPO?

I mean, I said as much.

My point, though, is that I've never heard a pre-IPO CEO claim that their company will probably transmute lead into gold or harness 100% of the power of a distant sun is the next 5-10 years.

Re: We must pace the frontier

#959

Post by pbrumm »

I think they should acknowledge that they are liable for damage done now and in the future by their servers

Publish all their external ips and if hacking is attempted through them have easy and strict monetary payout with independent review process. Model it after the CMS IDR process

If they are caught using proxies 10x the fines

Limit their GPU quantities and force them to be more efficient with their resources.

Enforce attribution on their results

Re: We must pace the frontier

#960

Post by isthispermanent »

cuuupid wrote: post #433

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

100%, rules for thee not for me
Post reply on HN