Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

951–960 of 965 posts

Re: We must pace the frontier

#951

Post by Davidzheng »

Davidzheng wrote: post #950
augment_me wrote: post #638

Earlier quoted context omitted.

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".

Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero.

Caveat I know nothing about economics lol--but this is my uninformed opinion.

Re: We must pace the frontier

#952

Post by Davidzheng »

ls612 wrote: post #550
Davidzheng wrote: post #291

Earlier quoted context omitted.

??? Why It can use the compute of the computers it hacks.

lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth , these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.

Re: We must pace the frontier

#953

Post by the_gipsy »

boshalfoshal wrote: post #803
un_montagnard wrote: post #598

Anthropic is about to go public, but they figured they can't keep making improvement at the same pace they used to. If the frontier is paced, they can continue hyping their unreleased capabilities that coincidentally cannot be released due to things outside their control. Which buy them more time to try to improve the models.

This is just completely false lol. They most certainly can make better models - why cant people here just read this for face value? I think anthropic is legitimately concerned about the safety implications of stronger models. Race dynamics necessitate that they make better and better models, which they clearly think is bad. They have an internal model which could solve a millenium problem end to end with no intervent…

> why cant people here just read this for face value?

Why should we? Yes, reading the comments, it seems like the IT (or maybe it's just the AI) community has taken up an attitude of simply believing our Dear Tech Leader's by their word. What happened?

Re: We must pace the frontier

#954

Post by tripzilch »

jrs100000 wrote: post #934
aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Yes this seems like a perfectly sensible approach? You say that as if it's a bad thing

But let's be honest, if I hooked up a PRNG to a terminal and somehow against all odds, it ended up hacking something, who is to blame?

I don't see how that assessment should change if the PRNG gets even better and is more likely to to be hacking stuff.

You can replace PRNG with Markov Chain, or whatever, if it helps.

What may also help is the age old saying: If everybody else jumps off a bridge, doesn't mean you should too.

Re: We must pace the frontier

#955

Post by tripzilch »

imtringued wrote: post #946
jrs100000 wrote: post #934

Earlier quoted context omitted.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Human approval does not prevent autonomous agents from running without supervision. The fact that you believe that to be the case is exactly what's wrong with "agentic computing as it's currently envisioned".

If the LLM's output doesn't reach my bash terminal, what's it gonna do? Argue with me?

Re: We must pace the frontier

#956

Post by tripzilch »

itake wrote: post #933
aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

I’m trying to imagine where a Waymo passenger (the one “operating” the vehicle, commanding the AI to drive from A to B) being held responsible for the car doing something illegal on the way to achieve that goal. Do you really think that the passenger should be responsible for how the car/agent achieves the goal, when they only set the destination? Giving passenger override controls and monitoring seems to defeat the…

TBH I don't really think another new technology we haven't figured out the ethics of, as an example, is going to get you very far in way of insights ...

Re: We must pace the frontier

#957

Post by olelele »

spopejoy wrote: post #870
cavemandaveman wrote: post #658

Earlier quoted context omitted.

America doesn't want to pay the price of handling Iran. There's a difference. Clearly the US could conquer Iran but it requires a land war with mass casualties, not just sporadic bombing. Obviously China is in an entirely different league.

> Clearly the US could conquer Iran Vietnam (and Ukraine) would like a word

Korea & Afghanistan too.

The consequences of the Iraq blunder are also still resonating.

Re: We must pace the frontier

#959

Post by Matl »

0xDEAFBEAD wrote: post #211
Matl wrote: post #162

Earlier quoted context omitted.

For example he got into a spat with the Department of War as if he cared for how his AI could be used during war and yet said he's fine with Claude targeting a girl's school in Iran. That's apart from the general fact that he continues to race towards the very thing he claims he's afraid of, because that's where his net worth comes from.

>said he's fine with Claude targeting a girl's school in Iran Where?

https://www.youtube.com/shorts/7DOQlIQxz5M

and you know, that use case doesn't even violate their terms

Re: We must pace the frontier

#960

Post by maciejzj »

cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

Had they issued an initiative on slowing down when they had the best model in the game, it would have been both more (anyhow?) effective and less dubious.
Post reply on HN