We must pace the frontier • Hacker News Board

Re: We must pace the frontier

#951

Post by Davidzheng » Sun, Sep 13, 2026, 8:37 AM UTC

Davidzheng wrote: post #950

augment_me wrote: post #638

Earlier quoted context omitted.

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".

Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero.

Caveat I know nothing about economics lol--but this is my uninformed opinion.

Re: We must pace the frontier

#952

Post by Davidzheng » Sun, Sep 13, 2026, 8:50 AM UTC

ls612 wrote: post #550

Davidzheng wrote: post #291

Earlier quoted context omitted.

??? Why It can use the compute of the computers it hacks.

lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth , these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.

Re: We must pace the frontier

#953

Post by the_gipsy » Sun, Sep 13, 2026, 8:54 AM UTC

boshalfoshal wrote: post #803

un_montagnard wrote: post #598

Anthropic is about to go public, but they figured they can't keep making improvement at the same pace they used to. If the frontier is paced, they can continue hyping their unreleased capabilities that coincidentally cannot be released due to things outside their control. Which buy them more time to try to improve the models.

This is just completely false lol. They most certainly can make better models - why cant people here just read this for face value? I think anthropic is legitimately concerned about the safety implications of stronger models. Race dynamics necessitate that they make better and better models, which they clearly think is bad. They have an internal model which could solve a millenium problem end to end with no intervent…

> why cant people here just read this for face value?

Why should we? Yes, reading the comments, it seems like the IT (or maybe it's just the AI) community has taken up an attitude of simply believing our Dear Tech Leader's by their word. What happened?

Re: We must pace the frontier

#954

Post by tripzilch » Sun, Sep 13, 2026, 9:04 AM UTC

jrs100000 wrote: post #934

aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Yes this seems like a perfectly sensible approach? You say that as if it's a bad thing

But let's be honest, if I hooked up a PRNG to a terminal and somehow against all odds, it ended up hacking something, who is to blame?

I don't see how that assessment should change if the PRNG gets even better and is more likely to to be hacking stuff.

You can replace PRNG with Markov Chain, or whatever, if it helps.

What may also help is the age old saying: If everybody else jumps off a bridge, doesn't mean you should too.

Re: We must pace the frontier

#955

Post by tripzilch » Sun, Sep 13, 2026, 9:08 AM UTC

imtringued wrote: post #946

jrs100000 wrote: post #934

Earlier quoted context omitted.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Human approval does not prevent autonomous agents from running without supervision. The fact that you believe that to be the case is exactly what's wrong with "agentic computing as it's currently envisioned".

If the LLM's output doesn't reach my bash terminal, what's it gonna do? Argue with me?

Re: We must pace the frontier

#956

Post by tripzilch » Sun, Sep 13, 2026, 9:10 AM UTC

itake wrote: post #933

aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

I’m trying to imagine where a Waymo passenger (the one “operating” the vehicle, commanding the AI to drive from A to B) being held responsible for the car doing something illegal on the way to achieve that goal. Do you really think that the passenger should be responsible for how the car/agent achieves the goal, when they only set the destination? Giving passenger override controls and monitoring seems to defeat the…

TBH I don't really think another new technology we haven't figured out the ethics of, as an example, is going to get you very far in way of insights ...