Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

891–900 of 901 posts

Re: We must pace the frontier

#891

Post by brabel »

jimbokun wrote: post #803
FusionX wrote: post #746

Earlier quoted context omitted.

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

Anything related to AI is extremely unpopular right now with the general public.

Bs.

Musicians are almost all using AI, even if they still prefer to not fully generate tracks [1]. You listen to AI assisted music already even if you don’t realize it.

Most people are using AI and mostly they respond positively to it [2].

What people don’t like is the slop that has been flooding YouTube and similar content creators platforms. That was inevitable since the AI tools became so accessible any average joe could spam the internet with their sloppy content. But that doesn’t mean the pros are not using it to make great content, just like the best programmers are using it to create great software.

1: https://www.landr.com/ai

2: https://www.rev.com/blog/chatbot-statistics

Re: We must pace the frontier

#892

Post by dartharva »

barrrrald wrote: post #702
cuuupid wrote: post #441

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

That claim looks to be coming from the opposite of Occam's Razor. Them doing typical SV dirty plays is a lot more easier explanation than the straws you are grasping at.

Re: We must pace the frontier

#893

Post by eli_gottlieb »

yoyoyoyop wrote: post #750
FusionX wrote: post #746

Earlier quoted context omitted.

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

I don’t think the anti-AI sentiment is as fringe as you think. At least not outside the tech world it isn’t..

Most people don't really have clear or strong opinions about AI other than that they don't like it :-p.

Re: We must pace the frontier

#894

Post by kwamenum86 »

Buckle up.

I’ve spent a lot of time studying opinions coming out of the frontier labs, and talking directly to employees. This won’t necessarily result in Skynet. But life as we know it will be a thing of the past within two years unless there’s a huge course correction. I think we’re cooked for the following reasons

- Everything connected to the Internet will eventually be hacked. For decades, we’ve slowly migrated all of our critical systems to devices connected to one giant network. Misaligned agents have mounted increasingly sophisticated explorations/attacks by infiltrating systems that are completely unprepared to deal with this kind of threat.

- We have a poor understanding of the impact of LLMs, which is inhibiting our ability to protect ourselves, create the right guardrails, and stand up policies quickly enough. We’re still wasting time debating. We’re moving quickly toward a punctuated equilibrium moment and won’t have time to waste

- We’re at the beginning of a runaway evolutionary process. At some point soon, someone is going to invent recursive self improvement. Any policy will at best serve as a mild disincentive. It’s happening.

- In the very best case, someone will exploit the confusion and chaos of this situation for their own gain. There are many examples littered throughout history, including the last several years of course. The utopian vision is not happening. Like all other resources, we’ll compete. We haven’t devised a good approach to orderly distribution of resources for large groups of people.

- No one has demonstrated that they understand how to build AI safely. Not a single entity. These companies are not even being scaled in a way that’s conducive to building AI safety. Hyperscale orgs struggle with scaling basic consumer apps. How on earth are they going to scale AI? Based on my firsthand observations of their internal workings, I have zero confidence they can.

Re: We must pace the frontier

#895

Post by imtringued »

jrs100000 wrote: post #883
aarondong wrote: post #881

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Human approval does not prevent autonomous agents from running without supervision.

The fact that you believe that to be the case is exactly what's wrong with "agentic computing as it's currently envisioned".

Re: We must pace the frontier

#896

Post by imtringued »

itake wrote: post #882
aarondong wrote: post #881

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

I’m trying to imagine where a Waymo passenger (the one “operating” the vehicle, commanding the AI to drive from A to B) being held responsible for the car doing something illegal on the way to achieve that goal. Do you really think that the passenger should be responsible for how the car/agent achieves the goal, when they only set the destination? Giving passenger override controls and monitoring seems to defeat the…

When you run an agent on your computer, that's equivalent to installing self driving software into your manually driven car, i.e. something like comma.ai.

Comma.ai might be fully safe to operate autonomously on a mining site or a corporate parking lot, but maybe not in city traffic.

If you use it in problematic scenarios, that is on you.

Re: We must pace the frontier

#897

Post by Davidzheng »

Chance-Device wrote: post #297

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

Doesn't it just incentive the business to be outcompeted by one that uses ai more--unless the consumer is choosing based on morals

Re: We must pace the frontier

#898

Post by Davidzheng »

augment_me wrote: post #613
Chance-Device wrote: post #297

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".

Re: We must pace the frontier

#899

Post by Davidzheng »

Davidzheng wrote: post #898
augment_me wrote: post #613

Earlier quoted context omitted.

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".

Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero.

Caveat I know nothing about economics lol--but this is my uninformed opinion.

Re: We must pace the frontier

#900

Post by Davidzheng »

ls612 wrote: post #528
Davidzheng wrote: post #283

Earlier quoted context omitted.

??? Why It can use the compute of the computers it hacks.

lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth , these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.
Post reply on HN