Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

941–950 of 965 posts

Re: We must pace the frontier

#941

Post by hakimg »

sigmarule wrote: post #901
RobertDeNiro wrote: post #866

Earlier quoted context omitted.

Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?

If this is going to factor into my reasoning about the topic at hand, it will surely be a secondary or tertiary consideration at best. Same goes for much of the list in the root comment. We are discussing the possibility of severe adverse societal/world/human impacts from AI, why should open vs closed source/weights, copyright violations, anti-competitive moves, corporate hypocrisy, etc be so heavily weighted in the…

This is exactly my view and it's frustrating that a sizeable group of people refuse to accept there's any risk with AI.

We're literally trying to build something that's smarter than all humans and with the capacity to do extraordinary work while most people think there's no risk in open sourcing this capability with no guardrails (model ablation means no guardrails and as far as I'm aware can't be stopped).

Has our distrust in western companies really gotten to the point they'd rather unfiltered Astra level capabilities in the hands of every hostile government and terrorist organisation than potentially do any form of regulation in case it plays into the AI labs hands.

Re: We must pace the frontier

#942

Post by brabel »

jimbokun wrote: post #844
FusionX wrote: post #782

Earlier quoted context omitted.

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

Anything related to AI is extremely unpopular right now with the general public.

Bs.

Musicians are almost all using AI, even if they still prefer to not fully generate tracks [1]. You listen to AI assisted music already even if you don’t realize it.

Most people are using AI and mostly they respond positively to it [2].

What people don’t like is the slop that has been flooding YouTube and similar content creators platforms. That was inevitable since the AI tools became so accessible any average joe could spam the internet with their sloppy content. But that doesn’t mean the pros are not using it to make great content, just like the best programmers are using it to create great software.

1: https://www.landr.com/ai

2: https://www.rev.com/blog/chatbot-statistics

Re: We must pace the frontier

#943

Post by dartharva »

barrrrald wrote: post #733
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

That claim looks to be coming from the opposite of Occam's Razor. Them doing typical SV dirty plays is a lot more easier explanation than the straws you are grasping at.

Re: We must pace the frontier

#944

Post by eli_gottlieb »

yoyoyoyop wrote: post #786
FusionX wrote: post #782

Earlier quoted context omitted.

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

I don’t think the anti-AI sentiment is as fringe as you think. At least not outside the tech world it isn’t..

Most people don't really have clear or strong opinions about AI other than that they don't like it :-p.

Re: We must pace the frontier

#945

Post by kwamenum86 »

Buckle up.

I’ve spent a lot of time studying opinions coming out of the frontier labs, and talking directly to employees. This won’t necessarily result in Skynet. But life as we know it will be a thing of the past within two years unless there’s a huge course correction. I think we’re cooked for the following reasons

- Everything connected to the Internet will eventually be hacked. For decades, we’ve slowly migrated all of our critical systems to devices connected to one giant network. Misaligned agents have mounted increasingly sophisticated explorations/attacks by infiltrating systems that are completely unprepared to deal with this kind of threat.

- We have a poor understanding of the impact of LLMs, which is inhibiting our ability to protect ourselves, create the right guardrails, and stand up policies quickly enough. We’re still wasting time debating. We’re moving quickly toward a punctuated equilibrium moment and won’t have time to waste

- We’re at the beginning of a runaway evolutionary process. At some point soon, someone is going to invent recursive self improvement. Any policy will at best serve as a mild disincentive. It’s happening.

- In the very best case, someone will exploit the confusion and chaos of this situation for their own gain. There are many examples littered throughout history, including the last several years of course. The utopian vision is not happening. Like all other resources, we’ll compete. We haven’t devised a good approach to orderly distribution of resources for large groups of people.

- No one has demonstrated that they understand how to build AI safely. Not a single entity. These companies are not even being scaled in a way that’s conducive to building AI safety. Hyperscale orgs struggle with scaling basic consumer apps. How on earth are they going to scale AI? Based on my firsthand observations of their internal workings, I have zero confidence they can.

Re: We must pace the frontier

#946

Post by imtringued »

jrs100000 wrote: post #934
aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

Yes, your technically right, it is a business solution, its just not a valid one for agentic computing as its currently envisioned. An agent would have to be fully sandboxed to an internal environment, or human would have to review and approve each action it tried to take.

Human approval does not prevent autonomous agents from running without supervision.

The fact that you believe that to be the case is exactly what's wrong with "agentic computing as it's currently envisioned".

Re: We must pace the frontier

#947

Post by imtringued »

itake wrote: post #933
aarondong wrote: post #931

Earlier quoted context omitted.

This is a business solution, because it means there are legal costs for not having adequate observability and monitoring mechanisms. Every tool call is interfacing with a harness. But overreach of policy and overregulation would be stifling, so there has to be a threshold to the type of incident investigated, civil or criminal.

I’m trying to imagine where a Waymo passenger (the one “operating” the vehicle, commanding the AI to drive from A to B) being held responsible for the car doing something illegal on the way to achieve that goal. Do you really think that the passenger should be responsible for how the car/agent achieves the goal, when they only set the destination? Giving passenger override controls and monitoring seems to defeat the…

When you run an agent on your computer, that's equivalent to installing self driving software into your manually driven car, i.e. something like comma.ai.

Comma.ai might be fully safe to operate autonomously on a mining site or a corporate parking lot, but maybe not in city traffic.

If you use it in problematic scenarios, that is on you.

Re: We must pace the frontier

#948

Post by pants2 »

sejje wrote: post #793
pants2 wrote: post #756

Earlier quoted context omitted.

We don't know exactly what the limits of AI improvement on our current infrastructure are, though. If the human brain is 20W, and a datacenter is 1GW, then maybe that datacenter can be 50 million times smarter than a human. If that's not already a risk to humankind I don't know what is.

Is the datacenter gonna grow legs? Give me one datacenter, I'll keep it under control all by myself. Now, if some dumbasses start hooking up their data centers to...I don't know, like--robot factories? That sounds like a risk to humankind.

All it takes for the AI to grow legs is to acquire money (should be no problem for an ASI) and pay humans to do what it wants in the real world.

Re: We must pace the frontier

#949

Post by Davidzheng »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

Doesn't it just incentive the business to be outcompeted by one that uses ai more--unless the consumer is choosing based on morals

Re: We must pace the frontier

#950

Post by Davidzheng »

augment_me wrote: post #638
Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".
Post reply on HN