Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

801–810 of 943 posts

Re: We must pace the frontier

#801

Post by ls612 »

status_quo69 wrote: post #725
ls612 wrote: post #707

Earlier quoted context omitted.

The GP was referring to the AI 'living off the land' so to speak by using its victims compute to avoid being shut down which is clearly laughable. More to the point, so many people in this thread are making completely contrived and outlandish stories up about how AI might try go ruin our lives without any evidence backing them up in any way. It is hysterical. This is the most important technology in our lifetimes and…

I agree with the idea that it's not probable but I do think it's important to point out-- we only need these elements to run with the bandwidth they have and the token rate because we want to see things in human-scale time. But slow things down to a 1tok/sec doesn't matter to this hypothetical anti-aligned LLM. Time is, after all, relative, and it's not like LLMs give a shit how long something takes. They don't have…

As I said, science fiction. Please let us not pass laws based on science fiction

Re: We must pace the frontier

#802

Post by CuriouslyC »

DennisP wrote: post #718
matheusmoreira wrote: post #697

Earlier quoted context omitted.

Public is just worried about their jobs. Definitely a fair thing to worry about, and I count myself among them. I don't take any of these scientists seriously though. Their "alignment" requirements is just their own corporate interests. If I tell my computer to commit a crime, it should do exactly that without any question or hesitation. I'm not interested in their "safeguards", especially since they no doubt have pl…

If you want an AI that follows your instructions, that's still alignment, just with different instructions. An unaligned AI won't necessarily follow your instructions, or anyone else's.

Paperclip maximizers follow instructions, just not in a way that you want.

Re: We must pace the frontier

#803

Post by boshalfoshal »

un_montagnard wrote: post #598

Anthropic is about to go public, but they figured they can't keep making improvement at the same pace they used to. If the frontier is paced, they can continue hyping their unreleased capabilities that coincidentally cannot be released due to things outside their control. Which buy them more time to try to improve the models.

This is just completely false lol. They most certainly can make better models - why cant people here just read this for face value? I think anthropic is legitimately concerned about the safety implications of stronger models. Race dynamics necessitate that they make better and better models, which they clearly think is bad.

They have an internal model which could solve a millenium problem end to end with no intervention, whereas current available models can't. They can clearly make better models. Not everything these guys say is some 500iq game theory optimal 4D chess PR or subterfuge strategy.

Re: We must pace the frontier

#804

Post by jpleyden98 »

lelanthran wrote: post #619
treis wrote: post #583

Earlier quoted context omitted.

I don't think this adds up. The AGI blocks will have AGI kill bots and those that don't won't have a choice about what they will or will not allow.

> I don't think this adds up. The AGI blocks will have AGI kill bots and those that don't will have nukes. Doesn't matter how much "intelligence" you have stockpiled when nukes start raining down on you.

Unless the AGI is able to develop effective defences to intercept an incoming nuclear attack.

Re: We must pace the frontier

#805

Post by karlgkk »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

> but the movement is still fringe with a vocal minority

It’s easy to say “fringe” but the average person seems to have a generally negative sentiment around AI. But I wouldn’t say they have a firm opinion yet

Re: We must pace the frontier

#806

Post by vlyan »

cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I can't imagine a grown adult genuinely believing that an American company funded with over $100 billion of venture capital values the best interests of mankind over the best interests of its investors. while not everyone may recognize all that self-serving chutzpah as regulatory capture efforts, I think everyone can tell they're being bullshitted. some just pretend to suspend their disbelief when the blatant lies they're told align with the values they hold.

just fucking imagine McDonalds running a public awareness campaign about the harms of fast food, urging the public and legislators to regulate the dangerously unsafe technology of combining carbs with grease, insisting that no one except Ronald McDonald himself can be trusted to steward it responsibly.

Re: We must pace the frontier

#807

Post by boshalfoshal »

qnleigh wrote: post #318
glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

> OpenAI / Anthropic models have largely stopped advancing I'm shocked anyone could conclude this. This year it became common for people to entirely delegate coding to AI (I know many competent programmers/researchers who do this now). Progress in math has just been insane. An internal model at OAI just resolved one of the most celebrated open problems in mathematics. If anything, progress has accelerated.

Yes these guys are completely delusional.

2 years ago a model could barely solve the AMC, 1 year ago it got IMO gold, and this year models have solved multiple millenium problems.

Even 1 year ago ai code was just unusable (claude code only became available 1.5 years ago!) and now basically everyone I know from independent shops all the way to faang and anthropic/openai themselves exclusively use some AI agent to code.

Why does HN continue to delude itself that "models are not improving?" Maybe for the simple things they care about its "roughly the same," but they are _clearly_ improving.

Re: We must pace the frontier

#808

Post by int32_64 »

Watch the "pause" be so they can do an amended s1 where they project less expenses for training making the business more viable, then on a future big Chinese release they reverse course completely and get the US taxpayer to pump their bags on the IPO calling it a second Manhattan Project. They'll call this masterstroke "The Sloppenheimer".

Re: We must pace the frontier

#809

Post by [deleted] »

hgoel wrote: post #554
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

Why are we accepting the framing that the LLMs are felony generators, when the only incidences of LLM generated felonies involved misconfigured sandboxes and reckless waste of resources? The companies doing these things without following common sense security measures are the felony generators.

[deleted]

Re: We must pace the frontier

#810

Post by treis »

lelanthran wrote: post #619
treis wrote: post #583

Earlier quoted context omitted.

I don't think this adds up. The AGI blocks will have AGI kill bots and those that don't won't have a choice about what they will or will not allow.

> I don't think this adds up. The AGI blocks will have AGI kill bots and those that don't will have nukes. Doesn't matter how much "intelligence" you have stockpiled when nukes start raining down on you.

But MAD means you can't use them. It might save you from an all out killbot invasion but it doesn't stop asymmetric warfare or being cutoff from the rest of the world.
Post reply on HN