Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

151–160 of 216 posts

Re: Pacing the frontier

#151

Post by 0xDEAFBEAD »

peyton wrote: post #74
Davidzheng wrote: post #70

Earlier quoted context omitted.

Not just engineers - I see people attributing it to strong priors that nothing happens but in my opinion it's more some form of psychological defense mechanism against understanding there will be an intelligence vastly superior to humans which is a deeply uncomfortable thought (to many).

Why should I trust these randos instead of Claude? I know Claude. I would hope Claude is smarter than whoever wants to make rules about my use of Claude. Replace Claude with Kimi, etc. if needed. I don’t find a petition from people I don’t know who don’t care about me more compelling than my own direct experience with these models.

I suggest you give Claude a link to the petition and ask Claude what it thinks.

(I did so previously and pasted the response here, but it got auto-flagged for AI content.)

Re: Pacing the frontier

#152

Post by [deleted] »

areoform wrote: post #95
versteegen wrote: post #88

Earlier quoted context omitted.

Thank you for taking this seriously enough to write this, and anyone else likewise. But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. Being at risk of killing yours…

But this seems like it's attacking a strawman. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. I'm glad that you brought that up, genuinely so. I want to ask you a question, do you feel like that this is the problem bei…

[deleted]

Re: Pacing the frontier

#153

Post by 0xDEAFBEAD »

red_green_yell wrote: post #62
testerteert000a wrote: post #60

Earlier quoted context omitted.

> ...I see how no-guardrails LLMs could be truly disastrous... by all accounts this is what's what because of how... > You can do CRISPR in your kitchen. .. and because of the same dynamic we see occurring in cybersec. If you're good enough at cybersec, but not good-good at specific skillsets like offensive, then LLMs and a comp sci degree makes you quite capable in the ways that count. This is a fact, full stop. If…

If the accuracy of your biology claim is "hard to say" then why would pushback be self-evidently myopic? You and others are making a claim about LLMs enabling biological dangers. Some people disagree with you, pointing out that the incremental knowledge an LLM provides over say google or a library card may not be that much. What seems myopic to me is un-skeptically accepting "AI dangerous, must regulate" messaging fr…

>frontier labs that have trillions of dollars of vested interests in this message.

OP suggests the opposite: "each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration."

Re: Pacing the frontier

#154

Post by 0xDEAFBEAD »

GardenLetter27 wrote: post #134
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

As a Europoor I've seen the devastating consequences of rampant safetyism first hand. Entire industries and energy security destroyed to satisfy the precautionary principle. We need to accelerate as much as possible and create abundance as fast as we can. Life is short.

"This building is too cold. Time to set it on fire."

Let's not swing from one extreme to the other.

If American hypercapitalists, living in Silicon Valley of all places, tell you that you need to regulate, maybe regulation really is called for in these specific circumstances.

Re: Pacing the frontier

#155

Post by 0xDEAFBEAD »

fwlr wrote: post #119

Maybe I am too cynical, but this just sounds like a plea for the government to step in and save them from their investors.

Investors want to see a return on their investment ASAP. The Economist recently published an article:

"AI revenues are growing fast, but not fast enough -- The returns on trillions of dollars of spending are deeply uncertain"

https://www.economist.com/finance-and-economics/2026/07/28/a...

As OP states: "each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration."

Let's not overwrite human civilization for the sake of the shareholders.

Re: Pacing the frontier

#156

Post by watwut »

0xDEAFBEAD wrote: post #141
NitpickLawyer wrote: post #118

Earlier quoted context omitted.

> We should want the AI Labs to make binding agreements to proceed safely and slower That's a red herring, IMO. Once the cat is out of the bag, there's no "slowing down", only "slowing up". What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capt…

>What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capture, which is a proven net downside for everyone else. This sort of cynicism feels un-falsifiable. If companies advocate against regulation, the cynics will say: "These corporations want ca…

Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. We are cynical, because it is blatantly clear these are not good faith actors. And yes, regulations they are for have an effect of restricting their competition.

> I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement.

I would like to see a token effort of good faith engagement from AI companies. For a change.

Re: Pacing the frontier

#157

Post by soundworlds »

0xDEAFBEAD wrote: post #147
soundworlds wrote: post #126

Earlier quoted context omitted.

It really comes down to this: Sam and Dario are leading companies that who made the reckless decision to train on everyone's IP without proper process or consent. It has been proven they trained on pirated content. There is no question about this. Then, as soon as there are open models who threaten their businesses, they cry "distillation and IP theft". It's the last 3 years of pure hypocrisy and fear-baiting for inv…

You're missing the point of the OP: "there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems... each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration" >To be honest, I think people would now prefer the risk of misused open models, when the alternative is giving these guys more power. Then…

I suppose I have personally lost faith in centralized institutions that would be implementing said regulation, partly because they let OpenAI and Anthropic (along with Suno, Audio, etc etc) get this far uninhibited. Centralized regulation simply can't keep up.

My hopes now lay with a decentralized approach of open models, trusting people to adapt them to local needs and security, rather than a nationwide paternalism.

Re: Pacing the frontier

#158

Post by [deleted] »

atleastoptimal wrote: post #51
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

So many otherwise smart engineers, overrepresented on HN, have this weird mental block where they fail to take AI progress seriously. I think it's a fear that their status-defining skill could be commodified, so they adopt a mental framework that aims to dismantle the seeming legitimacy of any genuine leap in AI progress as some sort of mirage

[deleted]

Re: Pacing the frontier

#159

Post by 0xDEAFBEAD »

fidotron wrote: post #15

Anyone serious about this should participate in pacing the frontier by quitting their job so as to not potentially contribute to the problematic progress. Be the change you want to see! Curious how that isn't what happens.

FWIW, here's an account from a guy who quit his job at Google Deepmind recently:

https://turntrout.com/why-i-left-google-deepmind

Re: Pacing the frontier

#160

Post by 0xDEAFBEAD »

soundworlds wrote: post #157
0xDEAFBEAD wrote: post #147

Earlier quoted context omitted.

You're missing the point of the OP: "there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems... each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration" >To be honest, I think people would now prefer the risk of misused open models, when the alternative is giving these guys more power. Then…

I suppose I have personally lost faith in centralized institutions that would be implementing said regulation, partly because they let OpenAI and Anthropic (along with Suno, Audio, etc etc) get this far uninhibited. Centralized regulation simply can't keep up. My hopes now lay with a decentralized approach of open models, trusting people to adapt them to local needs and security, rather than a nationwide paternalism.

I don't exactly see how "trusting people to adapt [open models] to local needs and security" solves the problem of "automating AI research" where "capability development rapidly accelerates beyond our ability to understand or control the resulting systems".

If regulation has been too slow, that's not really an argument for making it even slower?

Post reply on HN