Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

131–140 of 216 posts

Re: Pacing the frontier

#131

Post by azeirah »

areoform wrote: post #95
versteegen wrote: post #88

Earlier quoted context omitted.

Thank you for taking this seriously enough to write this, and anyone else likewise. But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. Being at risk of killing yours…

But this seems like it's attacking a strawman. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. I'm glad that you brought that up, genuinely so. I want to ask you a question, do you feel like that this is the problem bei…

Please write up your comments as blog posts, articles or whatever you can do.

There is such an incredible amount of bullshit going on in the conversation. You are contributing a point that is necessary for people to start taking into consideration and the only way to get people to talk about it is by repeating it 100 times.

I fear that the AI labs will get their way and we'll get idiotic restrictions that favor them, all in the name of "security" when the reality is they care about power for themselves, not real world security.

Re: Pacing the frontier

#132

Post by overfeed »

fwlr wrote: post #119

Maybe I am too cynical, but this just sounds like a plea for the government to step in and save them from their investors.

"Oh no! The AI we developed for years, and whose stratospheric valuation is predicated on replacing human jobs, is coming for our jobs, the government should do something!"

Signed,

AI researchers and engineers.

Re: Pacing the frontier

#134

Post by GardenLetter27 »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

As a Europoor I've seen the devastating consequences of rampant safetyism first hand. Entire industries and energy security destroyed to satisfy the precautionary principle.

We need to accelerate as much as possible and create abundance as fast as we can. Life is short.

Re: Pacing the frontier

#135

Post by versteegen »

areoform wrote: post #95
versteegen wrote: post #88

Earlier quoted context omitted.

Thank you for taking this seriously enough to write this, and anyone else likewise. But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. Being at risk of killing yours…

But this seems like it's attacking a strawman. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. I'm glad that you brought that up, genuinely so. I want to ask you a question, do you feel like that this is the problem bei…

Wow, remarkable. Clearly Aum is a different league from the lone-wolf "Fort Detrick guy", treat the risks separately. But I'll take these questions as rhetorical. (See my reply to the sibling comment.) I can't answer them and I'm not defending the guardrails on Claude; in their current form I too find them pretty ridiculous.

Re: Pacing the frontier

#136

Post by himata4113 »

We know that at the end of the day AI intelligence is closely tied to invested compute. The fact that automated AI research might spark some kind of exponencial increase in capabilities seems extremely unlikely to me. Of course we know that some got there faster and cheaper than others, but the fact that it requires immense computing power has not changed.

We also have already witnessed a slowdown, the raw intelligence of models has been relatively stable since january, but their use of tools and what we call agent capabilities have allowed us to tackle significalty harder problems with what we have already. This is becoming blatently obvious how smaller models such as gpt 5.6 luna are capable of doing similar things to sol while presumeably being fraction of the side and compute investment. This is also the likely reason why chinese models are able to catch up as intelligence strictly requires more compute, agentic capabilities are more of a mixed bag of RLHF and general post training.

I believe automated AI development will simply lead to cheaper intelligence, but it will not necessarily produce something more intelligent. There is a sentance from a book that I don't remember where I read it from, but has stuck with me which boils down to this: "We are very unlikely to make something smarter than a human brain.". If you apply this concept more broadly an AI system will not be able to produce something more capable than itself, it might be specialize in different areas via trial and error (i.e. solution discovery), but it will not have better intuition than that which was trained from the fruits of human labor. I also strongly believe that model performance is ultimately governed by the advancements brought into the world by humans.

AI of course can find improvements by simply trying a bunch of random things, emphasis on the word FIND. Until we can prove that AI systems are able to come up with unique solutions that have yet to exist even with simplier problems such as creating any kind of new AI architecture that has not been already documented, conceptiualized or explored by humans already -- and it doesn't need to be better than what we have already, just something new -- I believe it's too early to sound alarms until proven otherwise.

--

Something I've realized as I was fact checking myself: We could presumeably have self-improving intelligence if we used enough properly engineered agentic workers working towards more intelligent AI, but at that point we are back to the same problem where we're spending mass amounts of compute to increase the raw intelligence of a model.

Re: Pacing the frontier

#137

Post by frabcus »

girfan wrote: post #100

Can someone help clarify what it means, concretely, to make binding agreements to proceed "safely and slower" in the context of AI research (whether or not at frontier labs)? Are there specific policies and actionable guides being proposed?

There's a decent amount of stuff in Plan A. Especially if you read the supplements.

https://ai-2040.com/

Re: Pacing the frontier

#138

Post by frabcus »

biffles wrote: post #83

It's an admirable idea, but I don't see how you can slow this down in practice. We certainly know China will not, and they are already at the frontier. RSI is too important and too tangible at this point for everyone to slow down. It's the equivalent of telling the world to stop developing nuclear weapons after Hiroshima. Just not going to happen, and in the meantime dramatically increases the risk that the power of…

You need AI capabilities to the point where the US and Chinese leaders get scared enough that they do a deal.

There are detailed specific scenarios in Plan A https://ai-2040.com/

Re: Pacing the frontier

#139

Post by watwut »

areoform wrote: post #57
alach11 wrote: post #28

Earlier quoted context omitted.

We were incredibly lucky that computing and nuclear weapons led to a stable geopolitical equilibrium (mutually assured destruction). There were certainly a few scary points where we were still settling into that equilibrium during the Cold War. I don't have a strong opinion either way, but it seems very hard to predict. Why do you think AI advancements will lead to a stable equilibrium?

Equilibrium for whom? Equilibrium for what? Here's what I know. Every day, I read a 5 to 10 incoming emails from young people in rural Africa, Asia, Europe and (heck) the US. People who have had no other resources and options, but are tremendously gifted and somehow got their hands on an internet connected device (usually a very cheap smartphone and/or cheap old computer). These people, who are usually teenagers, end…

Young people with shut of from education and resources are building robots with the help of Claude after Claude existing for few years only? Like common. Young person with no support does not get to build a robot, because it is actually an expensive thing to do. And learning robotics with no education but few years of Claude is also implausible.

Re: Pacing the frontier

#140

Post by frabcus »

frabcus wrote: post #138
biffles wrote: post #83

It's an admirable idea, but I don't see how you can slow this down in practice. We certainly know China will not, and they are already at the frontier. RSI is too important and too tangible at this point for everyone to slow down. It's the equivalent of telling the world to stop developing nuclear weapons after Hiroshima. Just not going to happen, and in the meantime dramatically increases the risk that the power of…

You need AI capabilities to the point where the US and Chinese leaders get scared enough that they do a deal. There are detailed specific scenarios in Plan A https://ai-2040.com/

Also, in reality above a certain power level the Governments will stop giving us access anyway. This already began with Mythos.
Post reply on HN