Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

701–710 of 946 posts

Re: We must pace the frontier

#701

Post by vouaobrasil »

stale2002 wrote: post #492
xg15 wrote: post #426

Earlier quoted context omitted.

Work like writing blogs and drawing pictures?

Yes, there are many parts of those things that some people want to be automated. I and many others are able to execute on our creative vision much fasters and more effectively thanks to automation of that work.

Personally I find it more difficult now to execute my creative vision compared to years ago with a little less tech. I think there's an optimal point to automation that is easily reached and after that the benefit from automation actually goes away. People just don't like to acknowledge that because it goes against what they're doing and that creates unpleasant cognitive dissonance.

Re: We must pace the frontier

#702

Post by le-mark »

throwaway7783 wrote: post #677
bennydog224 wrote: post #563

Earlier quoted context omitted.

I agree it’s not all altrusim. It’s a little less clear what you mean at the end though. For these companies, is your argument that “pacing the frontier” is their attempt to be nationalized and protect their investments?

Ban non US models and form a cabal, with the blessings of the government. That's what it is looking like, no?

In a world where AI advancement depended only on human ingenuity this would make sense. In that world each political power block would be in an existential race for AI supremacy. In our world compute is the limiting resource. Since the US can control who gets compute, the US already has a defacto supremacy so far as frontier model development. Now if it comes about via human (with AI assist?) ingenuity that compute is no longer a restraint, then the situation is much more dire.

Re: We must pace the frontier

#703

Post by pessimizer »

The real secret: the models have stopped improving, and they're even hitting brick walls on the harnesses.

They need to go public in order to dump the companies, which are at 1st world nation-state levels of debt. They've engineered a series of publicity stunts like the Huggingface hack and Navier-Stokes (a failed stunt) in order to convince the public to "force" them to stop improving.

They are trying to get the government to relax antitrust law so they can collude to raise prices while not improving, to keep them floating before that debt comes due. They will go public. They will sell off most of their holdings during this time, and when the debt comes due, everything crashes, and the public is left holding the bag, also benefit from the huge bailout.

They at that point will be pure Musk-style financial scammers. Then, unless LLMs are a complete long-term failure (which is unlikely, I find them useful), they will buy back into their positions at a huge discount and maybe even go private again.

Re: We must pace the frontier

#704

Post by jayd16 »

aswegs8 wrote: post #592
zozbot234 wrote: post #577

Earlier quoted context omitted.

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

Billions of years of evolution hasn't hit on it. Seems pretty unlikely.

Re: We must pace the frontier

#705

Post by throwup238 »

api wrote: post #621
zozbot234 wrote: post #577

Earlier quoted context omitted.

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

My personal belief, or at least strong hypothesis, is that this kind of recursive self improvement without real world embodied feedback of some kind is impossible. I think it violates a conservation law. RSI “foom” to superintelligence is an informatic analog to an infinite energy or perpetual motion machine. To get smarter you must try to solve real problems in the universe and then do some kind of meta learning (na…

I also strongly hold this belief largely due to Moravec’s paradox, which is kind of approaching this issue from the side.

Sort of like large language models work on top of what our language has encoded in our massive training datasets, I think biological intelligence is built on top of the parts of the brain that encode the real physical world. These parts grow/train from embodied experimentation and instinct early on in an organism’s life and only then is higher intellect built on top of it (that’s my hypothesis). Their specialization and interconnections give rise to the hardest parts of intelligence long before we’re “thinking”.

Stuff like LLMs and chess engines work because we’ve done all the job of encoding the world into tokens/positions/etc they understand, but that’s wholly inadequate for the kind of AGI we’re striving for. Next up is giving it the tools to interact with the physical world and to really experiment with some self directed “play”. Time will tell just how high the resolution of sensor and mechanical control they’ll need (hopefully not the entire human visual cortex and entire sensory input worth). I think most of the RSI will have to occur in those lower level encoders, not LLMs.

Re: We must pace the frontier

#706

Post by throwaway7783 »

afthonos wrote: post #693
throwaway7783 wrote: post #673

Earlier quoted context omitted.

When corporations are involved, it is always a good bet to err towards cynisim. From my own standpoint, Claude has started sucking really bad (incoherent, uncontrollable verbosity slow and so on) and I stopped using it. OpenAI started experimenting with ads. So the security issues not withstanding (no different than a human doing it or using it, but at scale), I would put my money on cynisim.

You are incorrectly cynical. They are telling you things are bad, and because you refuse to countenance they could be worse, you assume they must be better to comply with your mandate to disbelieve. A true cynic looks at the statements by the AI labs, assumes things are worse because the labs want to seem better than they truly are. And it takes a special kind of mass delusion to drive a sane person to think “AI is c…

The cynicism is about motivations and not that they are inherently not bad. Perhaps they are as bad as they claim. Or perhaps they're worse. All we have a couple of run of the mill breach examples and some people inside the talking about how dangerous it is. Yes, they are far more qualified than I am (or most people here), and perhaps there is a grain of truth. It is the motivation - and it is always money with corporations.

Re: We must pace the frontier

#707

Post by ls612 »

lelanthran wrote: post #657
ls612 wrote: post #550

Earlier quoted context omitted.

lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth , these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

> The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work. An attack like this doesn't really need the exploited computers to run inference, do they? If I was an LLM bent on destruction of the internet, I'd be writing programs to run on each computer, not turning each computer into an LLM itself. A few programs to break in,…

The GP was referring to the AI 'living off the land' so to speak by using its victims compute to avoid being shut down which is clearly laughable.

More to the point, so many people in this thread are making completely contrived and outlandish stories up about how AI might try go ruin our lives without any evidence backing them up in any way. It is hysterical. This is the most important technology in our lifetimes and people want to freak out and turn it into the next nuclear power, with progress banned in all but name.

Re: We must pace the frontier

#708

Post by wslh »

snaking0776 wrote: post #527
Loquebantur wrote: post #470

Earlier quoted context omitted.

Your idea about the world being inhabited by mindless seekers of short-sighted "competitive advantages" is the fantasy. Getting people to agree isn't easy, that doesn't mean it wasn't worthwhile. There often even isn't any viable alternative to reaching an agreement in social settings. When your hyper-competitive mode of thinking leads to undesirable consequences, you have to change your mindset. Reality doesn't conf…

Yes I think this level of pessimism doesn’t actually hold up to the history of the last 100 years as it relates to nuclear weapons, cloning, wars etc. In fact I think the history actually points to the fact that believing such deals were impossible was a source of much of the strife. In many cases as soon as people started to actually speak to each other about what they had been fighting about they quickly realized t…

> Agreeing to end nuclear testing saved us from the Cuban Missile Crisis.

The treaty came after the crisis, not before. And it only banned atmospheric, underwater, and space tests. The US and USSR kept testing underground on a large scale, while France and China never signed it and continued atmospheric tests for years.

Re: We must pace the frontier

#709

Post by teamonkey »

KaiserPro wrote: post #553
Aurornis wrote: post #397

Earlier quoted context omitted.

> Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. Regulate what, exactly? Limit what AI corporations can use? Other countries would love that more than anything. Basically a free gift to any competitors or any startup that quietly avoids the rules. There’s a theoretical version where all the countries in the world join hands and agree not to compete…

> world join hands and agree not to compete with each other We kinda do for other areas. Largely the world agrees not to pirate software, tv, other IP. THe issue of joblessness is well understood in china, they know that jobless people means a drop in living standards, a drop in living standards means the "contract" has failed. So it makes sense that countries like america and the constituents of the EU understand th…

> If the plebs are unemployed and unemployable, they (the billionaires) are going to be lynched.

Hence the robot dogs

Re: We must pace the frontier

#710

Post by laimewhisps »

simianwords wrote: post #325
bilsbie wrote: post #231

Right as open source models catch up to frontier closed source ones for 1/10th (or less) of the cost suddenly it's time to hit the brakes! Funny how that works. https://x.com/BasedTorba/status/2098795920720547916

I'm willing to at least listen to conspiracy theories but this makes no sense.. OpenAI and Anthropic have more to lose by pausing than these rando Chinese companies.

They would only be pausing for public models. There's no world that exists where the US government, Israel.. don't continue to push as hard and as fast as they can. Amodei, Musk, Altman... will happily sell them that, I'm sure they already do. This is just taking the models away from the public and trying to crush all competition.
Post reply on HN