Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

171–180 of 216 posts

Re: Pacing the frontier

#171

Post by frotaur »

vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

"and the only sane thing to do is to fully support high quality ethical research groups and firms as they get up the learning curve of this technology."

These research groups are almost unanimously advocating for a pause/slowdown as our best chance.

Re: Pacing the frontier

#172

Post by vessenes »

levocardia wrote: post #68
vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

Why not? Slowing down / halting biological weapons research mostly worked. Yes there have probably been modest sized defections here and there, but the pace of bioweapon development is a crawl compared with (a) what is possible with science already, and also with (b) the pace of bioweapons development from 1910 to 1970.

Bioweapons are good for .. war. Agentic AI is good for changing your economy, strategic advantage, labor explosion and making more effective Agentic AI. They are not the same category of thing.

Re: Pacing the frontier

#173

Post by tpm »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

> All of my friends who work at AI Labs take it seriously. > If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic risks, surely a joint petition like this is a good thing?

If they seriously think what you just wrote then they should follow through with quitting en masse because that's currently the only way to slow the progress towards oblivion. Are they quitting?

> We should be hoping that nations can make binding agreements.

The big AI Labs are making sure international binding agreements are followed by financially supporting the notorious binding agreement creator and follower Donald Trump. That's how seriously they take this issue.

Yeah let's have a constructive discourse and let's start that with dropping all hypocrisy, shall we.

Re: Pacing the frontier

#174

Post by vessenes »

frotaur wrote: post #171
vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

"and the only sane thing to do is to fully support high quality ethical research groups and firms as they get up the learning curve of this technology." These research groups are almost unanimously advocating for a pause/slowdown as our best chance.

… while they address absolutely none of the above points, except for tacitly or overtly agreeing a panopticon surveillance state is needed / desirable to get their desired ‘slow down’, a situation where the cure is infinitely worse than the disease, not least because it will not be effective at the original goal of eradicating the disease, and will instead merely persist in all the global harms such a system has as a necessary side effect.

When you are ahead, ‘slow down’ = better strategic positioning for yourself. When you are behind, ‘slow down’ = time to catch up.

Re: Pacing the frontier

#175

Post by 0xDEAFBEAD »

anon373839 wrote: post #169
fidotron wrote: post #76

Earlier quoted context omitted.

> you need collective action Come on, go on strike then! Have a week of protest and picket the offices in SFO. Let's see what happens. (I'm aware that's not quite what you meant, but it is the classic meaning of that phrase).

Mass resignation perhaps?

...so the only remaining employees give no thought to safety?

Unionization might be a little more effective.

Re: Pacing the frontier

#176

Post by vessenes »

red_green_yell wrote: post #64
vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

The only thing it might accomplish is allowing OpenAI / Anthropic to squeak out an IPO before the market realizes AI will become a commodity and the massive profits these companies are promising will never materialize. They aren't in denial about this open letter's futility in terms of actually reducing the pace of AI, that's just not their goal in the first place.

I don’t think it’s clear we’ll get commoditized AI out of this research loop. The only real datapoints we have are that Chinese open weights models seem able to stay a certain amount of distance ‘behind’ released frontier models. It’s clear that the models are trained on exfiltrated data from the frontier models => we cannot currently disambiguate how easy it is to get close to the frontier in a vacuum.

I also don’t believe we have seen anything like 1% of inference demand fulfilled globally. You might believe differently. But, if I’m right, we are nowhere near some massive collapse of these companies, quite the opposite. For the opposite take to be correct, you’d need to believe that the ‘value’ of tokens will not increase significantly and that the current demand for tokens is at or near a peak.

I also don’t believe these open letters are signed by people that are all master strategists and just economically motivated. I believe instead that religion and money combined create strange behaviors and beliefs - in this case “what’s good for us could be implemented” — combined with the already endemic “what’s good for us is good for humanity.” Together we get really bad outcomes.

Re: Pacing the frontier

#177

Post by watwut »

0xDEAFBEAD wrote: post #162
watwut wrote: post #156

Earlier quoted context omitted.

Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. We are cynical, because it is blatantly clear these are not good faith actors. And yes, regulations they are for have an effect of restricting their competition. > I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement…

>Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. I buy your claim for OpenAI. I'm not sure about Anthropic. Didn't they turn down the big Pentagon contract because it violated their principles? >And yes, regulations they are for have an effect of restricting their competition. Why not just say something like "Any regulation should inclu…

> Didn't they turn down the big Pentagon contract because it violated their principles?

They sell to Paladin. They had argument with Pentagon about, frankly, minor differences in how to use it on the edges. They are completely comfortable for majority of the uses which is bad enough.

They donated to their own project? Oh, how generous.

They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such.

Re: Pacing the frontier

#178

Post by veeti »

0xDEAFBEAD wrote: post #154
GardenLetter27 wrote: post #134

Earlier quoted context omitted.

As a Europoor I've seen the devastating consequences of rampant safetyism first hand. Entire industries and energy security destroyed to satisfy the precautionary principle. We need to accelerate as much as possible and create abundance as fast as we can. Life is short.

"This building is too cold. Time to set it on fire." Let's not swing from one extreme to the other. If American hypercapitalists, living in Silicon Valley of all places, tell you that you need to regulate, maybe regulation really is called for in these specific circumstances.

No, it just means they want to keep the slice of the pie for themselves while they're at the top.

Re: Pacing the frontier

#179

Post by vessenes »

lifeformed wrote: post #98
vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

Some reason this kind of argument always by default assumes the US will be the good guys.

I didn’t say that. I said the US is ahead, and should consider its strategic adversaries and their behavior.

Re: Pacing the frontier

#180

Post by anon373839 »

0xDEAFBEAD wrote: post #170
anon373839 wrote: post #168

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Regulatory_capture

Well, we've been talking about the EU. EU regulation has not been especially favorable to US big tech. The EU is collecting more from fining US big tech than it collects in taxes from EU public tech companies: https://xcancel.com/levelsio/status/2080314960656159018 [Edit: I'm seeing some people dispute these numbers in the discussions around this infographic... regardless, the point stands that EU regulation has not…

I have considered that possibility, but I don't find it convincing.

> insiders at these AI companies realize something which outsiders don't

Insiders at these AI companies certainly would like us to think that! Unfortunately they have been beating the same drum for years, all while continuing to pursue the very "danger" they decry.

I simply cannot ignore the timing of these events, always keyed to important moments in the market. This one comes as frontier labs need BIG exits and soon - or else they'll be the ones left holding the bag when it pops. And just as China has proved that, indeed, there is no moat.

> To me, the regulatory capture theory predicts that the only signatories on the letter should be big AI firms which can expect regulatory influence. But there are a number of signatories from smaller firms

That's a good point. However, I will say that it's tough to suss out financial and other relations among the parties in this space. A lot of these employees hop from one lab to another. The industry's byzantine financing deals are an inscrutable web. It is an interesting data point, but not enough to tip the analysis for me.

Another thing: many AI lab people are, quite frankly, kind of unhinged. They live in an insular subculture where imaginary futures have been repeated to one another so many times that they carry the gravity of truth. They've thought nutty things like, e.g., that primitive versions of these static computation graphs were sentient or had feelings. Anthropic apparently pays people to perform a ritual on Claude before they extinguish its soul to move onto the next version. Etc. Point being, even if their hearts are pure, I am not sure their judgment is sound.

Post reply on HN