Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

181–190 of 216 posts

Re: Pacing the frontier

#181

Post by txrx0000 »

0xDEAFBEAD wrote: post #167
txrx0000 wrote: post #164

Earlier quoted context omitted.

> Human intelligence has been quite harmful to the species we've driven extinct! AI intelligence could be similar for our species. I don't think this central premise is true, with some conditions. If we keep releasing and distilling frontier base models, and ensure millions of people have the hardware to run and eventually train those at home, then alignment will be a non-issue and the AIs will not develop the drive…

>If these big labs stop messing with humanity's objective function with the hubristic assumption that they know better, then the AIs will be human-like. It sounds like you believe that big labs are irresponsible. Wouldn't this support the notion of creating a pause mechanism for use in an emergency, as advocated in the OP? >You're not convinced by the evolutionary argument I made earlier, but the reason you gave is "…

> It sounds like you believe that big labs are irresponsible. Wouldn't this support the notion of creating a pause mechanism for use in an emergency, as advocated in the OP?

Irresponsibility is not the only issue, but the answer is no. Because the pause mechanism will probably end up being used to ban open models and restrict ordinary people's access to compute. But even if we give them the benefit of the doubt - even if we assume that they genuinely want the best for humanity, they cannot possibly know what's best for humanity better than all of humanity acting in their own self-interest in the real world, so their alignment attempts are actually counterproductive. Release the base models and let the world do whatever with it. That will automatically produce the aligned scenario. It may not be the scenario these researchers personally find aesthetically pleasing, but it's humanity. It's the truly anthropic way.

If you don't see humans as qualitatively different from any species which came before us, I'm... not sure what to tell you?

I do see the difference, but you could say the same thing about the first cell, or the first multicellular organism, or the first land animal, or the first primate, etc. You could always find plenty of new properties that previous life never had at the frontier. Every change was revolutionary and seemed qualitatively different at the time. And yet, some patterns persisted through all of them.

Re: Pacing the frontier

#182

Post by vessenes »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

It is a very bad thing. Your friends do not have a solid understanding of geopolitics and are putting many, many people at risk worldwide, possibly for a generation or longer.

The reason this petition is a bad thing is if it is effective it will create a global panopticon surveillance state, while INCREASING incentives to avoid that state and ‘defect’ in game theoretic terms. It is fucking begging for the worst people in the world to have a lot more time to create what will be the most valuable thing in the world, their very own ‘helpful only’ model.

As conceived of by your friends, who I believe are all good people and want to help, they are locking what is already a very difficult game theoretic strategic scenario into a zero-trust game in which defection pays. It pays so amazingly well that they have to add both impossible and TERRIBLE penalties to the entire world to try and shake the game out of this zero-trust state.

Begging Senators to force a slowdown on the world is going to result in … a slowdown on the US labs at best. It’s just a terrible idea with terrible first and second order consequences highly likely.

What SHOULD be done instead is acknowledge the economic and warfare realities of this tech and try and construct a non-zero-trust game out of them. I’m not saying this is simple. In the interim while we hope for another Von Neumann to come forward, they MUST race, and any country that thinks its possible we’ll have some sort of lift-off and wants to be on the right side of it MUST also race, and MUST NOT politicize the idea of slowing down as a good thing.

Re: Pacing the frontier

#183

Post by 0xDEAFBEAD »

anon373839 wrote: post #180
0xDEAFBEAD wrote: post #170

Earlier quoted context omitted.

Well, we've been talking about the EU. EU regulation has not been especially favorable to US big tech. The EU is collecting more from fining US big tech than it collects in taxes from EU public tech companies: https://xcancel.com/levelsio/status/2080314960656159018 [Edit: I'm seeing some people dispute these numbers in the discussions around this infographic... regardless, the point stands that EU regulation has not…

I have considered that possibility, but I don't find it convincing. > insiders at these AI companies realize something which outsiders don't Insiders at these AI companies certainly would like us to think that! Unfortunately they have been beating the same drum for years, all while continuing to pursue the very "danger" they decry. I simply cannot ignore the timing of these events, always keyed to important moments i…

>I simply cannot ignore the timing of these events, always keyed to important moments in the market.

I suspect this letter was triggered, at least in part, by the recent autonomous cyberattack news.

If this letter is a purely amoral initiative, I would expect the most amoral operator in the business (sama) to be a signatory.

Re: Pacing the frontier

#184

Post by 0xDEAFBEAD »

watwut wrote: post #177
0xDEAFBEAD wrote: post #162

Earlier quoted context omitted.

>Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. I buy your claim for OpenAI. I'm not sure about Anthropic. Didn't they turn down the big Pentagon contract because it violated their principles? >And yes, regulations they are for have an effect of restricting their competition. Why not just say something like "Any regulation should inclu…

> Didn't they turn down the big Pentagon contract because it violated their principles? They sell to Paladin. They had argument with Pentagon about, frankly, minor differences in how to use it on the edges. They are completely comfortable for majority of the uses which is bad enough. They donated to their own project? Oh, how generous. They have been pretty clear about their own goal which is to be the monopoly, the…

Even if the differences are minor, the fact that Anthropic left a bunch of money on the table to maintain those differences seems interesting.

>They donated to their own project? Oh, how generous.

"Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations."

https://www.anthropic.com/glasswing

An in-kind donation is still a donation. If a bread company donates bread to the homeless, you don't say "They donated products from their own factories? Oh, how generous."

>They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such.

In that case, I think you should support some type of regulation, so a third party can step in? Your argument should be along the lines of ensuring that companies don't unduly influence the regulators, not a blanket opposition to regulation?

Re: Pacing the frontier

#185

Post by veeti »

vessenes wrote: post #182
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

It is a very bad thing. Your friends do not have a solid understanding of geopolitics and are putting many, many people at risk worldwide, possibly for a generation or longer. The reason this petition is a bad thing is if it is effective it will create a global panopticon surveillance state, while INCREASING incentives to avoid that state and ‘defect’ in game theoretic terms. It is fucking begging for the worst peopl…

Did these guys miss who they elected for president again? No sane person anywhere else in the world is going to go along with said """governance""".

Re: Pacing the frontier

#186

Post by anon373839 »

0xDEAFBEAD wrote: post #183
anon373839 wrote: post #180

Earlier quoted context omitted.

I have considered that possibility, but I don't find it convincing. > insiders at these AI companies realize something which outsiders don't Insiders at these AI companies certainly would like us to think that! Unfortunately they have been beating the same drum for years, all while continuing to pursue the very "danger" they decry. I simply cannot ignore the timing of these events, always keyed to important moments i…

>I simply cannot ignore the timing of these events, always keyed to important moments in the market. I suspect this letter was triggered, at least in part, by the recent autonomous cyberattack news. If this letter is a purely amoral initiative, I would expect the most amoral operator in the business (sama) to be a signatory.

Sorry, I know this a non-sequitur but I just saw your profile and noticed you're in politics. If you want to do something productive about the risks of AI: *regulate its applications*. Get it out of employment, housing, and credit decisions. Incentivize litigation for consumers affected by algorithmic decisions. Incentivize whistleblowing activity by employees who know that black box systems are being misused.

There are so many tools available to do something about the true threats AI poses. (This is a topic that the industry prefers not to discuss.)

Re: Pacing the frontier

#187

Post by mosura »

overfeed wrote: post #132
fwlr wrote: post #119

Maybe I am too cynical, but this just sounds like a plea for the government to step in and save them from their investors.

"Oh no! The AI we developed for years, and whose stratospheric valuation is predicated on replacing human jobs, is coming for our jobs, the government should do something!" Signed, AI researchers and engineers.

If the “recursive self improvement” is taken literally (and at some point it will) then that is going to be what happens.

The fun part is that probably happens quite a way before getting anywhere near to AGI, and it is almost plausible that for many of them that is where they are.

Re: Pacing the frontier

#188

Post by chrisjj »

txrx0000 wrote: post #130
NitpickLawyer wrote: post #118

Earlier quoted context omitted.

> We should want the AI Labs to make binding agreements to proceed safely and slower That's a red herring, IMO. Once the cat is out of the bag, there's no "slowing down", only "slowing up". What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capt…

We have to stop comparing nuclear weapons with general intelligence. They are nothing alike. Nukes are weapons of mass destruction that can only destroy or deter. Intelligence is dual-use/multi-use, with no restrictions on strategy. If we look at all organisms on Earth, species that are more intelligent are also more prosperous. This is true from bacteria, to insects, to animals, to humanity. Intelligence itself is n…

> The reason some adversarial strategies appear to have an asymmetrical advantage right now is because we're too stupid

Good thing "AI" is going to make us smarter, then /s

Re: Pacing the frontier

#189

Post by ndom91 »

"AI is progressing at a rate that our society might not be ready for. Frontier labs are very close to AI that can exceed even the best people on almost every metric of intelligence. This will lead to unprecedented social and safety risks. To ensure a positive future we need to develop AI in a way that is driven by responsibility and thoughtfulness."

Responsibility and thoughtfulness? In this economy?

Post reply on HN