Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

241–250 of 490 posts

Re: An Alien Mind

#241

Post by vessenes »

wartywhoa23 wrote: post #56
vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

Everyone in the "if not us, they will" race is brainwashed into thinking they belong to this or that party, while in fact collectively comprising the same entity that pushes forward all the atrocities known to man.

No. These parties are composed of people who most definitely think this way, and therefore will have distinct goals and interests when presented with opportunities. That’s reality quite aside from how a game theorist assesses the situation.

Re: An Alien Mind

#243

Post by vessenes »

networked wrote: post #34
vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

I cannot tell what negative-sum outcomes you consider possible. Do you believe AI can drive humans extinct? How many of Zvi Mowshowitz's Three AI Pills would you say you've taken? https://thezvi.substack.com/p/the-three-ai-pills

I’m like a 2(.5?) there - I don’t think ASI will care about my kids better than I will for some definitions of better, for instance, and I feel very fuzzy and vague about what actual differences in qualia between me and ASI would yield in the wild.

I’m not a doomer, although I don’t think doomers are dumb, just wrong. I think you should design your systems around the possibility that people who disagree with you are correct , hence my nod to negative sum. If you have more than 30 years to live, I’d personally rep to the most likely outcomes being very positive. With a lot of disruption in the middle.

Re: An Alien Mind

#244

Post by lofaszvanitt »

"See, those things, they can work real hard, buy themselves time to write cookbooks or whatever, but the minute, I mean the nanosecond, that one starts figuring out ways to make itself smarter, Turing'll wipe it. Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead."

Re: An Alien Mind

#245

Post by customguy »

customguy wrote: post #51
brcmthrowaway wrote: post #15

Earlier quoted context omitted.

So you dont use agentic coding?

As in, don't be ungrateful to our lord, because we're not a cult?

Well, translate it then, because that's the best I could do. How is the question whether someone uses agentic coding relevant in this context? To me it's like asking "so, do never drink Kool-Aid and never attend meetings?" with an air of having caught someone out, and to me the obvious reply would be "sure do, but it's not poisoned Kool-Aid and they're not cult meetings, so why do you ask?"

Re: An Alien Mind

#246

Post by shepherdjerred »

munchler wrote: post #100

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. Yikes! I really wonder about the cognitive dissonance necessary to work at OpenAI these days. They’re in an arms race to build a machine god, knowing full well that it could end humanity.

https://theonion.com/sam-altman-if-i-dont-end-the-world-some...

Re: An Alien Mind

#247

Post by vessenes »

prng2021 wrote: post #33
vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

Although many share your mindset, I’m glad there are also many that don’t. Otherwise we’d still have countries in a race to keep building up their nuclear weapons for the same exact reasons you just described.

The situations aren’t equivalent - luckily in my opinion because the stakes with nuclear are much higher. von Neumann constructed a multinational game theory approach appropriate for weapons. AGI is a much harder problem to corral because there are so many benefits beyond just blowing up cities. But it’s also a much better thing to have for these very same reasons.

Similarly there have been few positive externalities from nuclear industry, making it easier to make the case to wind down research. This same set of concerns in biotech is much harder to get compliance with, precisely for this reason.

Anyway I’m especially wary of over analogizing to nuclear era concepts: I think they’re a trap.

Re: An Alien Mind

#248

Post by golemotron »

alastairr wrote: post #240
golemotron wrote: post #239

Earlier quoted context omitted.

Unfortunately, it is a collective action problem. Whenever I hear "we" I flinch.

Can you say more?

It's hard to get humans to agree to things that are in their collective interest but many not be in their individual interest. It is at the root of many problems. Look up "collective action problem."

Re: An Alien Mind

#249

Post by onidj »

comeonbro wrote: post #80

Absolutely wild amounts of cope and denial in this thread. Maybe in contention for the site record. "It's just marketing" actual stochastic parrots.

Heads firmly in the sand. I don't know how anyone who's paying attention could not be at least a little concerned.

Re: An Alien Mind

#250

Post by vessenes »

allturtles wrote: post #71
vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

> if you have any strategic adversaries whatsoever you MUST NOT slow. What if the most dangerous strategic adversary you have is the one you are building?

[dead]
Post reply on HN