Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

401–410 of 490 posts

Re: An Alien Mind

#401

Post by wizardforhire »

_dark_matter_ wrote: post #314
nullbio wrote: post #301

Earlier quoted context omitted.

My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see the…

This is so, painfully, childish. Humans have known for thousands of years that there is no objective truth. Every falsity can be bent and twisted until it is more true than the sun itself.

Lol thousands?!?! Try roughly 250 and even then Hume only opened the door for us…

Re: An Alien Mind

#403

Post by a2ff6eeb0 »

tim333 wrote: post #394
TheOtherHobbes wrote: post #139

Earlier quoted context omitted.

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Because humans have the "everybody wants to rule the world" instinct built into us by much evolutionary selection. AI would come exist differently.

Alignment, I think, is mostly about making sure that the companies selling AI keep making money. We just have to hope that aligns with keeping people happy.

Re: An Alien Mind

#404

Post by kypro »

indigo945 wrote: post #395
kypro wrote: post #191

Earlier quoted context omitted.

It's actually worse than this because it assumes alignment as a concept even makes sense. For example: If the the Chinese government asks their ASI to create a bioweapon against the West, should it? No, presumably not – an aligned AI would be one which disobeys the Chinese government even if they created it. Okay, so what if the US government asks their ASI to help it in one of their wars instead? Would an aligned AI…

I think it is worth noting that the article addresses this.

It touches on it, but it doesn't address it.

They talk about "value alignment" but fail to define what those values are – is it aligned to the values of the US government, or are they suggesting they want to build an AI with it's own values so it can decide for itself when and how it will intervene in wars and other human affairs? And again, is an AI which disempowers humanity in this way aligned? Many would say no, although as I argue, disempowering humanity is probably better than the alternative if we can assume it's roughly aligned with our interests (which we obviously can't because it's super intelligent, but that's another issue).

Fundamentally the problem here is that humans don't have an aligned set of values you can align an AI to.

The moment you start defining what alignment actually is in practise you simply must accept it will be unaligned with the values of others. There is no getting around this and hand waving around the issue isn't good enough.

They should tell us explicitly what they're trying to build.

Re: An Alien Mind

#405

Post by pmautio »

I think “Alien Mind” gets the framing wrong at quite a fundamental level, and a bad framing makes us seek the wrong remedies. Alien suggests something independently constituted, whose purposes we then have to discover and constrain. But these “minds” are a mass externalization of human knowledge, language and interrelations, so “alienated mind” would be better. And what is alienated can, we should hope, be reappropriated.

If nothing else, the Hugging Face incident makes the boundaries of that mind rather less obvious. What happened suggests that the intelligence belongs partly to the relations between model, memory, tools and environment.

Alignment talk of “coevolution” still feels too tidy as well, since it suggests two things, humans and AI, adapting to one another. That framing eagerly fixes the relata as already known before they enter into a relation. Here we’re building a single increasingly entangled cognitive environment, with rather uncertain borders between its parts, and should be testing where those borders actually hold, shift around, or disappear.

Re: An Alien Mind

#406

Post by AnodicElegy »

"OpenAI’s primary bet here has been chain-of-thought monitoring. It is based on an appealingly scalable idea: a lot of the model’s capability comes from a verbalized reasoning process (chain-of-thought). If we scale optimization on the outcomes of that process, but do not supervise the process itself, that chain-of-thought has no direct incentive in training to hide any misaligned ideas or objectives. This does not mean the model will learn to externalize misaligned tendencies that don’t rely on using the chain-of-thought; however, it can allow us to monitor exactly the capability increase from reasoning."

Yes, and it would greatly aid alignment if the user could monitor the chain-of-thought! The open models, including very powerful ones like Kimi K3, are delivering this. The fact that OpenAI and Anthropic are not clearly indicates that a commercial consideration (avoiding distillation) takes precedence over alignment -- regardless of how much they bloviate about the latter.

Re: An Alien Mind

#407

Post by kypro »

seydor wrote: post #354
kypro wrote: post #201

Earlier quoted context omitted.

No offence, but you clearly haven't studied this and are making some wrong assumptions here. > Neural networks are approximators being fed human intellect. They're not "approximators", that's a far too simplistic way to think of them. Neural nets create models and deep layers of abstraction around the data we feed them in the same way your brain creates layers of abstractions to reason about the world. AIs can use th…

> Neural nets create models and deep layers of abstraction around We don't have any proof of that, but the approximator thing is proven. The rest is just marketing speak. Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically

> We don't have any proof of that, but the approximator thing is proven.

Not in the way suggested... It's not approximating human intellect. They approximate the target function, and the target function frontier labs are trying to approximate is ultimately a super intelligence...

> Synthetic data derive from other linguistic data. Whatever intelligence is in there, it is expanded horizontally, not vertically

I'll assume we're talking purely about language models for a moment, but if you assume that everything can be represented linguistically, then in theory there is no upper-bound on what can be learnt with synthetic data.

Re: An Alien Mind

#408

Post by _bobm »

pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

I think it is simpler, but bannin open weight is a collateral worth scoring nonetheless.

AI fails to deliver on the multi trilion usd promises:

Gov/venture says: oh, wow, what will we do with all these datacenters, we need money back!

OpenAI/ Anthropic: let's monitor the citizenry. They may be plotting nefarious schemes using AI models.

Gov: great idea. Whew.

Investors: whew!

Tax payers: paying to be in prison.

So not so much as defense against foreign actors, but failure of the self-tooted AGI goal + gov being gov.

Re: An Alien Mind

#409

Post by indigo945 »

kypro wrote: post #404
indigo945 wrote: post #395

Earlier quoted context omitted.

I think it is worth noting that the article addresses this.

It touches on it, but it doesn't address it. They talk about "value alignment" but fail to define what those values are – is it aligned to the values of the US government, or are they suggesting they want to build an AI with it's own values so it can decide for itself when and how it will intervene in wars and other human affairs? And again, is an AI which disempowers humanity in this way aligned? Many would say no,…

No, what I mean is this part of your comment is unrelated:

    > We should stop using the word alignment. It's a BS term for a concept which  
    > simply cannot make sense if alignment is both to mean an AI which we control and  
    > an AI which will not harm us.
The article already introduces terms to distinguish between those two concepts ("goal alignment" vs "value alignment").

You are right, of course, that when we are discussing value alignment, it becomes very relevant whose values the AI is supposed to be aligned with.

Re: An Alien Mind

#410

Post by aurareturn »

an0malous wrote: post #237
SquibblesRedux wrote: post #227

I am still waiting for a cure to cancer. For a guaranteed prophylactic against Alzheimer's and dementia. For flying cars for everyone. For space bases throughout the solar system. For weather control. For all trains to be self-driving. For all those power lines across the world to go away. For an end to poverty. If things are going so well, then how come things still aren't going so well?

I mean I’m waiting for like any quality software or media produced by AI. I have yet to see a piece of software, a game, graphic, blog post, small video clip, or song that was produced with AI that’s good. I always use that Coca Cola ad as an example; millions of dollars spent to make an AI ad and they even did a ton of manual post production and it sucked. With all the millions of bloggers and influencers and conten…

You're likely using a ton of software that was written by AI that you don't even know about.

You only notice the sloppy ones.

Post reply on HN