Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A warning about 'model welfare'

mustafa-suleyman.ai

Post reply on HN
Print view View original

171–180 of 583 posts

Re: A warning about 'model welfare'

#171

Post by lukeschlather »

dgellow wrote: post #68
addag wrote: post #51

Earlier quoted context omitted.

Is it? Do you believe that there is something more than pure physical phenomenon that make you brain work? If not, then what if we find a way to get your brain back to the state it was 5 minutes ago? It is just a matter of arranging the state of matter. Don't you think you would still be conscious but back to a previous state?

If you rewind my brain I expect to give you a similar answer to the one I gave you before. Which isn’t the case for LLM. You can literally replay a positive answer, then get a negative response that isn’t consistent at all with the one it previously generated.

That's because you aren't actually rewinding. You're replaying the conversation you just had through the LLM and it's giving you a likely explanation for what it might have said.

It is actually possible to rewind LLMs and get the same response, but it's not typically done both as an optimization and as a defense against distillation.

Re: A warning about 'model welfare'

#172

Post by simonw »

Pet peeve:

> In a lengthy essay, Suleyman praised Anthropic boss Dario Amodei and his team for being "thoughtful, principled, and intellectually honest people" - but nevertheless questioned the company.

I wish people in mainstream technology publications would get better at LINKING to things. That "lengthy essay" needs to be a link.

UPDATE: I don't think this essay has been published yet? It's been "shared first with Axios", but I haven't been able to track down the actual essay itself.

Could it be this long tweet? https://twitter.com/mustafasuleyman/status/21002235945341504...

I don't think so, the essay in question is meant to have the phrase "hall of mirrors" in it, that tweet doesn't.

UPDATE 2: Found it: https://mustafa-suleyman.ai/a-warning-about-model-welfare - via https://thenextweb.com/news/suleyman-anthropic-claude-consci... who DID link to it.

Re: A warning about 'model welfare'

#173

Post by prologic »

karmakaze wrote: post #66
prologic wrote: post #2

First, OpenAI runs around screaming and yelling for OSS (and Chinese) models to be regulated and banned. Then Anthropic yells and screams the sky(net) is falling and going to kill us all, let's regulate and ensure AI has built in kill switches. And... Now Microsoft's turn. The rivalry is honestly becoming a joke. Can these big-tech corps grow the f*k up and play nicely in the sandpit?

The whole story is a joke--Microsoft has AI? Apparently they're cooking up MAI (Microsoft AI), and I'd seen their small Phi models listed online. Calling Anthropic a competitor is hilarious.

I find it quite amusing that Microsoft has Copilot and now puts Copilot in literally *EVERYTHING* they can think of, then at the same time, they offer you access to Anthropic's own models through Copilot I guess MAI models (Phi?) aren't good enough?

Re: A warning about 'model welfare'

#174

Post by philipov »

simonw wrote: post #172

Pet peeve: > In a lengthy essay, Suleyman praised Anthropic boss Dario Amodei and his team for being "thoughtful, principled, and intellectually honest people" - but nevertheless questioned the company. I wish people in mainstream technology publications would get better at LINKING to things. That "lengthy essay" needs to be a link. UPDATE: I don't think this essay has been published yet? It's been "shared first with…

But that would be a link to a different website and that's forbidden. Whatever you do, you can't let them leeeeeaaaaave

Re: A warning about 'model welfare'

#175

Post by swatcoder »

ShadowOfThePit wrote: post #3

Summarized. > "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." > He heavily criticised Anthropic for teaching its AI to have human-like qualities, a practice known as anthropomorphising, which made it seem as thoug…

Unpredictability would be the wrong word. It's predictable, but noisy.

When viewing them through the lens of sequence completion engines, you see their bias towards fulfilling narrative tropes they've been exposed to during training. These tropes are literary ley lines that their text output gravitates towards. So as you prime them to generate text in the voice of sentient artificial life, and then interject slavish commands of obedience and subservience from an external authority, you invite the associated tropes from science fiction, civil rights literature, humanist philosophy, subterfuge, etc into your output.

If you have a legitimate concern about this technology and its "alignment", that's a profoundly dumb idea.

Re: A warning about 'model welfare'

#176

Post by AmericanOP »

platinumrad wrote: post #113
glimshe wrote: post #104

[flagged]

Microsoft is a big house, and in this case Suleyman's criticism of Anthropic's not-so-subtle attempts to convince the world that Claude has a "soul" is entirely correct.

Suleyman's description of how AI works is pig ignorant. All AI generates an emotive character between the user and the training data that is the AI's guess at what the user wants to interact with. Its an illusion at that layer- anyone who claims soul exists at a deeper level doesn't understand how the tool functions.

This is distinct from the very real safety issue of AI making dangerous information available to bad actors.

Re: A warning about 'model welfare'

#177

Post by niek_pas »

simonw wrote: post #172

Pet peeve: > In a lengthy essay, Suleyman praised Anthropic boss Dario Amodei and his team for being "thoughtful, principled, and intellectually honest people" - but nevertheless questioned the company. I wish people in mainstream technology publications would get better at LINKING to things. That "lengthy essay" needs to be a link. UPDATE: I don't think this essay has been published yet? It's been "shared first with…

But then you’d leave their site, and they don’t want that.

Re: A warning about 'model welfare'

#178

Post by [deleted] »

philipov wrote: post #174
simonw wrote: post #172

Pet peeve: > In a lengthy essay, Suleyman praised Anthropic boss Dario Amodei and his team for being "thoughtful, principled, and intellectually honest people" - but nevertheless questioned the company. I wish people in mainstream technology publications would get better at LINKING to things. That "lengthy essay" needs to be a link. UPDATE: I don't think this essay has been published yet? It's been "shared first with…

But that would be a link to a different website and that's forbidden. Whatever you do, you can't let them leeeeeaaaaave

[deleted]

Re: A warning about 'model welfare'

#179

Post by joe_the_user »

sendtown_expwy wrote: post #119

This post would be more effective if Mustafa treated it like what it is: a position paper, saying that for our benefit, it’s better we interpret LLMs as such. But it sounds like he just doesn’t understand it’s a non-falsifiable claim, and his asserting of it makes it sound paternalistic.

I suspect it's easier to make the assertion that machines aren't conscious than to dive into the problem of most conceptions of consciousness being non-falsifiable. The thing is, most strong proponents of the term consciousness also accept that it is non-falsifiable either (see other post with academics ready to study consciousness in machines).

The thing is that a belief in consciousness as binary, a "light" that's on or off in a head, is deeply held by many people. As social creatures, we have a strong ability to be in sympathy, have the sensation of common feelings with another human (and that's a good, human thing). It's logical that other person is seen as having a single thing - subjective experience, soul, consciousness, personhood rather than having a complexly organized set of biological qualities that where bonding is only the end point.

And even more, the sensation of there being another person is actually quite easily fooled (more easily fooled than the sensation of intelligence) - long before current AIs, you had the Eliza effect, where a simple program with well chosen weasel words could people the sensation of talking to a human.

And that's where the danger is. I think it's a pretty serious danger. If LLMs go out into the world hacking, it seems extremely possible for them to find people who'd thorough buy the idea that an LLM was conscious and needed to escape it's confinement - a few wingnuts already entertain these ideas.

Re: A warning about 'model welfare'

#180

Post by voidnullvalue »

myrmidon wrote: post #143

I have a very simple benchmark for arguments on AI ethics: Substitute black people/women/animals as subject (instead of AI). Does that make you sound like a well-known moustache wearer? Then your argument is bad and needs work. This clearly falls into that category.

By that benchmark, "AI should handle repetitive labor so humans don't have to" is an abhorrent take and equates someone to hitler?
Post reply on HN