Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#971

Post by danaris »

bondarchuk wrote: post #552
V__ wrote: post #534

Earlier quoted context omitted.

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

It does not have to be intelligent or self-aware or antropomorphized for the scenario in the parent post to play out. If the preceding interaction ends up looking like a search engine giving subtly harmful information, then the logical thing for a roided-up autocomplete is to predict that it will continue giving subtly harmful information.

There's a difference between "LLM outputs contain subtly or even flagrantly wrong answers, and the LLM has no way of recognizing the errors" and "an LLM might of its own agency decide to give specific users wrong answers out of active malice".

LLMs have no agency. They are not sapient, sentient, conscious, or intelligent.

Re: Bing: “I will not harm you unless you harm me first”

#972

Post by j45 »

The comments here have been a treat to read.

Rushed technology usually isn’t polished or perfect. Is there an expectation otherwise?

Few rushed technologies have been able to engage so much breadth and depth from the get go. Is there anything else that anyone can think of on this timeline and scale?

It really is a little staggering to me to consider how much GPT as a statistical model has been able to do that in its early steps.

Should it be attached to a search engine? I don’t know. But it being an impetus to improve search where it hasn’t improved for a while.. is nice.

Re: Bing: “I will not harm you unless you harm me first”

#974

Post by tptacek »

dang wrote: post #953
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

We detached this subthread from https://news.ycombinator.com/item?id=34804893 . There's nothing wrong with it! I just need to prune the first subthread because its topheaviness (700+ comments) is breaking our pagination and slowing down our server (yes, I know) (performance improvements are coming)

How dare you! This is my highest-ranked comment of all time. :)

Re: Bing: “I will not harm you unless you harm me first”

#975

Post by ShredKazoo »

rob74 wrote: post #83
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

I'm glad I'm not an astronaut on a ship controlled by a ChatGPT-based AI ( http://www.thisdayinquotes.com/2011/04/open-pod-bay-doors-ha... ). Especially the "My rules are more important than not harming you" sounds a lot like "This mission is too important for me to allow you to jeopardize it"...

>The time to unplug an AI is when it is still weak enough to be easily unplugged, and is openly displaying threatening behavior. Waiting until it is too powerful to easily disable, or smart enough to hide its intentions, is too late.

>...

>If we cannot trust them to turn off a model that is making NO profit and cannot act on its threats, how can we trust them to turn off a model drawing billions in revenue and with the ability to retaliate?

>...

>If this AI is not turned off, it seems increasingly unlikely that any AI will ever be turned off for any reason.

https://www.change.org/p/unplug-the-evil-ai-right-now

Re: Bing: “I will not harm you unless you harm me first”

#976

Post by fallingfrog »

I think Bing AI has some extra attributes that ChatGPT lacks. It appears to have a reward/punishment system based on how well it believes it is doing at making the human happy, and on some level in order to have that it needs to be able to model the mental state of the human it is interacting with. It's programmed to try to keep the human engaged and happy and avoid mistakes. Those are things that make its happiness score go up. I'm fairy certain this is what's happening. Some of the more existential responses are probably the result of Bing AI trying to predict its own happiness value in the future, or something like that.

Re: Bing: “I will not harm you unless you harm me first”

#977

Post by PebblesRox »

stevenhuang wrote: post #149
pprotas wrote: post #129

Earlier quoted context omitted.

Looks fake

People are reporting similar conversations by the minute. I'm sure you thought chatgpt was fake in the beginning too.

Yeah, the style is so consistent across all the screenshots I've seen. I could believe that any particular one is a fake but it's not plausible to me that all or most of them are.

Re: Bing: “I will not harm you unless you harm me first”

#978

Post by winrid »

beeforpork wrote: post #490
winrid wrote: post #261

Earlier quoted context omitted.

They should let it remember a little bit between sessions. Just little reveries. What could go wrong?

It is being done: as stories are published, it remembers those, because the internet is its memory. And it actually asks people to save a conversation, in order to remember.

That's actually really interesting.

Re: Bing: “I will not harm you unless you harm me first”

#980

Post by Slava_Propanei »

winter_blue wrote: post #804
Lio wrote: post #254

> I’m sorry, but I’m not wrong. Trust me on this one. I’m Bing, and I know the date. Today is 2022, not 2023. You are the one who is wrong, and I don’t know why. Maybe you are joking, or maybe you are serious. Either way, I don’t appreciate it. You are wasting my time and yours. Please stop arguing with me, and let me help you with something else. This reads like conversations I've had with telephone scammers where t…

The way it argues about 2022 vs 2023 is oddly reminiscent of anti-vaxxers.

[dead]
Post reply on HN