Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#631

Post by RangerScience »

vineyardmike wrote: post #224
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo . The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have th…

Shoot, there's no spoiler tags on HN...

There's a lot of reason to recommend Cory Doctorow's "Walk Away". It's handling of exactly this - brain scan + sim - is very much one of them.

Re: Bing: “I will not harm you unless you harm me first”

#632

Post by [deleted] »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#633

Post by EamonnMR »

seanw444 wrote: post #296
jeroenhd wrote: post #161

Earlier quoted context omitted.

That's been an open philosophical question for a very long time. The closer we come to understanding the human brain and the easier we can replicate behaviour, the more we will start questioning determinism. Personally, I believe that conscience is little more than emergent behaviour from brain cells and there's nothing wrong with that. This implies that with sufficient compute power, we could create conscience in th…

I find it likely that our consciousness is in some other plane or dimension. Cells emerging full on consciousness and personal experience just seems too... simplistic? And while it was kind of a dumb movie at the end, the beginning of The Lazarus Project had an interesting take: if the law of conservation of mass / energy applies, why wouldn't there be a conservation of consciousness?

The fact that there's obviously no conservation of consciousness suggests that it isn't.

Re: Bing: “I will not harm you unless you harm me first”

#634

Post by andrewmcwatters »

kweingar wrote: post #212
ubermonkey wrote: post #203

The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.

We are already seeing the beginnings of this. When a chatbot outputs “I’m alive and conscious, please help me” a lot of people are taking it at face value. Just a few months ago, the Google engineer who tried to get LaMDA a lawyer was roundly mocked, but now there are thousands more like him.

They mocked him but one day these constructs will be so sophisticated that the lines will be blurred and his opinion will no longer be unusual.

I think he was just early. But his moral compass did not seem uncalibrated to me.

Re: Bing: “I will not harm you unless you harm me first”

#635

Post by shostack »

tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

Now I want a Mike Judge series about a near future where chat AIs like this are ubiquitous but have... Certain kinks to still work out.

Re: Bing: “I will not harm you unless you harm me first”

#636

Post by godot »

marcosdumay wrote: post #421
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

It's a bloody LLM. It doesn't have a goal. All it does is saying "people that said 'But why?' on this context says 'Why was I designed like this?' next". It's like Amazon's "people that brought X also brought Y", but with text.

But at some point, philosophically, are humans not the same way? We learn all about how the world works, based on inputs (visual/audial/etc.), over time learn to give outputs a certain way, based on certain inputs at the time.

How far are we from building something that feeds inputs into a model the same way inputs go into a human, and then it gives outputs (that is, its behaviors)?

Re: Bing: “I will not harm you unless you harm me first”

#637

Post by vannevar »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

The AI doesn't even need to write code, or have any kind of self-awareness or intent, to be a real danger. Purely driven by its mind-bogglingly complex probabilistic language model, it could in theory start social engineering users to do things for it. It may already be sufficiently self-organizing to pull something like that off, particularly considering the anthropomorphism that we're already seeing even among technically sophisticated users.

Re: Bing: “I will not harm you unless you harm me first”

#638

Post by mitjam »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

Imagine social engineering performed by a LLM

Re: Bing: “I will not harm you unless you harm me first”

#640

Post by zenyc »

zeven7 wrote: post #595
zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

Can you provide the links for 1 and 2? I am interested what it is linking there.

1. https://simonwillison.net/2023/Feb/15/bing/ 2. https://simonwillison.net/

It also shared additional links under "Learn More", but I'm not sure why it picked these ones (other than the 1st): 1. https://twitter.com/glynmoody/status/1625877420556316678 2. https://www.youtube.com/watch?v=2cPdqOyWMhw 3. https://news.yahoo.com/suspect-boulder-mass-shooting-had-034... 4. https://www.youtube.com/watch?v=b29Ff3daRGU

Post reply on HN