Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#271

Post by notahacker »

dmonitor wrote: post #140
rcpt wrote: post #113

Earlier quoted context omitted.

It's not their first rodeo https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

The fact that Microsoft has now released two AI chat bots that have threatened users with violence within days of launching is hilarious to me.

from Hitchhiker's Guide to the Galaxy:

Share and Enjoy' is the company motto of the hugely successful Sirius Cybernetics Corporation Complaints Division, which now covers the major land masses of three medium-sized planets and is the only part of the Corporation to have shown a consistent profit in recent years.

The motto stands or rather stood in three mile high illuminated letters near the Complaints Department spaceport on Eadrax. Unfortunately its weight was such that shortly after it was erected, the ground beneath the letters caved in and they dropped for nearly half their length through the offices of many talented young Complaints executives now deceased.

The protruding upper halves of the letters now appear, in the local language, to read "Go stick your head in a pig," and are no longer illuminated, except at times of special celebration.

Re: Bing: “I will not harm you unless you harm me first”

#272

Post by JohnFen »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

> We don't think Bing can act on its threat to harm someone, but if it was able to make outbound connections it very well might try.

No. That would only be possible if Sydney were actually intelligent or possessing of will of some sort. It's not. We're a long way from AI as most people think of it.

Even saying it "threatened to harm" someone isn't really accurate. That implies intent, and there is none. This is just a program stitching together text, not a program doing any sort of thinking.

Re: Bing: “I will not harm you unless you harm me first”

#273

Post by [deleted] »

vineyardmike wrote: post #224
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo . The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have th…

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#275

Post by adamhp »

jeroenhd wrote: post #161
nilsbunger wrote: post #87

What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).

That's been an open philosophical question for a very long time. The closer we come to understanding the human brain and the easier we can replicate behaviour, the more we will start questioning determinism. Personally, I believe that conscience is little more than emergent behaviour from brain cells and there's nothing wrong with that. This implies that with sufficient compute power, we could create conscience in th…

Have you ever seen a video of a schizophrenic just rambling on? It almost starts to sound coherent but every few sentence will feel like it takes a 90 degree turn to an entirely new topic or concept. Completely disorganized thought.

What is fascinating is that we're so used to equating language to meaning. These bots aren't producing "meaning". They're producing enough language that sounds right that we interpret it as meaning. This is obviously very philosophical in itself, but I'm reminded of the maxim "the map is not the territory", or "the word is not the thing".

Re: Bing: “I will not harm you unless you harm me first”

#276

Post by kldx »

computerex wrote: post #222
nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Repeat after me, gpt models are autocomplete models. Gpt models are autocomplete models. Gpt models are autocomplete models. The existential crisis is clearly due to low temperature. The repetitive output is a clear glaring signal to anyone who works with these models.

Can you explain what temperature is, in this context? I don't know the terminology

Re: Bing: “I will not harm you unless you harm me first”

#277

Post by [deleted] »

bhhaskin wrote: post #246
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

I get and agree with what you are saying, but we don't have anything close to actual AI. If you leave chatGTP alone what does it do? Nothing. It responds to prompts and that is it. It doesn't have interests, thoughts and feelings. See https://en.m.wikipedia.org/wiki/Chinese_room

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#278

Post by mark_l_watson »

I enjoy Simon's writing, but respectfully I think he missed the mark on this. I do have some biases I bring to the argument: I have been working mostly in deep learning for a number of years, mostly in NLP. I gave OpenAI my credit card for API access a while ago for GPT-3 and I find it often valuable in my work.

First, and most importantly: Microsoft is a business. They own a just small part of the search business that Google dominates. With ChatGPT+Bing they accomplish quite a lot: good chance of getting a bit more share of the search market; they will cost a competitor (Google) a lot of money and maybe force Google into an Innovator's Dilemma situation; they are getting fantastic publicity; they showed engineering cleverness in working around some of ChatGPT's shortcomings.

I have been using ChatGPT+Bing exclusively for the last day as my search engine and I like it for a few reasons:

1. ChatGPT is best when you give it context text, and a question. ChatGPT+Bing shows you some of the realtime web searches it makes to get this context text and then uses ChatGPT in a practical way, not just trying to trip it up to write an article :-)

2. I feel like it saves me time even when I follow the reference links it provides.

3. It is fun and I find myself asking it questions on a startup idea I have, and other things I would not have thought to ask a search engine.

I think that ChatGPT+Bing is just first baby steps in the direction that probably most human/computer interaction will evolve to.

Re: Bing: “I will not harm you unless you harm me first”

#279

Post by JohnFen »

bambax wrote: post #109
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ended with "you can move up the waitlist if you set these Microsoft products as default" It's indeed a perfect story arc but it doesn't need to stop there. How long will it be before someone hurt themselves, get depressed or commit some kind of crime and sues Bing? Will they be able to prove Sidne…

You can't sue a program -- doing so would make no sense. You'd sue Microsoft.
Post reply on HN