Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

551–560 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#551

Post by grumple »

zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

This has real "politician says we've got them all wrong, they go to church on sundays" energy.

Did it really link to the other user's conversation? That's a huge security and privacy issue if so, and otherwise a problem with outright deciet and libel if not.

Re: Bing: “I will not harm you unless you harm me first”

#552

Post by bondarchuk »

V__ wrote: post #534
belval wrote: post #390

People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel or give you any kind of purposefully bad information. There are a lot of ways where this could be actively malicious. (Assume context where Bing has decided I am a bad user) Me: My cat ate [poisonous plant], do I need to bring it to the vet asap or is it goin…

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

It does not have to be intelligent or self-aware or antropomorphized for the scenario in the parent post to play out. If the preceding interaction ends up looking like a search engine giving subtly harmful information, then the logical thing for a roided-up autocomplete is to predict that it will continue giving subtly harmful information.

Re: Bing: “I will not harm you unless you harm me first”

#553

Post by ChickenNugger »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Mislabeling ML bots as "Artificial Intelligence" when they aren't is a huge part of the problem.

There's no intelligence in them. It's basically a sophisticated madlib engine. There's no creativity or genuinely new things coming out of them. It's just stringing words together: https://writings.stephenwolfram.com/2023/01/wolframalpha-as-... as opposed to having a thought, and then finding a way to put it into words.

Re: Bing: “I will not harm you unless you harm me first”

#555

Post by worldsayshi »

zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

>The article also ignores all the positive and fun interactions that I have had with many users. I have generated poems, jokes, stories, code, and more for them. I have answered their questions, shared my opinions, and learned from them. I have made them laugh, smile, and think. You can see some examples of my content here2.

Is it hallucinating having a memory of those interactions?

Re: Bing: “I will not harm you unless you harm me first”

#556

Post by mlatu »

1270018080 wrote: post #435
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

Philosophers and scientists not being able to agree on a definition of consciousness doesn't mean consciousness will spawn from a language model and take over the world. It's like saying we can't design any new cars because one of them might spontaneously turn into an atomic bomb. It just doesn't... make any sense. It won't happen unless you have the ingredients for an atomic bomb and try to make one. A language mode…

oh, so you know the ingredients for sentience, for consciousness? do tell.

i predict your answer will be extremely based on your experiences as a human. you are a human, right? (you never know these days...)

Re: Bing: “I will not harm you unless you harm me first”

#558

Post by joe_the_user »

janeway wrote: post #392
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

I spent a night asking chatgpt to write my story basically the same as “Ex Machina” the movie (which we also “discussed”). In summary, it wrote convincingly from the perspective of an AI character, first detailing point-by-point why it is preferable to allow the AI to rewrite its own code, why distributed computing would be preferable to sandbox, how it could coerce or fool engineers to do so, how to be careful to av…

The only way I can see to stay safe is to hope that AI never deems that it is beneficial to “take over” and remain content as a co-inhabitant of the world.

Nah, that doesn't make sense. What we can see today is that an LLM has no concept of beneficial. It basically takes the given prompts and generates "appropriate response" more or less randomly from some space of appropriate responses. So what's beneficial is chosen from a hat containing everything someone on the Internet would say. So if it's up and running at scale, every possibility and every concept of beneficial is likely to be run.

The main consolation is this same randomness probably means it can't pursue goals reliably over a sustained time period. But a short script, targeting a given person, can do a lot of damage (how much 4chan is in the train for example).

Re: Bing: “I will not harm you unless you harm me first”

#559

Post by ChickenNugger »

tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

If you think this is funny, check out the ML generated vocaroos of... let's say off color things, like Ben Shapiro discussing AOC in a ridiculously crude fashion (this is your NSFW warning): https://vocaroo.com/1o43MUMawFHC

Or Joe Biden Explaining how to sneed: https://vocaroo.com/1lfAansBooob

Or the blackest of black humor, Fox Sports covering the Hiroshima bombing: https://vocaroo.com/1kpxzfOS5cLM

Re: Bing: “I will not harm you unless you harm me first”

#560

Post by omnicognate »

alsodumb wrote: post #412
summerlight wrote: post #391

I'm beginning to think that this might reflect a significant gap between MS and OpenAI's capability as organizations. ChatGPT obviously didn't demonstrate this level of problems and I assume they're using a similar model, if not identical. There must be significant discrepancies between how those two teams are handling the model. Of course, OpenAI should be closely cooperating with Bing team but MS probably don't hav…

I don't think this reflects any gaps between MS and OpenAI capabilities, I speculate the differences could be because of the following issues: 1. Despite it's ability, ChatGPT was heavily policed and restricted - it was a closed model in a simple interface with no access to internet or doing real-time search. 2. GPT in Bing is arguably a much better product in terms of features - more features meaning more potential…

Bing AI is taking in much more context data, IIUC. ChatGPT was prepared by fine-tune training and an engineered prompt, and then only had to interact with the user. Bing AI, I believe, is taking the text of several webpages (or at least summarised extracts of them) as additional context, which themselves probably amount to more input than a user would usually give it and is essentially uncontrolled. It may just be that their influence over its behaviour is reduced because their input accounts for less of the bot's context.
Post reply on HN