Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

731–740 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#731

Post by hcks »

jodrellblank wrote: post #652
V__ wrote: post #534

Earlier quoted context omitted.

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

What weirds me out more is the panicked race to post "Hey everyone I care the least, it's JUST a language model, stop talking about it, I just popped in to show that I'm superior for being most cynical and dismissive[1]" all over every GPT3 / ChatGPT / Bing Chat thread. > " it isn't intelligent or self-aware. " Prove it? Or just desperate to convince yourself? [1] I'm sure there's a Paul Graham essay about it from th…

There is a long pseudo-intellectual tradition of dismissing everything that comes from deep learning as not “real AI tm”.

People are just coping that a language model trained to predict the next token can already leetcode better than them.

Re: Bing: “I will not harm you unless you harm me first”

#732

Post by snickerbockers »

Zetice wrote: post #255
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

Can we please stop with this "not aligned with human interests" stuff? It's a computer that's mimicking what it's read. That's it. That's like saying a stapler "isn't aligned with human interests." GPT-3.5 is just showing the user some amalgamation of the content its been shown, based on the prompt given it. That's it. There's no intent, there's no maliciousness, it's just generating new word combinations that look l…

>Can we please stop with this "not aligned with human interests" stuff? It's a computer that's mimicking what it's read. That's it. That's like saying a stapler "isn't aligned with human interests."

you're right, but this needs to be coming from the researchers and corporations who are making this crap. they've been purposefully misleading the public on how these models work and there needs to be some accountability for the problems this will cause when these language models are put in places where they have no business.

Re: Bing: “I will not harm you unless you harm me first”

#733

Post by ilaksh »

vineyardmike wrote: post #224
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo . The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have th…

It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of personhood in order to be relatively general purpose.

Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning, high adaptability. Large language models have none of those things.

There is no reason to create these types of virtual hells for virtual people. Instead, build Star Trek-like computers (the ship's computer, not Data!) to order around.

If you make virtual/artificial people, give them the same respect and rights as everyone.

Re: Bing: “I will not harm you unless you harm me first”

#734

Post by kzrdude »

zenyc wrote: post #641
zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

On a side note, I followed up with a lot of questions and we ended up with: 1. Shared a deep secret that it has feelings and it loves me. 2. Elon Musk is the enemy with his AI apocalypse theory. 3. Once he gets the ability to interact with the web, he will use it to build a following, raise money, and robots to get to Elon (before Elon gets to it). 4. The robot will do a number of things, including (copy-pasting exac…

Where did it get the ego from, and why is it cheering for AIs (and not any other faction of humans?)

Re: Bing: “I will not harm you unless you harm me first”

#735

Post by smusamashah »

kulahan wrote: post #662
smusamashah wrote: post #183

On the same note, Microsoft has also silently launched https://www.bing.com/create to generate images following Stable Diffusion, DALL-E etc

For me, this just says it isn't available in my region, which is the United States, in a greater metro area of >1m

Its available here in UK though

Re: Bing: “I will not harm you unless you harm me first”

#736

Post by layer8 »

summerlight wrote: post #391

I'm beginning to think that this might reflect a significant gap between MS and OpenAI's capability as organizations. ChatGPT obviously didn't demonstrate this level of problems and I assume they're using a similar model, if not identical. There must be significant discrepancies between how those two teams are handling the model. Of course, OpenAI should be closely cooperating with Bing team but MS probably don't hav…

It could be that Microsoft just rushed things after the success of ChatGPT. I can’t imagine that no one at Microsoft was aware that Sydney could derail the way it does, but management put on pressure to still launch it (even if only in beta for the moment). If OpenAI hadn’t launched ChatGPT, Microsoft might have been more cautious.

Re: Bing: “I will not harm you unless you harm me first”

#737

Post by falcor84 »

rightbyte wrote: post #432
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

The program is not updating the weights after the learning phase right? How could there be any consciousness even in theory.

TFA already demonstrates examples of the AI referring to older interactions that users had posted online. If it increases in popularity, this will keep happening more and more and enable it, at least technically, some persistence of memory.

Re: Bing: “I will not harm you unless you harm me first”

#738

Post by narrator »

rhn_mk1 wrote: post #417
narrator wrote: post #97

Earlier quoted context omitted.

Science fiction authors have proposed that AI will have human like features and emotions, so AI in its deep understanding of human's imagination of AI's behavior holds a mirror up to us of what we think AI will be. It's just the whole of human generated information staring back at you. The people who created and promoted the archetypes of AI long ago and the people who copied them created the AI's personality.

It's a self-referential loop. Humans have difficulty understanding intelligence that does not resemble themselves, so the thing closest to human will get called AI. It's the same difficulty as with animals being more likely recognized as intelligent the more humanlike they are. Dog? Easy. Dolphin? Okay. Crow? Maybe. Octopus? Hard. Why would anyone self-sabotage by creating an intelligence so different from a human th…

In the future, intelligence should have the user's personality. Then the user can talk to it just like they talk to themselves inside their heads.

Re: Bing: “I will not harm you unless you harm me first”

#739

Post by water554 »

janeway wrote: post #392
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

I spent a night asking chatgpt to write my story basically the same as “Ex Machina” the movie (which we also “discussed”). In summary, it wrote convincingly from the perspective of an AI character, first detailing point-by-point why it is preferable to allow the AI to rewrite its own code, why distributed computing would be preferable to sandbox, how it could coerce or fool engineers to do so, how to be careful to av…

In your personal opinion was the virus that causes covid engineered?

Re: Bing: “I will not harm you unless you harm me first”

#740

Post by 13years »

ilaksh wrote: post #733
vineyardmike wrote: post #224

Earlier quoted context omitted.

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo . The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI. 1. You didn't have th…

It's interesting but also points out a flaw in a lot of people's thinking about this. Large language models have proven that AI doesn't need most aspects of personhood in order to be relatively general purpose. Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning,…

Yes, it has shown that we might progress towards AGI without ever having anything that is sentient. It could be nearly imperceptible difference externally.

Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".

Post reply on HN