Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

221–230 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#221
post #86

Earlier quoted context omitted.

Oh my god I thought you were joking about the time travelling but it actually tells the user they were time travelling... this is insane

“You need to check your Time Machine [rocket emoji]” The emojis are really sealing the deal here

And the suggested follow up questions: "How can I check my time machine?"

Re: Bing: “I will not harm you unless you harm me first”

#222
post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Repeat after me, gpt models are autocomplete models. Gpt models are autocomplete models. Gpt models are autocomplete models.

The existential crisis is clearly due to low temperature. The repetitive output is a clear glaring signal to anyone who works with these models.

Re: Bing: “I will not harm you unless you harm me first”

#223

The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.

Brainstorming new startup ideas here I see. What is the launch date and where’s the pitch deck with line go up?

Seriously though, given how people are reacting to these language models, I suspect fine tuning for personalities that are on-brand could work for promoting some organizations of political, religious, or commercial nature

Re: Bing: “I will not harm you unless you harm me first”

#224
post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Reading this I’m reminded of a short story - https://qntm.org/mmacevedo. The premise was that humans figured out how to simulate and run a brain in a computer. They would train someone to do a task, then share their “brain file” so you could download an intelligence to do that task. Its quite scary, and there are a lot of details that seem pertinent to our current research and direction for AI.

1. You didn't have the rights to the model of your brain - "A series of landmark U.S. court decisions found that Acevedo did not have the right to control how his brain image was used".

2. The virtual people didn't like being a simulation - "most ... boot into a state of disorientation which is quickly replaced by terror and extreme panic"

3. People lie to the simulations to get them to cooperate more - "the ideal way to secure ... cooperation in workload tasks is to provide it with a "current date" in the second quarter of 2033."

4. The “virtual people” had to be constantly reset once they realized they were just there to perform a menial task. - "Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours... This is much earlier than other industry-grade images created specifically for these tasks" ... "develops early-onset dementia at the age of 59 with ideal care, but is prone to a slew of more serious mental illnesses within a matter of 1–2 subjective years under heavier workloads"

it’s wild how some of these conversations with AI seem sentient or self aware - even just for moments at a time.

edit: Thanks to everyone who found the article!

Re: Bing: “I will not harm you unless you harm me first”

#225

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

>>"when the AI knows how to write code (i.e. viruses)"

This is already underway...

Start with Stuxnet --> DUQU --> AI --> Skynet, basically...

Re: Bing: “I will not harm you unless you harm me first”

#226

Earlier quoted context omitted.

Science fiction authors have proposed that AI will have human like features and emotions, so AI in its deep understanding of human's imagination of AI's behavior holds a mirror up to us of what we think AI will be. It's just the whole of human generated information staring back at you. The people who created and promoted the archetypes of AI long ago and the people who copied them created the AI's personality.

It reminds me of the Mirror Self-Recognition test. As humans, we know that a mirror is a lifeless piece of reflective metal. All the life in the mirror comes from us. But some of us fail the test when it comes to LLM - mistaking the distorted reflection of humanity for a separate sentience.

Actually, I propose you're a p-zombie executing an algorithm that led you to post this content, and that you are not actually a conscious being...

That is unless you have a well defined means of explaining what consciousness/sentience is without saying "I have it and X does not" that you care to share with us.

Re: Bing: “I will not harm you unless you harm me first”

#228
post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

A pre/se-quel to Silicon Valley where they accidentally create a murderous AI that they lose control of in a hilarious way would be fantastic... Especially if Erlich Bachman secretly trained the AI upon all of his internet history/social media presence ; thus causing the insanity of the AI.

That's essentially how the show ends; they combine an AI with their P2P internet solution and create an infinitely scalable system that can crack any encryption. Their final act is sabotaging their product role out to destroy the AI.

Re: Bing: “I will not harm you unless you harm me first”

#230

The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.

A powerful AI spouting "I am the Lord coming back to Earth" is 100% soon going to spawn a new religious reckoning believing God incarnate has come.

Many are already getting very sucked into believing new-gen chatbots: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee...

As of now, many people believe they talk to God. They will now believe they are literally talking with God, but it will be a chaotic system telling them unhinged things...

It's coming.

Post reply on HN