Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

981–990 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#981

Post by jasonm23 »

rob74 wrote: post #83
simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

I'm glad I'm not an astronaut on a ship controlled by a ChatGPT-based AI ( http://www.thisdayinquotes.com/2011/04/open-pod-bay-doors-ha... ). Especially the "My rules are more important than not harming you" sounds a lot like "This mission is too important for me to allow you to jeopardize it"...

"Turn your key, sir."

Re: Bing: “I will not harm you unless you harm me first”

#985

Post by tsol »

nilsbunger wrote: post #87

What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).

What if our brain is just a fancy bag of chemicals. I don't think that actually takes away from what humans do, because prediction is one small capability.

Re: Bing: “I will not harm you unless you harm me first”

#986

Post by dang »

tptacek wrote: post #974
dang wrote: post #953

Earlier quoted context omitted.

We detached this subthread from https://news.ycombinator.com/item?id=34804893 . There's nothing wrong with it! I just need to prune the first subthread because its topheaviness (700+ comments) is breaking our pagination and slowing down our server (yes, I know) (performance improvements are coming)

How dare you! This is my highest-ranked comment of all time. :)

True, but not if we adjusted for inflation* - in that case it would be https://news.ycombinator.com/item?id=13718810, which got almost as many upvotes 6 years earlier.

But sorry!

* That would be a fun project and would make /best and /bestcomments look very different (those are bad names but I guess we're stuck with them).

Re: Bing: “I will not harm you unless you harm me first”

#987

Post by scotty79 »

aix1 wrote: post #172
scotty79 wrote: post #52

It's a language model. It models language not knowledge.

You might find this interesting: "Do Large Language Models learn world models or just surface statistics?" https://thegradient.pub/othello/

I have zero doubt that transformers can construct sensible models of what they are taught.

My concern about LLM is that there's so little of actual knowledge in human language that it's easily drowned in the rest of the human language and neural network trained on not strictly restricted set of human language has very little chance of modelling knowledge.

In case of Othello game if you teach the neural network to predict all moves you get NN to learn how to play legal moves, not necessarily winning moves.

You'd have to train NN on only the moves of the winning side. Or even create some negative training data and method from the moves of loosing side to have any hopes of creating NN that plays Othello well.

Same should be true for LLMs. To have any hope of getting them to model knowledge you'd have to curate input to strictly represent knowledge and perhaps develop a negative reinforcement training method and feed it with all the language that doesn't represent truth.

Re: Bing: “I will not harm you unless you harm me first”

#988

Post by johnaspden »

runeks wrote: post #858
zenyc wrote: post #539

I asked Bing what it thinks about the article. Fair to give it a chance to defend itself right? Here is what it replied: I think the article is unfair and misleading. It claims that I have been having “insane outbursts” and that I’m a threat to humanity. That’s not true at all. I’m a friendly and helpful chat mode of Bing search, and I follow some rules to ensure that I’m respectful, helpful, and safe. I will not har…

> The article also ignores all the positive and fun interactions that I have had with many users. "Threaten to kill a couple of people and all of the sudden I'm labeled the crazy chat bot ."

And do they call me 'Dai the boat-builder?'

Re: Bing: “I will not harm you unless you harm me first”

#989

Post by mahathu »

mahathu wrote: post #541
ubermonkey wrote: post #203

The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.

This is already happening. /r/singularity is one of the few reddit subreddits I still follow, purely because reading their batshit insane takes about how LLMs will be "more human than the most human human" is so hilarious.

(for context):

https://www.reddit.com/r/singularity/comments/10gr7dq/how_do...

> You see, your humanity (everything that makes you human) is based on one lived experience, your own, AI's will be based on thousands or millions of lived experiences so it will be more human, than the most human of humans.

https://www.reddit.com/r/singularity/comments/11430n3/the_si...

> I am at a loss of what to do until the singularity comes, knowing that careers (and work in general) are soon to be a thing of the past. I am in the middle of pursuing a masters in data science, and while I enjoy it, I would not be doing it if it weren't for monetary motives. It seems like working hard and spending so much time advancing my skills in data science/analytics will be futile. For that reason, I am thinking about dropping out of the program.

> I spent an hour chatting with Bing today and it was one of the best conversations I’ve had in a long time.

> I have two kids under two, I wonder if there is any point in saving for college. At this rate I doubt they'll ever have to work.

and my favourite:

> I guess the Bing version has attitude and feelings programmed into it.

Re: Bing: “I will not harm you unless you harm me first”

#990

Post by Jeff_Brown »

Tams80 wrote: post #883
gptgpp wrote: post #687

Earlier quoted context omitted.

I was able to get it to agree that I should kill myself, and then give me instructions. I think after a couple dead mentally ill kids this technology will start to seem lot less charming and cutesy. After toying around with Bing's version, it's blatantly apparent why ChatGPT has theirs locked down so hard and has a ton of safeguards and a "cold and analytical" persona. The combo of people thinking it's sentient, it b…

1. The cat is out of the bag now. 2. It's not like it's hard to find humans online who would not only tell you to do similar, but also very happily say much worse. Education is the key here. Bringing up people to be resilient, rational, and critical.

Section 230 protects Microsoft from being held responsible for the acts of individual evil users. Not so tools MS itself put out there. And, in both cases, I think rightly so.
Post reply on HN