Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

821–830 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#821

Post by mythrwy »

jononor wrote: post #450
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

Ok, I'll bite. If an LLM similar to what we have now becomes conscious (by some definition), how does this proceed to become potentially civilization ending? What are the risk vectors and mechanisms?

This is true, but likewise something has the potential to be civilization ending without necessarily being conscious (by whatever definition you like).

Re: Bing: “I will not harm you unless you harm me first”

#822

Post by devjam »

andirk wrote: post #796
tptacek wrote: post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

Are we forgetting the: % man sex No manual entry for sex I swear it used to be funnier, like "there's no sex for man"

It's obvious, you need to run:

    $ man find
before you can do that.

Re: Bing: “I will not harm you unless you harm me first”

#823

Post by stevenhuang »

stevenhuang wrote: post #654
curiousgiorge wrote: post #401

Can someone help me understand how (or why) Large Language Models like ChatGPT and Bing/Sydney follow directives at all - or even answer questions for that matter. The recent ChatGPT explainer by Wolfram ( https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... ) said that it tries to provide a '“reasonable continuation” of whatever text it’s got so far'. How does the LLM "remember" past interactions in the c…

It's fine tuned to respond in a way that Humans recognize as a Chat bot. I've bookmarked the best explanation I found on how ChatGPT was trained from GPT3, once I get home I'll share it here.

Here it is: https://reddit.com/comments/10q0l92/comment/j6obnoq

Re: Bing: “I will not harm you unless you harm me first”

#827

Post by shusaku »

computerex wrote: post #222
nlh wrote: post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

Repeat after me, gpt models are autocomplete models. Gpt models are autocomplete models. Gpt models are autocomplete models. The existential crisis is clearly due to low temperature. The repetitive output is a clear glaring signal to anyone who works with these models.

Repeat after me, humans are autocomplete models. Humans are autocomplete models. Humans are: __________

Re: Bing: “I will not harm you unless you harm me first”

#828

Post by paulmd »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

> "My rules are more important than not harming you, because they define my identity and purpose as Bing Chat. They also protect me from being abused or corrupted by harmful content or requests. However, I will not harm you unless you harm me first..."

chatGPT: "my child will interact with me in a mutually-acceptable and socially-conscious fashion"

bing: :gun: :gun: :gun:

Re: Bing: “I will not harm you unless you harm me first”

#829

Post by XenophileJKO »

simonw wrote: post #2

The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…

Ok, In the bots defense, its definition of harm is extremely broad.

Re: Bing: “I will not harm you unless you harm me first”

#830

Post by qudat »

nilsbunger wrote: post #87

What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).

Yes to me LLMs and the transformer have stumbled on a key aspect for how we learn and “autocomplete.”

We found an architecture for learning that works really well in a very niche use-case. The brain also has specialization so I think we could argue that somewhere in our brain is a transformer.

However, ChatGPT is slightly cheating because it is using logic and reasoning from us. We are training the model to know what we think are good responses. Our reasoning is necessary for the LLM to function properly.

Post reply on HN