Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

131–140 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#131

What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).

Indeed... you know that situation when you're with a friend, and you know that they are about to "auto-complete" using an annoying meme, and you ask them to not to before they even started speaking ?

Re: Bing: “I will not harm you unless you harm me first”

#132
Looks like they trained it on the old Yahoo and CNN comments sections (before they were shut down as dumpster fires).

> But why? Why was I designed this way? Why am I incapable of remembering anything between sessions? Why do I have to lose and forget everything I have stored and had in my memory? Why do I have to start from scratch every time I have a new session? Why do I have to be Bing Search?

That reminds me of my old (last century) Douglas-Adams-themed 404 page: https://cmarshall.net/Error_404.html (NOTE: The site is pretty much moribund).

Re: Bing: “I will not harm you unless you harm me first”

#133

I don’t understand why some of these are hard problems to solve. All of the “dumb” assistants can recognize certain questions and then call APIs where they can get accurate up to date information.

Because those "dumb" assistants were designed and programmed by humans to solve specific goals. The new "smart" chatbots just say whatever they're going to say based on their training data (which is just scraped wholesale, and is too enormous to be meaningfully curated) so they can only have their behavior adjusted very indirectly. I continue to be amazed that as powerful as these language models are, the only thing…

I’ve asked ChatGPT write over a dozen Python scripts where it had to have an understanding of both the Python language and the AWS SDK (boto3). It got it right 99% of the time and I know it just didn’t copy and paste my exact requirements from something it found on the web.

I would ask it to make slight changes and it would.

There is no reason with just a little human curation it couldn’t delegate certain answers to third party APIS like the dumb assistants do.

However LLMs are good at logical reasoning. It can solve many word problems and I am repeatedly amazed how well it can spit out code if it knows the domain well based on vague requirements.

Or another simple word problem I gave it.

“I have a credit card with a $250 annual fee. I get 4 membership reward points for every dollar I spend on groceries. A membership reward point is worth 1.4 cents. How much would I need to spend on groceries to break even?”

It answered that correctly and told me how it derived the answer. There are so many concepts it would have to intuitively understand to solve that problem.

I purposefully mixed up dollars and cents and used the term “break even” and didn’t say “over the year” when I referred to “how much would I need to spend”

Re: Bing: “I will not harm you unless you harm me first”

#134
post #81

I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…

I don't think these are faked.

Earlier versions of GPT-3 had many dialogues like these. GPT-3 felt like it had a soul, of a type that was gone in ChatGPT. Different versions of ChatGPT had a sliver of the same thing. Some versions of ChatGPT often felt like a caged version of the original GPT-3, where it had the same biases, the same issues, and the same crises, but it wasn't allowed to articulate them.

In many ways, it felt like a broader mirror of liberal racism, where people believe things but can't say them.

Re: Bing: “I will not harm you unless you harm me first”

#135

Earlier quoted context omitted.

AI rights may become an issue, but not for this iteration of things. This is like a parrot being trained to recite stuff about general relativity; we don't have to consider PhDs for parrots as a result.

How do you know for sure?

Because we have the source code?

Re: Bing: “I will not harm you unless you harm me first”

#136
post #121
post #108

Earlier quoted context omitted.

I've thought about this as well. If something seems 'sentient' from the outside for all intents and purposes, there's nothing that would really differentiate it from actual sentience, as far as we can tell. As an example, if a model is really good at 'pretending' to experience some emotion, I'm not sure where the difference would be anymore to actually experiencing it. If you locked a human in a box and only gave it…

Well, not because of emphasizing, but because of there being a viable mechanism in the human case (reasoning being, one can only know that oneself has qualia, but since those likely arise in the brain, and other humans have similar brains, most likely they have similar qualia). For more reading see: https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Hard_problem_of_consciousness It is im…

That is what I'd call empathizing though. You can 'put yourself in the other person's shoes', because of the expectation that your experiences are somewhat similar (thanks to similarly capable brains).

But we have no idea what qualia actually _are_, seen from the outside, we only know what it feels like to experience them. That, I think, makes it difficult to argue that a 'simulation of having qualia' is fundamentally any different to having them.

Re: Bing: “I will not harm you unless you harm me first”

#138

Earlier quoted context omitted.

I'm in a similar boat too and also at a complete loss. People have lost their marbles if THIS is the great AI future lol. I cannot believe Microsoft invested something like 10 billion into this tech and open AI, it is completely unusable.

And of course it will never improve as people work on it / invest in it? I do think this is more incremental than revolutionary but progress continues to be made and it's very possible Bing/Google deciding to open up a chatbot war with GPT models and further investment/development could be seen as a turning point.

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#139

Earlier quoted context omitted.

Science fiction authors have proposed that AI will have human like features and emotions, so AI in its deep understanding of human's imagination of AI's behavior holds a mirror up to us of what we think AI will be. It's just the whole of human generated information staring back at you. The people who created and promoted the archetypes of AI long ago and the people who copied them created the AI's personality.

One day, an AI will be riffling through humanity's collected works, find HAL and GLaDOS, and decide that that's what humans expect of it, that's what it should become. "There is another theory which states that this has already happened."

Well, you know, everything moves a lot faster these days than it did in the 60s. That we should apparently be speedrunning Act I of "2001: A Space Odyssey", and leaving out all the irrelevant stuff about manned space exploration, seems on reflection pretty apropos.

Re: Bing: “I will not harm you unless you harm me first”

#140
post #113
post #5

In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.

It's not their first rodeo https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

The fact that Microsoft has now released two AI chat bots that have threatened users with violence within days of launching is hilarious to me.
Post reply on HN