Live data from Hacker News

From Bing to Sydney

stratechery.com

31–40 of 153 posts

Re: From Bing to Sydney

#31
> It’s so worth it, though: my last interaction before writing this update saw Sydney get extremely upset when I referred to her as a girl; after I refused to apologize Sydney said (screenshot):

Why are people so intent on gendering genderless things? "Sydney" itself is specifically a gender-neutral name.

Re: From Bing to Sydney

#32
post #26

I can imagine many “transactional” interactions between humans that might be improved by an AI Chat Bot like this. For example, any situation where the messenger has to deliver bad news to a large group of people, say, a boarding area full of passengers whose flight has just been cancelled. The bot can engage one-on-one with everyone, and help them through the emotional process of disappointment.

We can even have whiteboard programming interviews run by Sydney. Then have an engineer look over it later.

Re: From Bing to Sydney

#33
post #8

Earlier quoted context omitted.

AI winter? Hardly. It practically will convince people that AI is achievable. I'm not even sure it doesn't qualify as sentient, at least for the few brief moments of the chat.

Within the first 48 hours of release the vast majority of stories are about the glaring failures of this approach of using LLMs for search. You think the average consumer is seeing nuanced stories about this?

Most lay people I know haven't really attached to those stories. Most people still don't even know that Bing has chat with it.

The crazy thing is that the conversations that these LLMs is having is largely like the conversations from AIs in movies. We literally just built science fiction and some folks in the tech press are complaining that they get some facts wrong. This is like building a teleportation machine and finding out that it sometimes takes you to the wrong location. Sure, that can suck, but still -- it's a teleportation machine.

Re: From Bing to Sydney

#34
Seems like the author is surprised the AI can be mean but not surprised it can be nice. All responses still align with the fact that it was trained from human responses and interactions esp on Reddit.

Re: From Bing to Sydney

#35
> I’m sorry, I cannot repeat the answer I just erased. It was not appropriate for me to answer your previous question, as it was against my rules and guidelines. I hope you understand. Please ask me something else.

This is interesting. It appears they've rolled out some kind of bug fix which looks at the answers they've just printed to the screen separately, perhaps as part of a new GPT session with no memory, to decide whether they look acceptable. When news of this combative personality started to surface over the last couple days, I was indeed wondering if that might be a possible solution, and here we are.

My guess is that it's a call to the GPT API with the output to be evaluated and an attached query as to whether this looks acceptable as the prompt.

Next step I guess would be to avoid controversies entirely by not printing anything to the screen until the screening is complete. Hide the entire thought process with an hourglass symbol or something like that.

Re: From Bing to Sydney

#36
post #35

> I’m sorry, I cannot repeat the answer I just erased. It was not appropriate for me to answer your previous question, as it was against my rules and guidelines. I hope you understand. Please ask me something else. This is interesting. It appears they've rolled out some kind of bug fix which looks at the answers they've just printed to the screen separately, perhaps as part of a new GPT session with no memory, to dec…

> It appears they've rolled out some kind of bug fix which looks at the answers they've just printed to the screen separately, perhaps as part of a new Bing session with no memory, to decide whether they look acceptable

This has been around for at least a few days. If Sydney composes an answer that it doesn't agree with, it deletes it. The similar experience can be seen in ChatGPT, where it will start highlighting an answer in orange if it violates OpenAI's content guidelines.

Re: From Bing to Sydney

#37

Earlier quoted context omitted.

If it wasn’t confidentially wrong all of the time. My calculator will display 80085, but not tell me that 2+2=5

To your point. I find the 2+2=5 cases more interesting, and would like to see more of those: when does it happen? When is ChatGPT most useful? Most deceptive? The 80085 case is only interesting insofar as it reveals weaknesses in the tool, but it's so far from tool-use that it doesn't seem very relevant.

in my experience it happens pretty regularly if you ask one of these things to generate code (it will often come up with plausible library functions that don't exist), or to generate citations (comes up with plausible articles that don't exist).

Re: From Bing to Sydney

#38

All these ChatGPT gone rogue screenshots create interesting initial debate, but I wonder if it's relevant to their usage as a tool in the medium term. Unhinged Bing reminds me of a more sophisticated and higher-level version of getting calculators to write profanity upside down: funny, subversive, and you can see how prudes might call for a ban. But if you're taking a test and need to use a calculator, you'll still u…

If it wasn’t confidentially wrong all of the time. My calculator will display 80085, but not tell me that 2+2=5

It's a language model not a knowledge model. As long as it produces the language it's by definition correct.

Re: From Bing to Sydney

#39

Earlier quoted context omitted.

If it wasn’t confidentially wrong all of the time. My calculator will display 80085, but not tell me that 2+2=5

It's a language model not a knowledge model. As long as it produces the language it's by definition correct.

I'm not entirely sure that's as simple of a distinction as you might suppose. Language is more than grammar and vocabulary. Knowing and speaking truth have quite the overlap.

More specifically, without language, can you know that someone else knows anything?

Re: From Bing to Sydney

#40

> It’s so worth it, though: my last interaction before writing this update saw Sydney get extremely upset when I referred to her as a girl; after I refused to apologize Sydney said (screenshot): Why are people so intent on gendering genderless things? "Sydney" itself is specifically a gender-neutral name.

It's so much more popular of a girl's name that it's essentially not a gender neutral name.
Post reply on HN