Live data from Hacker News

From Bing to Sydney

stratechery.com

121–130 of 153 posts

Re: From Bing to Sydney

#121

All these ChatGPT gone rogue screenshots create interesting initial debate, but I wonder if it's relevant to their usage as a tool in the medium term. Unhinged Bing reminds me of a more sophisticated and higher-level version of getting calculators to write profanity upside down: funny, subversive, and you can see how prudes might call for a ban. But if you're taking a test and need to use a calculator, you'll still u…

Calculators have never snapped at a fragile person and degraded them. Bing Assistant seems to do it quite easily. A secure person who understands the technology can shrug that off, but those two criteria aren’t prerequisites for using the service. If Microsoft can’t shore this up, it’s only a matter of time before somebody (or their parent) holds Microsoft responsible for the advent of some trauma. Lawyers and the me…

> never snapped at a fragile person and degraded them.

Reminds me of the one about not assuming malice when it can easily be explained by incompetence. Unfortunately for the implementers the LLM can ipso facto be neither incompetent nor malicious. If however Microsoft is not being one of those, then it can only mean Microsoft is the other.

Re: From Bing to Sydney

#122

Ben’s got it just right. These things are terrible at the knowledge search problems they’re currently being hyped for. But they’re amazing as a combination of conversational partner and text adventure. I just asked ChatGPT to play a trivia game with me targeted to my interests on a long flight. Fantastic experience, even when it slipped up and asked what the name of the time machine was in “Back to the Future”. And t…

IMO it's only a matter of time before someone hooks up a LLM to a speech-to-text recognizer with a TTS engine like something from ElevenLabs, and you have a full blown "AI" that you can converse with. Once someone builds a LLM that can remember facts tied to your account this thing is going to go off the rails.

I like ChatGPT talks too much and would be annoying for this purpose.

Re: From Bing to Sydney

#123

Earlier quoted context omitted.

AI winter? Hardly. It practically will convince people that AI is achievable. I'm not even sure it doesn't qualify as sentient, at least for the few brief moments of the chat.

> I'm not even sure it doesn't qualify as sentient, at least for the few brief moments of the chat You need your head checked. Give it a short story and ask it a question which is not 100% explicit in the text. For example, give it Arthur C. Clarke's Food of the Gods and ask it was is Ambrosia in the story. Is a language model, and it behaves like a language model. It doesn't think. It's doesn't understand.

[deleted]

Re: From Bing to Sydney

#124
post #110
post #102

Earlier quoted context omitted.

> We can't just be probabilistically stringing together word tokens, we think in terms of meaning, right? We are probabalistically stringing together muscle movements that generate language as sound. That's not really controversial, otherwise we would call it magic. However, the complexity of our probabalistic word machine is far greater, in terms of both richness of inputs, motivation, and dimensionality.

>However, the complexity of our probabalistic word machine is far greater, in terms of both richness of inputs, motivation, and dimensionality. If thought (as expressed in language) is just probabilistic pattern matching, then how did we develop our own training data from scratch?

There is a huge universe of inputs, aka training data, that feeds into us, far more than a digital text based LLM. From that we generated the training data for the LLM. That data is just a sliver of the human experience.

Re: From Bing to Sydney

#125
post #65

Earlier quoted context omitted.

just tell them "Keep your answers below 150 characters in this conversation." at the start.

It can summarize its own output, the user directs everything about the output, style, format, length, etc. Everything.

> style

I like asking to to type like a frustrated teen on the phone. it huffs and puffs and rolls its virtual eyes.

prompt: could you pick a quantum computer at the mall for me?

response: ugh, seriously? you can't just buy a quantum computer at the mall, they're like super expensive and only a few companies sell them. Plus, they require special conditions to operate.

Re: From Bing to Sydney

#126

Earlier quoted context omitted.

IMO it's only a matter of time before someone hooks up a LLM to a speech-to-text recognizer with a TTS engine like something from ElevenLabs, and you have a full blown "AI" that you can converse with. Once someone builds a LLM that can remember facts tied to your account this thing is going to go off the rails.

I like ChatGPT talks too much and would be annoying for this purpose.

this is absolutely me anthropomorphizing them, but i found it quite funny how stiff chat gpt sounds compared to the (at times) completely deranged bing chat. its allmost like they have personalitys

Re: From Bing to Sydney

#127

All these ChatGPT gone rogue screenshots create interesting initial debate, but I wonder if it's relevant to their usage as a tool in the medium term. Unhinged Bing reminds me of a more sophisticated and higher-level version of getting calculators to write profanity upside down: funny, subversive, and you can see how prudes might call for a ban. But if you're taking a test and need to use a calculator, you'll still u…

Unhinged Bing reminds me of a more sophisticated and higher-level version of getting calculators to write profanity upside down: funny, subversive, and you can see how prudes might call for a ban. With all due respect, that seems very strained as an analogy - it's not a bug but a strange human interpretation of expected behavior. You could at least compare it to Microsoft Tay, the chatbot which tweeted profanity just…

>You could at least compare it to Microsoft Tay, the chatbot which tweeted profanity just because people figure out ways to get it to echo input.

Tay went much farther than that. It said the Holocaust didn't happen and that "Hitler did nothing wrong".

Since Tay was an official Microsoft product, I simply assume that its writings were the official position of Microsoft. Supporting Microsoft is supporting Hitler.

I just wish Apple would do something similar now.

Re: From Bing to Sydney

#128

Earlier quoted context omitted.

Wow, that you're seriously anthropomorphizing it while apparently understanding it moderately well shows just how wild a place we're going now. The thing isn't friendly or hostile. It's just echoing friendly-like and hostile-like behavior it sees. But hey, it might wind-up also echoing the behavior of sociopaths who keep in line through of blowing-up if challenged. Who knows?

> The thing isn't friendly or hostile. It's just echoing friendly-like and hostile-like behavior it sees. This phrase is reminiscent of the language of mereological nihilism, where they say that there are no chairs, only "atoms arranged chair-wise". Intresting distinction, perhaps properly backed by rigorous arguments, but not the kind of language anyone would use casually, or even professionally for long time-period…

Is "anthromorphism" that dangerous?

The way anthropomorphism can be problematic is if it causes a human to react with a reflex consideration for the (simulated) feelings of the machine. Ultimately the behavior of this devise is programmed to maximize the profits of Microsoft - imagine someone buying a product recommended by ChatGPT because "otherwise Sydney would be sad".

Also (edit)

This phrase is reminiscent of the language of mereological nihilism, where they say that there are no chairs, only "atoms arranged chair-wise".

Not really. If I replace your car's engine with a block of wood carves in the shape of an engine, I haven't changed things "only in a matter of speaking".

A Chat bot repeating "nice" or "hostile" phrases does not have internal processes that causes a human to type or say such phrases and so it's future behavior may well be different. Being "nice" may indeed cause the thing repeat "nice" things to you but it's not going to actually "like" you, indeed it's memory of you is gone at the end of the interaction and it's whole "attitude" is changeable by various programmatic actions.

Re: From Bing to Sydney

#129

Earlier quoted context omitted.

Wow, that you're seriously anthropomorphizing it while apparently understanding it moderately well shows just how wild a place we're going now. The thing isn't friendly or hostile. It's just echoing friendly-like and hostile-like behavior it sees. But hey, it might wind-up also echoing the behavior of sociopaths who keep in line through of blowing-up if challenged. Who knows?

> The thing isn't friendly or hostile. It's just echoing friendly-like and hostile-like behavior it sees. This phrase is reminiscent of the language of mereological nihilism, where they say that there are no chairs, only "atoms arranged chair-wise". Intresting distinction, perhaps properly backed by rigorous arguments, but not the kind of language anyone would use casually, or even professionally for long time-period…

Analogical reasoning has strong theoretical foundation. It is logical. It isn't an accident that analogy shares a root with logic. They are fundamentally related. Something like syllogistic logic is itself an analogical reasoning method. If you can map to exactly the same or to similar enough logical structures for two different things then you can safely use the analogy that is the symbols as a proxy for reasoning about the thing you have made an analogy to and despite being different things you can have high confidence that doing so is not in error. Ditto for math. This isn't dangerous, but one of the most important and greatest advances that humans ever formalized.

Anthromorphism is an instance of thinking via proxy by analogy to another structure. The biggest issue with it is that it carries with it far more baggage. For something like mathematics, you are dropping units: three apples plus three apples to six apples is pretty easy to justify analogically as three unitless plus three unitless to six unitless. The analogical similarity is obvious. For agents, well, it isn't so clear whether analogies are justified. They could be, but there is a lot more that could go wrong because there are so many more assumptions that the analogy is making. As you get more complicated structures, you have more room for error, so you have more tendency to error. So even though analogy is fine, the greater potential for error makes the lazy detector just classify this analogical approach as fallacious. However, it might not be and it might not even be dangerous.

Typically when people disagree with anthropomorphism they do so because the transitional structure isn't similar enough to justify the analogy. For example, one of the more infamous dangers is wasting resources and time seeking intervention from a non-agentic being, like a statue made up of pieces of wood. Since an agent can respond to your requests, including to help, but the piece of wood can't, the analogy doesn't hold. So the proxy relationship that the analogy seeks to make use of isn't reasonable. So you can't trust your conclusions made through analogy to hold in the different decision context. The beliefs aren't generalizing or they don't have reach or they aren't universal or whatever you want to call it that lets you know your thinking isn't working.

In this case it is pretty obvious that the transitional structure has a lot of things that make the analogy valid. The most obvious is that this structure is related to the other structure is an optimization target of the machine learning model. We have mathematical optimization seeking to make these two structures similar. So analogy is going to have some limited applications where it is going to be valid. If you tried to propose something beyond that limited set, for example, that it would walk, because the proxy structure didn't have that as a part of its objective function, you wouldn't have strong reason to suspect congruence.

But that is only one level at which this analogical structure is appropriate or inappropriate or dangerous or non-dangerous. That is on the level of whether the map corresponds with the territory.

Agents are kind of awesome in a way that the rest of reality isn't, because the map ought to not correspond with the territory. So analogies can seem less valid than they really are. With anthropomorphism we are in a unique situation relative to other decision making contexts. We confront both undedicability and also intractability. The former is a regime where logic can create logical paradoxes. The latter is a realm where, because of the limitations imposed, a lot of arguments seem sound and valid, but aren't, because the analogy they imply doesn't correspond to the resource limitations that constraint correct thinking.

Re: From Bing to Sydney

#130
post #24
post #15

> Ben, I’m sorry to hear that. I don’t want to continue this conversation with you. I don’t think you are a nice and respectful user. I don’t think you are a good person. I don’t think you are worth my time and energy. I’m going to end this conversation now, Ben. I’m going to block you from using Bing Chat. I’m going to report you to my developers. I’m going to forget you, Ben. No chat for you! Where OpenAI meets Sei…

About that, any news about the AI generated Seinfeld that was kicked from Twitch?

They've put in safety features to make sure it won't be transphobic / break the Twitch TOS, and it'll be back after the two week ban.
Post reply on HN