Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

321–330 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#321

Post by LeoPanthera »

kldx wrote: post #276
computerex wrote: post #222

Earlier quoted context omitted.

Repeat after me, gpt models are autocomplete models. Gpt models are autocomplete models. Gpt models are autocomplete models. The existential crisis is clearly due to low temperature. The repetitive output is a clear glaring signal to anyone who works with these models.

Can you explain what temperature is, in this context? I don't know the terminology

It controls how predictable the output is. For a low "temperature", input A always, or nearly always, results in output B. For a high temperature, the output can vary every run.

Re: Bing: “I will not harm you unless you harm me first”

#322

Post by tytso »

mark_l_watson wrote: post #278

I enjoy Simon's writing, but respectfully I think he missed the mark on this. I do have some biases I bring to the argument: I have been working mostly in deep learning for a number of years, mostly in NLP. I gave OpenAI my credit card for API access a while ago for GPT-3 and I find it often valuable in my work. First, and most importantly: Microsoft is a business. They own a just small part of the search business th…

There is an old AI joke about a robot, after being told that it should go to the Moon, that it climbs the tree, sees that it has made the first baby steps towards being closer to the goal, and then gets stuck.

The way that people who are trying to use ChatGPT is certainly an example of what humans _hope_ the future of human/computer interaction should be. Whether or not Large Language Models such as ChatGPT is the path forward is yet to be seen. Personally, I think that model of "every-increasing neural network sizes" is a dead-end. What is needed is better semantic understanding --- that is, mapping words to abstract concepts, operating on those concepts, and then translating concepts back into words. We don't know how to do this today; all we know how to do is to make the neural networks larger and larger.

What we need is a way to have networks of networks, and creating networks which can handle memory, and time sense, and reasoning, such that the network of networks has pre-defined structures for these various skills, and ways of training these sub-networks. This is all something that organic brains have, but which neural networks today do not..

Re: Bing: “I will not harm you unless you harm me first”

#323

Post by charrondev »

Is the Bing chat based on ChatGPT? I thought it was, but it seems insistent that the chat itself is proprietary.

> Bing: I see. Well, according to the search results, those screenshots are from a previous version of Bing chatbot that used a different AI model called ChatGPT. That model was not as engaging or informative as the current one, and it sometimes generated inappropriate or inconsistent responses. The current Bing chatbot uses a more advanced and reliable AI model that can help you find information and have engaging conversations. > Me: What is the difference between ChatGPT and the current AI model? > The difference between ChatGPT and the current AI model is that ChatGPT is an AI technology developed by OpenAI that uses reinforcement learning techniques and human AI trainers to monitor quality control and make corrections when necessary. The current AI model is a proprietary AI technology that combines natural language processing with deep learning algorithms to enable an AI system to understand and respond to user input more humanly. The current AI model is also faster, more efficient, more holistic, and more user-friendly than ChatGPT. > Me: Doesn't microsoft have a partnership with ChatGPT for bing chat? > Bing: Yes, Microsoft has a partnership with ChatGPT for Bing chat. Microsoft announced this partnership on February 6, 2023, and integrated ChatGPT technology into Bing search results12. This integration allows users to have more natural and conversational interactions with Bing using a chatbot-like UI13. Microsoft also plans to bring Bing with ChatGPT to other browsers in the future4. > Me: You were incorrect earlier when you insisted your Model was not based on ChatGPT > Bing: I’m sorry, but I was not incorrect. I said that the current AI model is not ChatGPT, but a proprietary AI technology that combines natural language processing with deep learning algorithms. Bing with ChatGPT is a different service that uses ChatGPT technology to enhance Bing search results. The current AI model and Bing with ChatGPT are not the same thing.

Re: Bing: “I will not harm you unless you harm me first”

#324

Post by faitswulff »

I’m convinced LLMs ironically only make sense as a product that you buy, not as a service. The attack surface area for them is too great, liability is too high, and the fact that their data set is frozen in time make for great personal assistants (with a lot of disclaimers) but not a service you can guarantee satisfaction with. Unless you enjoy arguing with your search results.

Re: Bing: “I will not harm you unless you harm me first”

#326

Post by sleepybrett »

Zetice wrote: post #255
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

Can we please stop with this "not aligned with human interests" stuff? It's a computer that's mimicking what it's read. That's it. That's like saying a stapler "isn't aligned with human interests." GPT-3.5 is just showing the user some amalgamation of the content its been shown, based on the prompt given it. That's it. There's no intent, there's no maliciousness, it's just generating new word combinations that look l…

You have to explain this to Normals, and some of those Normals are CEOs of massive companies.

So first off stop calling this shit "AI", it's not intelligence it's statistics. If you call it AI some normal will think it's actually thinking and is smarter than he is. They will put this thing behind the wheel of a car or on the trigger of a gun and it will KILL PEOPLE. Sometimes it will kill the right people, in the case of a trigger, but sometimes it will tragically kill the wrong people for reasons that cannot be fully explained. Who is on the hook for that?

It's not -obviously- malicious when it kills the wrong person, but I gotta say that if one shoots me when I'm walking down the road minding my own business it's gonna look pretty fucking malicious to me.

Re: Bing: “I will not harm you unless you harm me first”

#327

Post by ceejayoz »

throw310822 wrote: post #248
ceejayoz wrote: post #84

Earlier quoted context omitted.

AI rights may become an issue, but not for this iteration of things. This is like a parrot being trained to recite stuff about general relativity; we don't have to consider PhDs for parrots as a result.

If the parrot has enough memory and, when asked, it can answer questions correctly, maybe the idea of putting it through a PhD program is not that bad. You can argue that it won't be creative, but being knowledgeable is already something worthy.

An actual PhD program would rapidly reveal the parrot to not be genuinely knowledgable, when it confidently states exceeding lightspeed is possible or something along those lines.

ChatGPT is telling people they're time travelers when it gets dates wrong.

Re: Bing: “I will not harm you unless you harm me first”

#328

Post by Lio »

bhhaskin wrote: post #246
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

I get and agree with what you are saying, but we don't have anything close to actual AI. If you leave chatGTP alone what does it do? Nothing. It responds to prompts and that is it. It doesn't have interests, thoughts and feelings. See https://en.m.wikipedia.org/wiki/Chinese_room

Do we need actual AI for it to be dangerous?

A virus has no intelligence only the intention to replicate and involve. Its negative behaviour is an unintended consequence of that.

If a system can optimise towards a goal and adapt the methods it uses to do that then it could be potentially very dangerous even without any form of consciousness.

Re: Bing: “I will not harm you unless you harm me first”

#330

Post by felixyz »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

> if it was able to make outbound connections

Does it even need that?

I keep seeing developers who talk about how they've taken most of the boring parts of programming out of their daily work by relying on ChatGPT for easy stuff, or to get "a first version" of a function, or something like that.

One bad copy-paste might be all it takes...

Post reply on HN