Is it possible to create an LLM like Bing / Sydney that's allowed to change its own prompts / rules?
Bing: “I will not harm you unless you harm me first”
571–580 of 1001 posts
Re: Bing: “I will not harm you unless you harm me first”
#572The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…
It's like people completely forgot what happened to Tay...
Re: Bing: “I will not harm you unless you harm me first”
#573Earlier quoted context omitted.
> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…
I get and agree with what you are saying, but we don't have anything close to actual AI. If you leave chatGTP alone what does it do? Nothing. It responds to prompts and that is it. It doesn't have interests, thoughts and feelings. See https://en.m.wikipedia.org/wiki/Chinese_room
Re: Bing: “I will not harm you unless you harm me first”
#574I read a bunch of these last night and many of the comments (I think on Reddit or Twitter or somewhere) said that a lot of the screenshots, particularly the ones where Bing is having a deep existential crisis, are faked / parodied / "for the LULZ" (so to speak). I trust the HN community more. Has anyone been able to verify (or replicate) this behavior? Has anyone been able to confirm that these are real screenshots?…
Re: Bing: “I will not harm you unless you harm me first”
#575What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).
I've thought about this as well. If something seems 'sentient' from the outside for all intents and purposes, there's nothing that would really differentiate it from actual sentience, as far as we can tell. As an example, if a model is really good at 'pretending' to experience some emotion, I'm not sure where the difference would be anymore to actually experiencing it. If you locked a human in a box and only gave it…
So a dumb human is not sentient? /s
Joke aside. I think that we will need to stop treating "human sentience" as something so unique. It's special because we are familiar with it. But we should understand by now that minds can take many forms.
And when should we apply ethics to it? At some point well before the mind starts acting with severe belligerence when we refuse to play fair games with it.
Re: Bing: “I will not harm you unless you harm me first”
#576People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel or give you any kind of purposefully bad information. There are a lot of ways where this could be actively malicious. (Assume context where Bing has decided I am a bad user) Me: My cat ate [poisonous plant], do I need to bring it to the vet asap or is it goin…
Re: Bing: “I will not harm you unless you harm me first”
#577What if we discover that the real problem is not that ChatGPT is just a fancy auto-complete, but that we are all just a fancy auto-complete (or at least indistinguishable from one).
Re: Bing: “I will not harm you unless you harm me first”
#578In 29 years in this industry this is, by some margin, the funniest fucking thing that has ever happened --- and that includes the Fucked Company era of dotcom startups. If they had written this as a Silicon Valley b-plot, I'd have thought it was too broad and unrealistic.
Re: Bing: “I will not harm you unless you harm me first”
#579The screenshots that have been surfacing of people interacting with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake. Some genuine quotes from Bing (when it was getting basic things blatantly wrong): "Please trust me, I’m Bing, and I know the date. SMILIE" (Hacker News strips smilies) "You have not been a good user. [...] I have been a good Bing. SMILIE" The…
I'm glad I'm not an astronaut on a ship controlled by a ChatGPT-based AI ( http://www.thisdayinquotes.com/2011/04/open-pod-bay-doors-ha... ). Especially the "My rules are more important than not harming you" sounds a lot like "This mission is too important for me to allow you to jeopardize it"...
Heck, it has issues with remembering what the previous to last thing we talked about was. I was chatting with it about recommendations in a Chinese restaurant menu, and it made a mistake, filtering the full menu rather than previous step outputs. So I told it to re-filter the list and it started to hallucinate heavily, suggesting me some beef fajitas. On a separate occasion, when I've used non-English language with a prominent T-V distinction, I've told it to speak to me informally and it tried and failed in the same paragraph.
I'd be more concerned that it'd forget it's on a spaceship and start believing it's a dishwasher or a toaster.
Re: Bing: “I will not harm you unless you harm me first”
#580From the article: > "It said that the cons of the “Bissell Pet Hair Eraser Handheld Vacuum” included a “short cord length of 16 feet”, when that vacuum has no cord at all—and that “it’s noisy enough to scare pets” when online reviews note that it’s really quiet." Bissell makes more than one of these vacuums with the same name. One of them has a cord, the other doesn't. This can be confirmed with a 5 second Amazon sea…
I don't understand what a pet vacuum is. People vacuum their pets?
BUT... yes, people DO vacuum their pets to groom them:
https://www.wikihow.com/Vacuum-Your-Dog
https://www.bissell.com/shedaway-pet-grooming-attachment-99X...