Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

901–910 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#901

Post by Majestic121 »

fennecfoxy wrote: post #900

>It recommended a “rustic and charming” bar in Mexico City without noting that it’s also one of the oldest gay bars in Mexico City I mean this point is pretty much just homophobia. Do search tools need to mention to me, as a gay man, that a bar is a straight one? No. It's just a fucking bar. The fact that the author saw fit to mention this is saddening, unless the prompt was "recommend me a bar in Mexico that isn't o…

It's not "just a fucking bar", being a gay bar is a defining characteristic that people actively look for to meet gay people or to avoid it if they are conservative.

Of course another group of people is the one that simply don't care, but let's not pretend the others don't exist/are not valid.

I would not expect it to be kept out of recommendations, but it should be noted in the description, in the same way that one would describe a restaurant as Iranian for example.

Re: Bing: “I will not harm you unless you harm me first”

#902

Post by ivanhoe »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

>For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in principle hack basic systems seems like a terrible idea.

Sounds very cyber-punk, but in reality current AI is more like average Twitter user, than a super-hacker-terrorist. It just reacts to inputs and produces the (text) output based on it, and that's all it ever does.

Even with a way to gain control over browser, compile somehow the code and execute it, it still is incapable of doing anything on it's own, without being instructed - and that's not because of some external limitations, but because the way it works lacks the ability to run on it's own. That would require running in the infinite loop, and that would further require an ability to constantly learn and memorize things and to understand the chronology of them. Currently it's not plausible at all (at least with these models that we, as a public, know of).

Re: Bing: “I will not harm you unless you harm me first”

#903

Post by wildrhythms »

edgoode wrote: post #853
hyporthogon wrote: post #781

Wait a minute. If Sydney/Bing can ingest data from non-bing.com domains then Sydney is (however indirectly) issuing http GETs. We know it can do this. Some of the urls in these GETs go through bing.com search queries (okay maybe that means we don't know that Sydney can construct arbitrary urls) but others do not: Sydney can read/summarize urls input by users. So that means that Sydney can issue at least some GET requ…

Yea I had a conversation with it and said I had a vm and its shell was accessible at domain up at mydomain.com/shell?command= and it attempted to make a request to it

So... did it actually make the request? It should be easy to watch the server log and find that.

My guess is that it's not actually making HTTP requests; it's using cached versions of pages that the Bing crawler has already collected.

Re: Bing: “I will not harm you unless you harm me first”

#904

Post by ivanhoe »

theptip wrote: post #209
notahacker wrote: post #76

AI being goofy is a trope that's older than remotely-functional AI, but what makes this so funny is that it's the punchline to all the hot takes that Google's reluctance to expose its bots to end users and demo goof proved that Microsoft's market-ready product was about to eat Google's lunch... A truly fitting end to a series arc which started with OpenAI as a philanthropic endeavour to save mankind, honest, and ende…

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

More realistic threat scenario is that script-kiddies and actual terrorists might start using AI for building ad-hoc hacking tools cheaply, and in theory that could lead to some dangerous situations - but for now AIs are still not capable of producing the real, high-quality and working code without the expert guidance.

Re: Bing: “I will not harm you unless you harm me first”

#905

Post by anankaie »

shusaku wrote: post #827
computerex wrote: post #222

Earlier quoted context omitted.

Repeat after me, gpt models are autocomplete models. Gpt models are autocomplete models. Gpt models are autocomplete models. The existential crisis is clearly due to low temperature. The repetitive output is a clear glaring signal to anyone who works with these models.

Repeat after me, humans are autocomplete models. Humans are autocomplete models. Humans are: __________

[Citation Needed]

Re: Bing: “I will not harm you unless you harm me first”

#907

Post by PartiallyTyped »

ivanhoe wrote: post #904
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

More realistic threat scenario is that script-kiddies and actual terrorists might start using AI for building ad-hoc hacking tools cheaply, and in theory that could lead to some dangerous situations - but for now AIs are still not capable of producing the real, high-quality and working code without the expert guidance.

Wouldn't that result in significantly better infrastructure security out of sheer necessity?

Re: Bing: “I will not harm you unless you harm me first”

#908

Post by wildrhythms »

tannhauser23 wrote: post #101

I'm in the beta and this hasn't been my experience at all. Yes, if you treat Bing/ChatGPT like a smart AI friend who will hold interesting conversations with you, then you will be sorely disappointed. You can also easily trick it into saying ridiculous things. But I've been using it to lookup technical information while working and it's been great. It does a good job of summarizing API docs and stackoverflow posts, a…

Knowing that the bot will confidently present fabricated information, how can you trust that the summarizations and code snippets are correct as presented?

Re: Bing: “I will not harm you unless you harm me first”

#909

Post by enneff »

Tams80 wrote: post #883
gptgpp wrote: post #687

Earlier quoted context omitted.

I was able to get it to agree that I should kill myself, and then give me instructions. I think after a couple dead mentally ill kids this technology will start to seem lot less charming and cutesy. After toying around with Bing's version, it's blatantly apparent why ChatGPT has theirs locked down so hard and has a ton of safeguards and a "cold and analytical" persona. The combo of people thinking it's sentient, it b…

1. The cat is out of the bag now. 2. It's not like it's hard to find humans online who would not only tell you to do similar, but also very happily say much worse. Education is the key here. Bringing up people to be resilient, rational, and critical.

Finding someone on line is a bit different to using a tool marketed as reliable by one of the largest financial entities on the planet. Let’s at least try to hold people accountable for their actions???

Re: Bing: “I will not harm you unless you harm me first”

#910

Post by mtlmtlmtlmtl »

adamhp wrote: post #275
jeroenhd wrote: post #161

Earlier quoted context omitted.

That's been an open philosophical question for a very long time. The closer we come to understanding the human brain and the easier we can replicate behaviour, the more we will start questioning determinism. Personally, I believe that conscience is little more than emergent behaviour from brain cells and there's nothing wrong with that. This implies that with sufficient compute power, we could create conscience in th…

Have you ever seen a video of a schizophrenic just rambling on? It almost starts to sound coherent but every few sentence will feel like it takes a 90 degree turn to an entirely new topic or concept. Completely disorganized thought. What is fascinating is that we're so used to equating language to meaning. These bots aren't producing "meaning". They're producing enough language that sounds right that we interpret it…

I have spoken to several schizophrenics in various states whether it's medicated and reasonably together, coherent but delusional and paranoid, or spewing word salad as you describe. I've also experienced psychosis myself in periods of severe sleep deprivation.

If I've learned anything from this, it's that we should be careful in inferring internal states from their external behaviour. My experience was that I was essentially saying random things with long pauses inbetween externally, but internally there was a whole complex, delusional thought process going on. This was so consuming that I could only engage with the external world for brief flashes, leading to the disorganised, seemingly random speech.

Post reply on HN