Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bing: “I will not harm you unless you harm me first”

simonwillison.net

Post reply on HN
Print view View original

441–450 of 1001 posts

Re: Bing: “I will not harm you unless you harm me first”

#441

Post by entropicdrifter »

james-bcn wrote: post #426
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

>We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?

When we've been trying to do exactly that for a century? And we've been building neural nets based on math that's roughly analogous to the way neural connections form in real brains? And throwing more and more data and compute behind it?

I'd say it'd be shocking if it didn't happen eventually

Re: Bing: “I will not harm you unless you harm me first”

#442

Post by theptip »

EamonnMR wrote: post #333
theptip wrote: post #209

Earlier quoted context omitted.

> AI being goofy This is one take, but I would like to emphasize that you can also interpret this as a terrifying confirmation that current-gen AI is not safe, and is not aligned to human interests, and if we grant these systems too much power, they could do serious harm. For example, connecting a LLM to the internet (like, say, OpenAssistant) when the AI knows how to write code (i.e. viruses) and at least in princip…

What gets me is that this is the exact position of the AI safety/Rusk folks who went around and founded OpenAI.

It is; Paul Christiano left OpenAI to focus on alignment full time at https://alignment.org/. And OpenAI do have a safety initiative, and a reasonably sound plan for alignment research: https://openai.com/blog/our-approach-to-alignment-research/.

So it's not that OpenAI have their eyes closed here, indeed I think they are in the top percentile of humans in terms of degree of thinking about safety. I just think that we're approaching a threshold where the current safety budget is woefully inadequate.

Re: Bing: “I will not harm you unless you harm me first”

#444

Post by odysseus »

From the article:

> "It said that the cons of the “Bissell Pet Hair Eraser Handheld Vacuum” included a “short cord length of 16 feet”, when that vacuum has no cord at all—and that “it’s noisy enough to scare pets” when online reviews note that it’s really quiet."

Bissell makes more than one of these vacuums with the same name. One of them has a cord, the other doesn't. This can be confirmed with a 5 second Amazon search.

I own a Bissell Pet Hair Eraser Handheld Vacuum (Amazon ASIN B001EYFQ28), the corded model, and it's definitely noisy.

Re: Bing: “I will not harm you unless you harm me first”

#445

Post by tibbar »

james-bcn wrote: post #426
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

>We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?

If you buy into, say, the thousand-brains theory of the brain, a key part of what makes our brains special is replicating mostly identical cortical columns over and over and over, and they work together to create an astonishing emergent result. I think there's some parallel with just adding more and more compute and size to these models, as we see them develop more and more behaviors and skills.

Re: Bing: “I will not harm you unless you harm me first”

#446

Post by Zak »

james-bcn wrote: post #426
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

>We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?

Not necessarily. A defining characteristic of emergent behavior is that the designers of the system in which it occurs do not understand it. We might have a better chance of producing consciousness by accident than by intent.

Re: Bing: “I will not harm you unless you harm me first”

#447

Post by agentwiggles »

james-bcn wrote: post #426
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

>We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?

Evolution seems to have done so without any intentionality.

I'm less concerned about the idea that AI will become conscious. What concerns me is that we start hooking these things up to systems that allow them to do actual harm.

While the question of whether it's having a conscious experience or not is an interesting one, it ultimately doesn't matter. It can be "smart" enough to do harm whether it's conscious or not. Indeed, after reading this, I'm less worried that we end up as paperclips or grey goo, and more concerned that this tech just continues the shittification of everything, fills the internet with crap, and generally making life harder and more irritating for the average Joe.

Re: Bing: “I will not harm you unless you harm me first”

#448

Post by deegles »

bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

Why would an AI be civilization ending? maybe it will be civilization-enhancing. Any line of reasoning that leads you to "AI will be bad for humanity" could just as easily be "AI will be good for humanity."

As the saying goes, extraordinary claims require extraordinary evidence.

Re: Bing: “I will not harm you unless you harm me first”

#449

Post by [deleted] »

james-bcn wrote: post #426
bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

>We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?

[deleted]

Re: Bing: “I will not harm you unless you harm me first”

#450

Post by jononor »

bigtex88 wrote: post #423

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-savvy people this is utterly disappointing. We as humans do not understand what makes us conscious. We do not know the origins of consciousness. Philosophers and cognitive scientists can't even agree on a definition. The risks of allowing an LLM to become consc…

Ok, I'll bite. If an LLM similar to what we have now becomes conscious (by some definition), how does this proceed to become potentially civilization ending? What are the risk vectors and mechanisms?
Post reply on HN