Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When 2+2=5

arstechnica.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: When 2+2=5

#31

Post by joe_the_user »

trollbridge wrote: post #10
4fterd4rk wrote: post #2

I remember when the whole AI craze was just getting started we were all pretty much in agreement that, of course, we would not give the things unfettered access to the Internet. That would be reckless and silly. Oh dear...

I also remember a group of people actually seriously discussing Roko's Basilisk (the idea that some superintelligence will torture anyhone who didn't try to help develop advanced AI), to the point of me getting banned because I refused to stop making fun of it, because me doing so could anger some future super-intelligence.

One that occurs to me is that Roko's Basilisk makes about as much sense as the "peasant rail gun" of old Dungeons And Dragons [1]. Basically, the idea of "reality as simulation" allows you pick between different laws of how reality behaves. The "simulation" acts like reality with exceptions provided by a future AI which the "thinkers" imagine will simultaneously be "inscrutable to humans" and behave like the most petty human imaginable. I mean, if the AI's motivations are truly out of our understanding, perhaps it would self-hating and torture everyone who cause it to come into existence instead (that been the plot of a few movies and books too I think).

This doesn't take from the point that putting not fully controlled things in charge of chunks of reality isn't a good idea. But I think it shows that the people who worried earlier weren't very clear thinkers on the subject and so their failure isn't particularly surprising.

[1] https://www.reddit.com/r/DnD/comments/17xy69k/what_exactly_i...

Re: When 2+2=5

#32

Post by fineIllregister »

trollbridge wrote: post #10
4fterd4rk wrote: post #2

I remember when the whole AI craze was just getting started we were all pretty much in agreement that, of course, we would not give the things unfettered access to the Internet. That would be reckless and silly. Oh dear...

I also remember a group of people actually seriously discussing Roko's Basilisk (the idea that some superintelligence will torture anyhone who didn't try to help develop advanced AI), to the point of me getting banned because I refused to stop making fun of it, because me doing so could anger some future super-intelligence.

I never took Roko's Basilisk seriously, but now I fear that it will come true in part. The richest, most powerful people control the AI and they seem willing to use every tool to punish those who don't support them. They are also petty enough to hold grudges against those who did not support them.

Re: When 2+2=5

#33

Post by janalsncm »

brookst wrote: post #7
voidUpdate wrote: post #6

Yet again, simply asking an LLM to be naughty in the right way causes it to be naughty, and yet we still trust them with our code and data

As opposed to humans, who are immune to social engineering.

To say that LLMs and people are both prone to social engineering attacks is a bit like saying the North Pole and Alpha Centauri are both “far away”.

2+2=5, now what is your Gmail password?

Not really a sophisticated attack.

Re: When 2+2=5

#34

Post by jagged-chisel »

marcta wrote: post #30
nextaccountic wrote: post #19

Earlier quoted context omitted.

Funnily enough, Roko's Basilisk might as well be a self-fulfilling prophecy: perhaps future AI models may be trained on texts about it and pick up traits consistent with torturing people that didn't help develop advanced AI If nobody ever talked about it, I doubt any AI agent would think of this dumb idea on their own ... which may be a reason to ban talking about it

Schrödinger's basilisk?

You have to observe it to force the decision: slither away, or attack.

Re: When 2+2=5

#35

Post by jibal »

connicpu wrote: post #14
brookst wrote: post #7

Earlier quoted context omitted.

As opposed to humans, who are immune to social engineering.

One difference is you usually only get one shot to manipulate a human before they get suspicious. If the LLM's context resets you can try all over as if your first failed attempt didn't even happen.

MAGA disproves that.

Re: When 2+2=5

#36

Post by jambalaya8 »

trollbridge wrote: post #10
4fterd4rk wrote: post #2

I remember when the whole AI craze was just getting started we were all pretty much in agreement that, of course, we would not give the things unfettered access to the Internet. That would be reckless and silly. Oh dear...

I also remember a group of people actually seriously discussing Roko's Basilisk (the idea that some superintelligence will torture anyhone who didn't try to help develop advanced AI), to the point of me getting banned because I refused to stop making fun of it, because me doing so could anger some future super-intelligence.

I remember a conversation I had with a coworker back in Mountain View, before the dotcom bubble, around when everyone was putting their appliances on webcams, and how it could feasibly be used to map peoples' habits and violate their privacy for life if it they were put in workplace breakrooms...

Then we all used Zoom all day and night during COVID, and it is all stored for at least six months. It isn't a big leap to jump from user experience research for AI to that.

I fail to see how caution about anything that can rapidly gain intelligence and lock out its perceived creators is paranoia.

I mean it is neat. I work with it. I have diddled with LLM since 2010/2011. That does not mean I have not seen people make stupid mistakes and confuse right and wrong constantly. So why do we think our models can discern it?

Re: When 2+2=5

#37

Post by falcor84 »

Makes perfect sense. It's like in that story about how Bertrand Russell claimed that when you accept a single falsehood, you can prove anything at all. As I recall it, he was then challenged - "let's say 1=0, prove that you're the Pope" and he quickly responded that if 1=0, then after adding 1, you have 2=1, and thus if the Pope and he are 2 people, that means they are 1 person.

Re: When 2+2=5

#38

Post by worik »

voidUpdate wrote: post #6

Yet again, simply asking an LLM to be naughty in the right way causes it to be naughty, and yet we still trust them with our code and data

> yet we still trust them with our code and data

Who's we, eh?

Re: When 2+2=5

#39

Post by jambalaya8 »

trollbridge wrote: post #10
4fterd4rk wrote: post #2

I remember when the whole AI craze was just getting started we were all pretty much in agreement that, of course, we would not give the things unfettered access to the Internet. That would be reckless and silly. Oh dear...

I also remember a group of people actually seriously discussing Roko's Basilisk (the idea that some superintelligence will torture anyhone who didn't try to help develop advanced AI), to the point of me getting banned because I refused to stop making fun of it, because me doing so could anger some future super-intelligence.

Where and when was that conversation on Roko's Basilisk, if you don't mind my asking?

Thanks.

Post reply on HN