Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clarity didn't work, trying mysterianism (2012)

gwern.net

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Clarity didn't work, trying mysterianism (2012)

#31

Post by Ari_Rahikkala »

fourthark wrote: post #8
beepbooptheory wrote: post #5

This came up the other day, can someone help explain this to me? I am not quite sharp enough to understand it. Context: https://news.ycombinator.com/item?id=48911884

Seems like a fairly direct parable against AI, suggesting that even if it's right it's better to think for yourself.

I'm a big fan of Scott's, but I haven't been happy to see how this story's failure to say what it means has in some ways only made it less relevant over the years as we've come closer to building something like the item that, on the surface, it talks about.

Not to say I think it's a bad thing at all that people have a name for this concern for giving up agency to an external system, even when that system really has your best intent in mind. You can call Claude a whispering earring and people will understand what you mean pretty well. The actual consequences you're looking out for are maybe a bit subtler than your brain atrophying, I really really never liked that detail in the story, but that's fine, people will get it. By the Death of the Author, the story's a great contribution as it is.

But it just feels like a loss that there really was a different thing that the author meant, and it got lost in translation. http://web.archive.org/web/20121007235422/http://squid314.li... clarifies it. It's quite similar to a Yudkowskyan fun theory concern (and IMO really not surprising that the story was released a couple of years after the fun theory sequence): If making the right choices and doing the right thing eventually comes down to a mechanical computation, and you've got all of your resource limitations and psychological hang-ups and such things out of the way, then there's no room left to do the right thing, and you've turned yourself into a mechanism.

I don't think that's really a possible state that any system with any trace of humanity left in it will ever enter, there's too many too large search processes in the way to fit in a finite universe. So it's a pretty abstract concern even among its peers. But it's one that's fun to talk about sometimes, and one lesson we've learned from the response to this story is that mysterious, easily misinterpreted parallels in fact do not point people in the right direction to see it.

Re: Clarity didn't work, trying mysterianism (2012)

#32

Post by thinking_cactus »

dudisubekti wrote: post #27
fourthark wrote: post #8

Earlier quoted context omitted.

Seems like a fairly direct parable against AI, suggesting that even if it's right it's better to think for yourself.

Eh I think the writer should visit poor countries and live among locals. Mistakes and poor judgements compound, and it leads to horrific living conditions (malnutrition, nonexistent hygiene, quack medicines, short-sighted violence, etc) In fact developed countries are built on top of these "earrings": it's called math (and by extension science in general). If the math is right, it's always right, and it has spared th…

No, this story takes it to the extreme, that you end up basically having no brain.

Take it this way. People seem to judge that LLMs are not conscious and thus have no real inner experience. And that humans do. So if you replaced a human brain with a very powerful LLM (with some simply residual neural matter), you've basically killed the human. He isn't feeling anything at all anymore, he's just an automaton, basically a flesh robot.

I think this story is a weirdly good discussion point because many people still don't seem to "get it" that success is completely fundamentally irrelevant, and what matters is having a good inner life. You are what happens in your mind -- overall good thoughts and inner life = good life; bad thoughts and inner life = bad life. You can theoretically be the richest man in the world and about the most miserable, and very poor and living the best life.

If we don't kill our definition of good life and success (i.e. reaffirming value as pertaining exclusively to the mind), it will sooner or later kill us, or kill what's good in us. The more we can orient our entire society toward good inner lives and not anything else, the better our lives will be.

Re: Clarity didn't work, trying mysterianism (2012)

#33

Post by NooneAtAll3 »

ccapitalK wrote: post #25

I wonder if there is a self consistent way to desire having your mind digitally uploaded into a computer, while detesting the idea of listening to the commands of the earring (and conditioning it to have all of my unconcious thoughts and desires). I guess I just discovered a bit of cognitive dissonance I didn't know I had.

mind upload is closer to the version in the comments, with gem learning to emulate brain and then brain gets removed

the difference is in targetted influence from outside aimed at changing you for its own goals and final form

consider the "accelerated version" of this earpiece - where you just get whackked with dose of heroine (making you "perpetually happy") and then taking over your body

Re: Clarity didn't work, trying mysterianism (2012)

#34

Post by Mikhail_Edoshin »

This is a good companion to the yesterday's HN post of ten steps to happiness.

I've read that Malachi Martin described what he called "a perfect possession". I do not know where as I haven't found this piece myself. But the idea was that typical cases of demonic possession are imperfect, the victim is struggling, and the struggle is visible to the outsiders. A perfect possession does not manifest itself.

Re: Clarity didn't work, trying mysterianism (2012)

#36

Post by Erem »

CamperBob2 wrote: post #12

I don't get it. First, how is it "better for you" to take the earring off, if "the wearer lives an abnormally successful life, usually ending out as a rich and much-beloved pillar of the community with a large and happy family?" Second, apparently some people do choose to take the earring off at some point, because the story refers to "the testimony of all living wearers." So it does not, in fact, appear to usurp, su…

Because, if you don't take it off, "you" will cease to exist and instead gradually become the rich and much beloved appendage of an earring.

The entire story is about the definition of self imho.

Re: Clarity didn't work, trying mysterianism (2012)

#37

Post by andai »

ccapitalK wrote: post #25

I wonder if there is a self consistent way to desire having your mind digitally uploaded into a computer, while detesting the idea of listening to the commands of the earring (and conditioning it to have all of my unconcious thoughts and desires). I guess I just discovered a bit of cognitive dissonance I didn't know I had.

I'll still be able to take my advice from Math.random()

Re: Clarity didn't work, trying mysterianism (2012)

#39

Post by ameliaquining »

beepbooptheory wrote: post #5

This came up the other day, can someone help explain this to me? I am not quite sharp enough to understand it. Context: https://news.ycombinator.com/item?id=48911884

The author explained (his view of) the story's meaning in a follow-up post, aptly titled "Mysterianism didn't work either, trying clarity again": https://web.archive.org/web/20121007235422/http://squid314.l...

Re: Clarity didn't work, trying mysterianism (2012)

#40

Post by goatlover »

citruscomputing wrote: post #35

I've been thinking about this story a lot lately. When giving away your agency gives strictly better results, should you? And if you shouldn't, do you really think you're strong enough to resist it?

Would you give away your agency to another human or human organization if they gave strictly better results? My guess is you value your own agency over your life more than that. It's your life to make mistakes and hopefully learn from (but that's up to you), not somebody else's to live for you.
Post reply on HN