Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eigenmorality

scottaaronson.com

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Eigenmorality

#51

Post by gone35 »

Eliezer wrote: post #33
mrb wrote: post #31

Wow: "The mathematical verdict of both eigenmoses and eigenjesus is unequivocal: the 98% are almost perfectly good, while the 2% are almost perfectly evil." The author says this diverge violently from most people’s moral intuitions, but actually this result is PRECISELY what moral relativism predicts. See, there are 2 school of thoughts attempting to explain where morality comes from: - either morality is an absolute…

That's... kinda not a very good description of the major contemporary schools of thought on metaethics. I don't know if any respected analytic philosophers take moral relativism seriously as philosophy; you can't ground the foundational meaning of the word 'good' as 'different cultures think different things are good', since there's no base case for the recursion. Well-known mainstream positions in metaethics hold th…

I don't know if any respected analytic philosophers take moral relativism seriously (...)

Well, many contemporary metaethicists argue that forms of moral relativism undergird / best justify non-cognivitism [1,2], for one. Also Gilbert Harman [3] and David Wong [4] have proposed that forms of moral relativism are associated with naturalism (!), and their work overall is an excellent reference I strongly recommend you check out.

[1] http://stanford.library.usyd.edu.au/entries/moral-relativism...

[2] http://stanford.library.usyd.edu.au/entries/moral-cognitivis...

[3] http://en.wikipedia.org/wiki/Gilbert_Harman

[4] http://en.wikipedia.org/wiki/David_Wong_(philosopher)

Re: Eigenmorality

#52

Post by cma »

gjm11 wrote: post #34
cma wrote: post #20

Needy babies are moral monsters according to many of these models..

Needy babies are also stupid according to many notions of intelligence, ugly according to many notions of beauty, etc. That doesn't mean there's anything wrong with those notions; it means they aren't designed for assessing babies.

And cripples and elderly.

Re: Eigenmorality

#53

Post by prestadige »

sage_joch wrote: post #37
JackFr wrote: post #18

Earlier quoted context omitted.

To each his own. I actually think its overly long, wordy, rambling, poorly conceived and short any novel or important ideas.

He is exploring how the Internet might be used to save civilization. What constitutes an "important idea" in your book?

Every tyranny is predicated on saving civilization.

Also, every idea that actually helps civilisation is incubated in a tiny minority (perhaps in just one mind). Since that minority is engaged in creative work, it is almost certainly an out-group. Adopting the morality of the ruling class and building connections with it are the surest way to power. But these are a full-time job.

I think the idea of quantifying morality might be improved by basing it not on cooperation but simply upon communication, e.g. how well do you know the opinions of those you disagree with? Note that this is almost the opposite approach of the path to power.

Re: Eigenmorality

#54

Post by jessaustin »

aruggirello wrote: post #42
spiritplumber wrote: post #23

Earlier quoted context omitted.

Occasionally "being nice" (cooperating when tit-for-tat suggests defection) dampens avalanche effects caused by defection-happy actors; it prevents the "an eye for an eye makes everyone blind" ending. In that sense, it has value.

Thus forgiving acts much like error correction in a data stream - by preventing propagation of defects, it limits damage.

Forgiveness also has great psychological benefits for those who practice it. In some cases people whose lives have been focused for years on some great wrong that was done to them, have only been able to reach their own personal goals after forgiving the offending party.

Of course vengeance can have psychological benefits too.

Re: Eigenmorality

#55

Post by jessaustin »

MichaelDickens wrote: post #14

This seems related to the idea of coherent extrapolated volition ( https://intelligence.org/files/CEV.pdf ). Both have some of the same problems--in particular, setting up the system requires making moral judgments about how to do so, so it's not actually value-neutral. (Aside: If I have two completely different thoughts about an article, should I post them in two separate comments or in the same comment?)

(My preference is two separate comments. Better threads, better ordered.)

Re: Eigenmorality

#56

Post by oofabz »

lohankin wrote: post #28

I was following Scott's posts for a while. Most notable feature of those posts: everything he says is predictable. Blog is designed to appeal to the liberal academic establishment, which knows answers to all important questions, and is never in doubt. I don't remember a single example of Scott's opinion which could be deemed controversial in any sense. "Eigenconformism" would be a better name for his blog.

I don't think the blog is consciously designed to appear to liberal academics. As an MIT professor, Aaronson IS the liberal academic establishment, so it is no mystery that his writing appeals to his peers.

I don't know about you, but I'm willing to admit Aaronson knows more answers to important questions than I do.

Re: Eigenmorality

#57

Post by argc »

JackFr wrote: post #18
knowtheory wrote: post #2

This is very long but worth reading. The modeling exercise herein is basically attempting to use a game theoretic model to test out some really dumb/simplified models of cooperation and whether the behaviors observed approximate anything approaching what our intuitions might say is moral behavior, up to and including an 'eigenjesus' and 'eigenmoses' up against tit_for_tat bots and the like.

To each his own. I actually think its overly long, wordy, rambling, poorly conceived and short any novel or important ideas.

Perhaps you have studied morality simulations in the past, whereas I have not. It was written simply. (In my opinion the occasional wordiness was a stylistic choice to mimic casual conversation rather than a strictly rigorous academic paper.. fitting for a blog post.) I found it a though-provoking read, and I have no qualms if his ideas were not "novel". Digesting old ideas and writing about them is not a crime. If anything, it is beneficial to those who did not read the original work (like me). As you said... to each his own.

Re: Eigenmorality

#58

Post by oofabz »

yason wrote: post #38

There is no right or wrong, just acts with unescapable consequences and your freedom to learn something from your choices.

The concepts of right and wrong certainly exist, or we couldn't be talking about them. But not everyone agrees what is right and what is wrong.

It sounds like you are saying that there is no absolute right or wrong, that right and wrong are human inventions prone to variation, not some fixed celestial law. That is exactly the stance which Aaronson took in his essay so I believe you two agree on that point.

Re: Eigenmorality

#59

Post by cscurmudgeon »

argc wrote: post #57
JackFr wrote: post #18

Earlier quoted context omitted.

To each his own. I actually think its overly long, wordy, rambling, poorly conceived and short any novel or important ideas.

Perhaps you have studied morality simulations in the past, whereas I have not. It was written simply. (In my opinion the occasional wordiness was a stylistic choice to mimic casual conversation rather than a strictly rigorous academic paper.. fitting for a blog post.) I found it a though-provoking read, and I have no qualms if his ideas were not "novel". Digesting old ideas and writing about them is not a crime. If a…

That is the problem here. For anyone who has studied ethics and morality, this is hair-pullingly bad. It is not easy to communicate these ideas.

An analogy: Would you trust a physicist who does not communicate often, or a creationist who writes popular science essays?

Re: Eigenmorality

#60

Post by sage_joch »

prestadige wrote: post #53
sage_joch wrote: post #37

Earlier quoted context omitted.

He is exploring how the Internet might be used to save civilization. What constitutes an "important idea" in your book?

Every tyranny is predicated on saving civilization. Also, every idea that actually helps civilisation is incubated in a tiny minority (perhaps in just one mind). Since that minority is engaged in creative work, it is almost certainly an out-group. Adopting the morality of the ruling class and building connections with it are the surest way to power. But these are a full-time job. I think the idea of quantifying moral…

I don't think "tyranny of the majority" applies here. The proposed system makes minority opinions more visible, if anything. There would even be an incentive to have a minority opinion, if you truly believed the majority was incorrect about something. In response to your last point: that sounds like an interesting modification: letting every bot see every other bot's (possibly evolving) code. But perhaps to avoid Skynet, bots should use the other bots' published APIs (which could opt to include a "getCode" method), and judge each other by their actions.
Post reply on HN