Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Intelligence Self-Limiting?

ieet.org

Post reply on HN
Print view View original

31–40 of 81 posts

Re: Is Intelligence Self-Limiting?

#31

Post by Androsynth »

dgallagher wrote: post #30
Androsynth wrote: post #22

Iirc, there was an article a while back (or maybe a link from a comment) about how video games were the real reason behind the fermi paradox. The argument was that a civilization's entertainment would constantly be improving to the point where we could jack into a self-controlled matrix-like world where we had complete control and, in essence, are deities in our self-fabricated worlds. If we built robots to ensure ou…

It only takes a tiny percentage to turn down said machine. If 10 billion people plug-in, but 1 million abstain and continue to evolve past the 10 billion, the majority stagnates. Those 1 million will eventually gain a competitive advantage over the 10 billion, evolve past them, and likely extinct them. -------------------- Prediction: Humans won't be around in 1,000 years if technology progresses at current rates. Su…

I completely agree with everything you say. I think that in order to avoid a cambrianesque extinction event within the human race (whether by 99% of the population entering the matrix or simply by the destructive capability of weapons becoming too much), we will need to upgrade ourselves. Most likely by meshing with machine technology.

Re: Is Intelligence Self-Limiting?

#33

Post by cheatercheater »

TLDR: author starts out by comparing the human being to mythical cyber-creatures that never feel pain or hunger, then continues to exploit the comparison by supposing if you do enough coke you can live forever. Finally, humans are described as anarchistic, self-destructive babies in grown up bodies that repeatedly crap themselves without noticing, while inadvertently performing genocide on the whole humanity at once.

Re: Is Intelligence Self-Limiting?

#34

Post by donnawarellp »

I have often thought that life is self limiting. For example, yeast is a living thing, it consumes sugars and it's waste is ehtanol and eventually they all kill themselves off by drowning in their own waste (which, I guess later over a meal I will be enjoying a glass of yeast pee, but I digress). From a molecular perspective, human DNA is not that different from yeast, and at a macro view our behaviour is not that different either it seems. Perhaps all sentient life is destine to the same fate, perhaps it is a law of nature. I recall reading about an argument that Stephen Hawking and Kip Thorne have regarding time travel. Stephen Hawking argues against time travel because we have never met any time travellers from our future. Maybe time travel is theoretically possible, but like yeast, we do not as a species survive long enough to develop the necessary technology. ugh, sorry to bring everyone down, might as well have a glass of yeast pee.

Re: Is Intelligence Self-Limiting?

#35

Post by robertskmiles »

Androsynth wrote: post #28
Androsynth wrote: post #19

Earlier quoted context omitted.

From a systems perspective, a pleasure center in the 'brain' is much more scalable than hardwiring something to 'do more x'. These are semi-sentient beings that he is describing, not dumb robots.

edit: I also want to point out that a murder-happy pill is a straw man argument. It will never happen for two reasons: 1-we have evolved to find murder repugnant, so the rewards wouldn't seem appealing to us 2-there are real life repercussions to murdering someone, so the joys would be limited in their duration. We already have happy medicines that are legal. Valium, alcohol, WoW... I would bet that most people who a…

It's not a straw man, it's a thought experiment. Obviously you can't actually do it. I'm sure you'd have trouble filling a room with rules about how to handle chinese symbols, or teaching a colourblind scientist everything it's possible to know about colour. It's hypothetical, it's a thought experiment.

Your first criticism is just invalid. The pill rewrites the workings of your brain to enjoy murder, the experience of murder and the consequences of murder. The fact that the brain originally evolved to work one way is irrelevant. The hypothetical pill changes the brain in a way just as powerful as the way an AI can rewrite its source code.

Your second criticism doesn't invalidate the experiment unless you claim that if the real life repercussions weren't a factor then you'd happily take the pill. I assume this is not the case.

Given all the facts, you won't choose to change what you fundamentally value, because that change would necessarily go against what you fundamentally value.

Re: Is Intelligence Self-Limiting?

#36

Post by pjscott »

randallsquared wrote: post #17
cousin_it wrote: post #3

Not all possible AI designs are based on reinforcement learning or reward channels. You can code the AI with a formally specified goal instead. For example, "is there a proof of the Riemann hypothesis in ZFC shorter than a million symbols?" If such a goal requires converting the solar system into computronium, then the AI will do that. It won't settle for wireheading because wireheading doesn't give an answer to the…

In fact there's ample reason to think that humans are at the lowest threshold of intelligence that makes a technological civilization possible, because if we'd reached that threshold earlier in our evolution, we'd have created civilization then instead of now. There's probably plenty of room above us. While I don't disagree that humans are essentially at the lowest level of intelligence that make civilization possibl…

Imagine a human genius. Now imagine that same genius with a brain that thinks the same thoughts, but a hundred times faster. Now imagine a million instances of that person.

At no point in this thought experiment have we changed the thoughts this person can think. If any of these sped-up million geniuses took an IQ test or something, the score would be the same. And yet, they would seem superhumanly intelligent by any reasonable definition of the phrase.

I doubt that evolution -- with a crazy biological substrate, no less -- somehow managed to find an upper limit on intelligence.

Re: Is Intelligence Self-Limiting?

#37

Post by carsongross »

robertskmiles wrote: post #4
carsongross wrote: post #2

Reality is self-limiting. There are no exponentials, only sigmoid curves. Unfortunately there is a lot of money to be made convincing people that an early stage sigmoid is actually an exponential.

Obviously intelligence tops out at the high end of a sigmoid curve - there's only so much matter and energy in the universe to convert to computronium - but that's no reason to suspect any value smaller than that as the limit. It must eventually level off, but the point at which it levels off may be hundreds of billions of times more intelligent than a human. So the fact that intelligence is limited, doesn't in any w…

We also have no reason to suspect that it is possible.

The wall that silicon has smashed into should give us (well, not me, really the Singularitists) pause.

Re: Is Intelligence Self-Limiting?

#38

Post by Dn_Ab »

randallsquared wrote: post #17
cousin_it wrote: post #3

Not all possible AI designs are based on reinforcement learning or reward channels. You can code the AI with a formally specified goal instead. For example, "is there a proof of the Riemann hypothesis in ZFC shorter than a million symbols?" If such a goal requires converting the solar system into computronium, then the AI will do that. It won't settle for wireheading because wireheading doesn't give an answer to the…

In fact there's ample reason to think that humans are at the lowest threshold of intelligence that makes a technological civilization possible, because if we'd reached that threshold earlier in our evolution, we'd have created civilization then instead of now. There's probably plenty of room above us. While I don't disagree that humans are essentially at the lowest level of intelligence that make civilization possibl…

I agree with you but only while limiting the scope to the best that can be done for a biological agent which must pass through a birth canal and evolved under strong resource limits.

Can you apply intelligence to more than one person? Can a group of 10 people be considered intelligent as a whole? What about 100? Can you distinguish between the intelligence of different equally sized groups? I say yes to all of those.

Let us say we put groups of 1, 10, 100, 10^n people in a box and gave each box intelligence tests. Which would be most intelligent? Eventually communication costs will outweigh size advantages but only at some n >>0. So a trivial example of an entity whose intelligence surpasses a single human intelligence. Now why do you expect that this level of computation and knowledge management is not replicable on a smaller more efficient device not distracted by the need to calculate proprioception?

Next consider effective intelligence. Do computers when used in the right way amplify intelligence? Is it possible that a human/computer interface is possible? Forget self awareness, is it possible that a computer could do what all of what we call intelligent work better than humans? The answer to all of these is more subjective, especially the last, but I say yes. The human brain is one architecture for intelligence. It is riddled with weaknesses that only make sense when considering the resource limited nature of its development. It was created under a specific set of constraints - a vehicle to manage the more effective transmissions of genes of an energy limited biological entity that must pass a birth canal. I believe that the human mind set on designing an intelligence can do better.

Will a superhuman mind do better at designing a mind in a proportionally similar way to how humans will surpass evolution? Or will they hit too many NP complete problems? The answer to this I cannot give but if forced to bet would be yes, yes.

Post reply on HN