Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evolution of AI and Amara's Law

n9o.xyz

Post reply on HN
Print view View original

31–37 of 37 posts

Re: Evolution of AI and Amara's Law

#31

Post by svieira »

ben_w wrote: post #7
nunocoracao wrote: post #5

Earlier quoted context omitted.

Amen. I wonder what "underestimating" something like AGI actually means though...

Assuming it's aligned and we're still around… AGI? Lots of people are still talking like it will be an assistant, perhaps using the "centaur" metaphor, and refusing to believe that it could[0] result in everyone's mental capacities having all the economic relevance of equine muscle power. ASI? We don't know how far above us IQ can go, but it seems reasonable to guess that the smallest possible margin above us is such…

> it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question.

"You mustn't be afraid to dream a little bigger" with regards to the hierarchy of intelligence - the "at worst" is much further away than human-to-lawn. Look at the intellectual power of angels in Catholic theology and consider an intelligence that can reason from universals to all particular instances as its default mode of operation. Then consider that each angelic being is distinct from each other one by nature (since they are immaterial). Which means that there is a species-level difference between each of them and the lowest one is already far above us. Then consider that there are (at least) billions of them, each one order of magnitude more intelligent than the one below it. Then consider the kind of intellect that can create those kinds of intellects.

Fortunately, you can't give what you don't have, so we're quite safe from the basilisk. But the wild imaginings of John C. Wright's Count to the Eschaton Sequence are probably not far off for "hey how weird could it get?"

Re: Evolution of AI and Amara's Law

#32

Post by ben_w »

svieira wrote: post #31
ben_w wrote: post #7

Earlier quoted context omitted.

Assuming it's aligned and we're still around… AGI? Lots of people are still talking like it will be an assistant, perhaps using the "centaur" metaphor, and refusing to believe that it could[0] result in everyone's mental capacities having all the economic relevance of equine muscle power. ASI? We don't know how far above us IQ can go, but it seems reasonable to guess that the smallest possible margin above us is such…

> it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question. "You mustn't be afraid to dream a little bigger" with regards to the hierarchy of intelligence - the "at worst" is much further away than human-to-lawn. Look a…

The point of my comparison was not to indicate the exact scale of thought difference, but the incomprehensibility gap. In this regard, while I've not read Count to the Eschaton, I'm confident any work written by humans isn't weird enough.

Re: Evolution of AI and Amara's Law

#33

Post by ben_w »

visarga wrote: post #16
nunocoracao wrote: post #8

Earlier quoted context omitted.

Agree, the path might be quite bumpy but so could the endgame. Whether AGI wants to help or kill the human race will depend on how aligned it is to our goals. Now, since "our" goals will depend on whoever has the metaphysical ear of such AI this could be problematic.

I think I read this kind of discussions too many times, imagine how the training set of GPT-5 will look like? all sorts of theories of "whether AGI wants to help or kill the human race" spread over the internet and analyzed to death. It is going to know this topic in-and-out. Will be able to write a masterful dissertation on the topic.

Yes, and that's good because GPT-5 is going to be coming from a team that cares about alignment and which has values only a little different from mine. They can still mess up: https://youtu.be/qV_rOlHjvvs?si=6SI_abC9BLHGoWjt

OTOH, an LLM trained by the FSB or CIA to help their agents, with no limits on what the agents are allowed to get as help because their bosses trust it's always in the service of the nation, will quite "happily" write that essay… and almost certainly get leaked eventually — 10 years, 50 years, doesn't matter — at which point someone who isn't so trustworthy will ask for a doomsday plan and it will be equally "happy" to write that too.

Hopefully the agencies like the FSB and CIA aren't quite as bad as I fear, but secrecy and what little has become public makes it impossible to trust them.

Re: Evolution of AI and Amara's Law

#34

Post by ben_w »

melagonster wrote: post #30
ben_w wrote: post #18

Earlier quoted context omitted.

> Why do we act like people all have an approximately uniform intelligence that will be surpassed by AI intelligence by orders of magnitude in all dimensions? I at least don't consider people to be uniform. If I act like I do, that's illusory. For orders of magnitude higher performance by AI: all have a massive intrinsic speed advantage already (which is currently balanced by an almost but not quite equally massive t…

yes, chatGPT had let people lose their jobs. this also prove that we have no chance when AI got everyone's job. do anyone care people who displace by chatGPT?

> this also prove that we have no chance when AI got everyone's job.

Not "prove", but it is suggestive. To prove it, ChatGPT would have to also prevent them getting another one.

> do anyone care people who displace by chatGPT?

I can't parse this sentence. Are you asking:

1. Does anyone care about the people who are displaced by ChatGPT?

2. Does anyone care for the people who are displaced by ChatGPT?

3. Does anyone care that people are displaced by ChatGPT?

4. Does anyone care about the people who are using ChatGPT to displace others?

Or something else?

Re: Evolution of AI and Amara's Law

#36

Post by melagonster »

ben_w wrote: post #34
melagonster wrote: post #30

Earlier quoted context omitted.

yes, chatGPT had let people lose their jobs. this also prove that we have no chance when AI got everyone's job. do anyone care people who displace by chatGPT?

> this also prove that we have no chance when AI got everyone's job. Not "prove", but it is suggestive. To prove it, ChatGPT would have to also prevent them getting another one. > do anyone care people who displace by chatGPT? I can't parse this sentence. Are you asking: 1. Does anyone care about the people who are displaced by ChatGPT? 2. Does anyone care for the people who are displaced by ChatGPT? 3. Does anyone c…

>Not "prove", but it is suggestive. To prove it, ChatGPT would have to also prevent them getting another one.

Maybe it won't take all jobs. but this displayed that it is impossible to implement UBI or other strategies before most of people losing job.

>I can't parse this sentence. thanks for your free English class, bro.

Re: Evolution of AI and Amara's Law

#37

Post by svieira »

ben_w wrote: post #32
svieira wrote: post #31

Earlier quoted context omitted.

> it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question. "You mustn't be afraid to dream a little bigger" with regards to the hierarchy of intelligence - the "at worst" is much further away than human-to-lawn. Look a…

The point of my comparison was not to indicate the exact scale of thought difference, but the incomprehensibility gap. In this regard, while I've not read Count to the Eschaton , I'm confident any work written by humans isn't weird enough .

Whereas, what I'm saying is, given that you can't give what you don't have, we don't have to worry about creating something more intelligent than we can be. But just how intelligent we can be is a pretty surprisingly large horizon in the first place.
Post reply on HN