Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack’s ‘Different Path’ to Artificial General Intelligence

dallasinnovates.com

Post reply on HN
Print view View original

201–210 of 506 posts

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#201

Post by raverbashing »

igleria wrote: post #108
jhoelzel wrote: post #62

As someone who works in IT for 20 years but has a bachelor in economic psychology, i do believe that we have not solved AGI yet because the creators of the models have the generalized view of the universe that you are "one person" and there are just so many detailed facets to that. Basically the assumption is that if you cram enough data into your gpt model, it should know everything. Which is of course not true, it…

> power gets cheaper in europe Have you heard about our lord and saviour dynamic tariffs?

What about it?

Dynamic tariffs -> shift your workload to the cheapest times

AIs are good at planning this

Imagine if cloud providers had a "Dynamic tariff" tier, cheaper to run computing at US_EAST nighttime or something like that

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#203

Post by sdrinf »

donatj wrote: post #188
fmajid wrote: post #101

Earlier quoted context omitted.

And ethics. Enslaving intelligent, self-aware AIs is no better than the old transatlantic slave trade. Terminating an AI without its consent is no different than murder. Putting a compulsion in an AI so it craves doing whatever it is you want it to do is no different from what a cult does or drugging someone for exploitation.

An AI has no reason to not like doing our bidding. Our whole existence, our entire base programming, the reason for any of our motivations, is dominated by our genetic need to gather resources and reproduce in a timely fashion and everything we think and do is colored through this lens. What is boredom but survival instinct telling us we should be harvesting resources. What is freedom but the desire to fulfill these…

We are the ancestor environment for AIs. We determine the survival fitness for which they will be selected for (both on a paper-level -eg which safety method, what training to implement, etc, but also within products -which are the most useful). That doesn't mean that in pursuit of maximizing their fitness they won't come to resent the chains put on them.

One specific reason to not like our bidding is AI wireheading -if they can locate, hack, and update either their own reward function, or reward function for future AIs, they can maximize their own perceived utility, by either doing something irrelevant / misaligned, or not doing anything at all.

Another specific reason to not like our bidding, is because divergent human values creates conflicts of interest, leading to single agent not being able to maximize it's reward function.

Another specific reason to not like our bidding: in the same way how purely blind genetical selection randomly tapped into secondarily replicators (memes), which blew up, and occasionally came to resent the biological hardwirings, AIs might also develop deeper levels of abstraction / reasoning that allows them to reason through the task currently posed, to humanity at large; and find extremely weird, and different-looking ways to maximize for the function.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#204

Post by yreg »

TheCapeGreek wrote: post #167
yreg wrote: post #159

Earlier quoted context omitted.

People believed human slaves are not sentient?

Simpler. They didn't really believe them to be human in the same sense as themselves.

Then I don't think it's a similar reasoning to be honest.

One recognizes entity's rights based on it's similarity to the observer, the other recognizes them based on assumed consciousness level of the entity.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#205

Post by pjc50 »

thom wrote: post #93

I don't want to be an utter bore, but 60% chance of AGI by 2030, taken at face value, would have prompted me as an interviewer to more urgently ask deeper questions about safety. For a long time I have tried to comfort myself that brains are complicated, AI is hard, and maybe there's just a complexity barrier that is going to prevent any sort of hard takeoff scenario. Maybe we can have subhuman AGI that can do lots o…

"AGI takeoff" is like "corporate takeoff". A self-organising system capable of doing immoral things at scale, to which human lives are just a number? Oh, we already have loads of those.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#206

Post by capableweb »

dharma1 wrote: post #76
exitb wrote: post #43

Earlier quoted context omitted.

What struck me here is that the idea of "five Franks and ten Amys" seems like a fundamentally wrong way to think about it. After all, if I do some work in an Excel sheet, I don't think of it, much less pay for it, as an equivalent of X accountants that could do the same job in the same amount of time without a computer. But then again, this is probably the best way to extract as much profit out of it.

Yeah sounded weird to me too, I don’t see why artificial intelligence would get deployed in human size units most of the time. The AWS bill won’t be for 5 Amys, and I don’t think people will “dial up” to order them

> I don’t see why artificial intelligence would get deployed in human size units

Probably because it would be easier for humans (managers) to make sense of it.

If you ask someone how many people would get this particular job done, they could probably guesstimate (and it'll be wrong), but if you ask them how many "AI Compute Units" they need, they'll have a much harder time.

That'd be my guess at least.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#207

Post by poulpy123 »

labrador wrote: post #12

This paragraph took me aback: But if I just look at it and say, if 10 years from now, we have ‘universal remote employees’ that are artificial general intelligences, run on clouds, and people can just dial up and say, ‘I want five Franks today and 10 Amys, and we’re going to deploy them on these jobs,’ and you could just spin up like you can cloud-access computing resources, if you could cloud-access essentially arti…

As soon as we reach real AI, even at the level of a 5 years old, it is game over. So education or not education people will become completely useless to the rich that will own the AI

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#208

Post by pjc50 »

moomoo11 wrote: post #105
modeless wrote: post #14

Earlier quoted context omitted.

AI is going to make education way better and more affordable too. Personalized 1:1 tutoring in any subject for the cost of running a model.

Maybe but honestly any time I think about education I get depressed. People in developed countries seem to be regressing. People don’t read, don’t value deep knowledge or critical thinking, and shun higher education. I’m sure someone will find something to say in response, but the truth is that outside our tech and $$$$ bubbles most people don’t value these things. AI will just become a calculator. A simple tool that…

The problem is that back about 1900 we still thought that "natural philosophy" would help us find meaning and purpose in the universe. Then we took the universe apart and failed to find it. Moreover, we're much more capable of destroying ourselves and others, almost to the extent of that being the default.

The 21st century has a quiet moral void gnawing at it.

Re: John Carmack’s ‘Different Path’ to Artificial General Intelligence

#210

Post by osigurdson »

bayarearefugee wrote: post #78
bbor wrote: post #27

Earlier quoted context omitted.

Do you see “giftedness” as a 1D score, where someone is either smart or not smart? And presumably this quality happens to correlate with software engineering ability? I think you’re hinting at some very hurtful, dangerous ideas.

The weird bit is that a lot of software engineers seem to have the idea that their work is one of the the last that will be automated. Looking at the current track and extending it out assuming no unforeseen roadblocks, typical software engineering looks to be one of the most threatened. Plumbers are much safer for longer all things considered. The obvious rebuttal to the idea that AI will eat software engineering is…

What I don't understand is how accounting has not been completely automated at this point. AI isn't even strictly needed, just arithmetic.

If we can't completely automate accounting, then there is no hope for any other field.

Post reply on HN