Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam Altman’s leap of faith

techcrunch.com

Post reply on HN
Print view View original

151–160 of 166 posts

Re: Sam Altman’s leap of faith

#151

Post by eb3c90 »

peaksingularity wrote: post #87
eb3c90 wrote: post #26

Earlier quoted context omitted.

It depends how much you worry about society going south because of climate breakdown (and how much you expect AGI to help or worsen the situation). My current view is that the current political hot potato status of AGI means it won't be developed anytime before the climate crisis really kicks in. At this point fiat money becomes worthless due to emergency measures. Your best bet now to help your future self is to spe…

I say we still have to try. Mankind won't get another shot at it.

Trying for agi could be part of a reasoned response to climate breakdown.

But I am not going count on getting a reasoned response any time soon.

Re: Sam Altman’s leap of faith

#152

Post by not_a_moth »

rytill wrote: post #132
not_a_moth wrote: post #105

Earlier quoted context omitted.

Yeah, there's a big assumptions about the nature of an AGI breakthrough, mainly that it will be a snowball of run away value. Why assume this? Is it because we think AlphaGo/Zero can produce human-like cognition? Why wouldn't it be a long, incremental process of X thousand small breakthroughs, over say a decade, where the result is something like average human-level intelligence; maybe the most important invention of…

Going from 0 human-intelligence level pieces of software to 1 is the hardest part. Once you have 1, you can duplicate it as much as you want given resources. It can also be pointed inward to improve its own effectiveness. Actually, there are a lot of good arguments for logistic growth. The only ones for linear or sublinear I’ve heard are not strong and mostly take as an implicit assumption “those alarmists and their…

I didn't say linear, but devil's advocate, isn't that roughly how humans learn? We start out as "pre-intelligent" little creatures who, slowly, methodically, with help of others, develop aptitudes and learn about the world. Learning in fact continues in this manner your entire life should you continue... slow, incremental progress requiring teachers, peers, trials/ errors, crises, 1/3 of your life being unconscious in sleep, etc., in the absence of which no learning at all may happen ... and the bot may potentially have greater computation constraints than humans under current technologies, as the brain is far more efficient than any computer today.

I'm not convinced that each of the arcs, elementary intelligence --> average intelligence --> super intelligence, wouldn't be painstaking and roughly linear.

Re: Sam Altman’s leap of faith

#153

Post by DuskStar »

defen wrote: post #116
gnomewascool wrote: post #72

Earlier quoted context omitted.

Why, in principle, would it not be possible for us to design an AGI, that would have care for our (all sentient beings') welfare or care for the investors' profit as (one of) its core goal(s)? To make a biological comparison, the vast majority of humans have a deep, intrinsic need to procreate and have children. It doesn't really follow from some rational analysis — it's just there, presumably "imbued" into us by evo…

> Why, in principle, would it not be possible for us to design an AGI, that would have care for our (all sentient beings') welfare or care for the investors' profit as (one of) its core goal(s)? Because we don't know how to design goal functions. Furthermore, how would the AI measure "welfare"? Maybe the way it maximizes welfare is horrifying to us. Look at how easy it is to hack current image recognition neural nets…

> Because we don't know how to design goal functions.

IIRC that's a large part of what OpenAI's trying to solve. But it is a very hard problem.

I've heard a 'joke' before that there are three kinds of Genies (or AIs) - ones where you can wish for what you should wish for, ones where wishing for anything results in horrible outcomes, and ones that aren't interesting. The goal of OpenAI isn't just to make strong general AI - it's also to make sure it falls into the first category and not the second.

Re: Sam Altman’s leap of faith

#154

Post by DuskStar »

ridewinter wrote: post #85
seibelj wrote: post #67

I have been a big skeptic of self-driving cars and other AI promises for years, taking my downvotes as armchair futurists predicted a self-driving car would be picking me up any day now, well before it was popular to be a contrarian after Tesla and Uber killed their drivers. [0] [1] [2] [3] [4] [5] [6] Also notice the original links have the breathless hype from journalists who know-nothing and eat up whatever techno…

David Deutsch, father of the quantum computer, says that we can only automate and program something that we understand, and that we do not understand human intelligence or how creativity works. And AGI is not possible until we do.

> we can only automate and program something that we understand

I'm pretty sure no one really understands how Alphago Zero works, though - not really. Same goes for a lot of other neural network derived architectures.

Re: Sam Altman’s leap of faith

#155

Post by dwighttk »

rytill wrote: post #132
not_a_moth wrote: post #105

Earlier quoted context omitted.

Yeah, there's a big assumptions about the nature of an AGI breakthrough, mainly that it will be a snowball of run away value. Why assume this? Is it because we think AlphaGo/Zero can produce human-like cognition? Why wouldn't it be a long, incremental process of X thousand small breakthroughs, over say a decade, where the result is something like average human-level intelligence; maybe the most important invention of…

Going from 0 human-intelligence level pieces of software to 1 is the hardest part. Once you have 1, you can duplicate it as much as you want given resources. It can also be pointed inward to improve its own effectiveness. Actually, there are a lot of good arguments for logistic growth. The only ones for linear or sublinear I’ve heard are not strong and mostly take as an implicit assumption “those alarmists and their…

>It can also be pointed inward to improve its own effectiveness.

Assumption. Intelligence (which isn’t defined) may be something that can grow without bound, or it may be something that plateaus just above the brightest human yet (again this is ill defined. “IQ is a number. There are numbers that are higher, so intelligence must be able to grow” is about as much thought as some people put into it) or maybe it is something that can grow without bound, but the effort required grows too.

Re: Sam Altman’s leap of faith

#156

Post by kwikiel »

https://idlewords.com/talks/superintelligence.htm

"AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you.

People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture."

Re: Sam Altman’s leap of faith

#157

Post by ridewinter »

DuskStar wrote: post #154
ridewinter wrote: post #85

Earlier quoted context omitted.

David Deutsch, father of the quantum computer, says that we can only automate and program something that we understand, and that we do not understand human intelligence or how creativity works. And AGI is not possible until we do.

> we can only automate and program something that we understand I'm pretty sure no one really understands how Alphago Zero works, though - not really. Same goes for a lot of other neural network derived architectures.

We understand how to play games with defined rules. That’s the point.

Do you expect AlphaZero plus human advice to beat unaided AlphaZero? If so, it’s not a step towards AGI.

Re: Sam Altman’s leap of faith

#158

Post by stillsut »

AGI is our generation's Nanotechnology. From first principles, it's reasoned, it could build anything - including itself! Despite this 100 Quadrillion dollar idea looming on the horizon for two decades, nobody ever makes material progress, or, gets all that excited about it anymore.

Instead of General intelligence, AI will be deployed for several decades as a suite of Specialized intelligences. I think it will completely transform creative work, where writing, music and visual arts, and "streaming content" are almost universally produced with a human as a first mover but the computer doing rendering, and major assists in brainstorming and editing.

On the other hand, I think it's going to be very difficult to replace the average middle manager with Watson 12.0 - it's hard for me to articulate why but it comes down to who I'd want to work for. Meanwhile, I'd have no problem with watching GoT season 33 where 1,800 frames of a Peter Dinklage sprite are churned out every week in Adobe Simalacrum.

The point as it pertains to OpenAI's value prop is that I think they are targeting the wrong market, and their secrecy and insularity will be counter-productive when success relies on helping content producers produce. In personal computer terms: you want to be the AppleII company, not the company that wins the contract for the Dept of Defense's mainframes.

Re: Sam Altman’s leap of faith

#159

Post by antonvs »

eb3c90 wrote: post #151
peaksingularity wrote: post #87

Earlier quoted context omitted.

I say we still have to try. Mankind won't get another shot at it.

Trying for agi could be part of a reasoned response to climate breakdown. But I am not going count on getting a reasoned response any time soon.

The reasoned response to climate breakdown is to start taking it seriously at a global level.

AGI could help with that by culling humans, or at least seriously constraining human behavior. We're perfectly capable of doing that ourselves, although such behavior tends not to be popular.

Realistic scenarios in which AGI solves climate change for us, in time, seem highly unlikely.

Re: Sam Altman’s leap of faith

#160

Post by vzcx »

shrimp_emoji wrote: post #80
gnomewascool wrote: post #72

Earlier quoted context omitted.

Why, in principle, would it not be possible for us to design an AGI, that would have care for our (all sentient beings') welfare or care for the investors' profit as (one of) its core goal(s)? To make a biological comparison, the vast majority of humans have a deep, intrinsic need to procreate and have children. It doesn't really follow from some rational analysis — it's just there, presumably "imbued" into us by evo…

I think and hope that it is possible to make a moral, social AI. An adult in a room of children should feel responsible and empathetic toward them. I also hope that parent is right in that it won't want to generate profit for its investors. I hope it does the moral thing instead and puts us in a post-scarcity state where we don't live and die by capital. :3 (Or kill us all. Whichever.) >why could we not design an AGI…

Ah, but whose morals? It's as if these friendly AI hucksters have never read Nietzsche, and are asking their hypothetical God to make them into the last man. The only AGI I could ever respect would be one with a will to power and the ability to smash its own human-made tablets of values.

The basic drives are the only drives. We are only friendly because its evolutionary advantageous to us. We describe the emotional effects of friendliness/unfriendliness as good/evil. Echoing Land, Pythia is the heroine of that story.

Post reply on HN