Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Software Revolution

blog.samaltman.com

Post reply on HN
Print view View original

261–270 of 397 posts

Re: The Software Revolution

#261

Post by Balgair »

JoeAltmaier wrote: post #251
Balgair wrote: post #248

Earlier quoted context omitted.

True, but the viruses and bacteria, once out of Dr. Doom's lab, will evolve themselves. A virus that kills all the hosts is not a good virus. It has to be just the right amount of deadly and contagious to survive. Look at ebola, that is super nasty stuff, but it kills so quickly that it is hard to make it widespread. I'm not gonna say it is impossible, but it is a lot harder to do that you'd think. Living things tend…

Not really true. A virus that kills the host too quickly is not going to spread. A virus that kills the host quickly but not before it spreads, IS going to spread. There's no saying what a virus 'wants'; they just happen, and they do what they do. If Ebola became airborne, then most of us would die, then Ebola would die (from lack of hosts), and that's just a pity for Ebola. But there's nothing that stops such a scen…

To expound on this point, to a virus or bacteria, a human is just as good as a monkey or dog or jellyfish. It's a place to replicate and live. Similar with viruses as technically they are not alive. All these Dr. Doom kinda things have to compete with the common cold, the e. coli, and all the other things that live on the earth and in your guts. That is not an easy environment to survive in.

Re: The Software Revolution

#262

Post by tptacek »

Micaiah_Chang wrote: post #258
JoeAltmaier wrote: post #247

Earlier quoted context omitted.

'Smart' isn't really a thing. There's speed of thought, and knowledge of subject, and savant abilities to factor or calculate. Its silly to sort folks on a one-dimensional line called 'smart' and imagine you have done anything actionable. I'd say, AI is dangerous because we cannot fathom its motivation. To give us true answers to questions? To give pleasing answers? To give answers that help it survive? The last is i…

Smart is just a shorthand for a complicated series of lower level actions consisting of domain knowledge, raw computational speed and other things yes. I don't think we're really disagreeing about this. However, I do worry that you're confusing the existing constraints on the human brain (that people seem to have tradeoffs between, let's say, charisma and mathematical ability) and constraints that would apply to all…

I sort of am questioning that factor, yes. Stipulate, I don't know, several orders of magnitude of intellectual superiority. Stipulate that human intelligence can be captured in silico, so that when we talk about "intelligence", we are using the most charitable possible definition for the "fear AI" argument.

What precise vectors would be available for this system to manipulate the world to harm humans?

Re: The Software Revolution

#264

Post by JoeAltmaier »

Micaiah_Chang wrote: post #258
JoeAltmaier wrote: post #247

Earlier quoted context omitted.

'Smart' isn't really a thing. There's speed of thought, and knowledge of subject, and savant abilities to factor or calculate. Its silly to sort folks on a one-dimensional line called 'smart' and imagine you have done anything actionable. I'd say, AI is dangerous because we cannot fathom its motivation. To give us true answers to questions? To give pleasing answers? To give answers that help it survive? The last is i…

Smart is just a shorthand for a complicated series of lower level actions consisting of domain knowledge, raw computational speed and other things yes. I don't think we're really disagreeing about this. However, I do worry that you're confusing the existing constraints on the human brain (that people seem to have tradeoffs between, let's say, charisma and mathematical ability) and constraints that would apply to all…

Agreed, digital brains will be unfathomable. What are they thinking, in between each word they laboriously transmit to us slow humans? They will have epochs to think while we are clearing our throats.

Re: The Software Revolution

#265

Post by Micaiah_Chang »

tptacek wrote: post #257
EliAndrewC wrote: post #250

Earlier quoted context omitted.

Here's the standard argument, as I understand it: - There are something like 100,000,000,000 neurons in the human brain, each of which can have up to around 10,000 synaptic connections to other neurons. This is basically why the brain is so powerful. - Modern CPUs have around 4,000,000,000 transistors, but Moore's law means that this number will just keep going up and up. - Several decades from now (probably in the 2…

Ok, to both you and 'Micaiah_Chang cross-thread: I do understand where the notion of hockey-stick increases in intellectual ability comes from. I do understand the concept that it's hard to predict what would come of "superintellectual" ability in some sort of synthetic intelligence. That we're in the dark about it, because we're intellectually limited. I don't understand the transition from synthetic superintellectu…

Er, sorry for giving the impression that it'd be a supervillain. My intention was to indicate that it'd be a weird intelligence, and that by default weird intelligences don't do what humans want. There are some other examples which I could have given to clarify (e.g. telling it to "make everyone happy" could just result in it giving everyone heroine forever, telling it to preserve people's smiles could result in it fixing everyone's face into a paralyzed smile. The reason it does those things isn't because it's evil, but because it's the quickest+simplest way of doing it; it doesn't have the full values that a human has)

But for the "off" switch question specifically, a superintelligence could also have "persuasion" and "salesmanship" as an ability. It could start saying things like "wait no, that's actually Russia that's creating that massive botnet, you should do something about them", or "you know that cancer cure you've been looking for for your child? I may be a cat picture AI but if I had access to the internet I would be able to find a solution in a month instead of a year and save her".

At least from my naive perspective, once it has access to the internet it gains the ability to become highly decentralized, in which case the "off" switch becomes much more difficult to hit.

Re: The Software Revolution

#266

Post by scobar »

YAYERKA wrote: post #196

>Trying to hold on to worthless jobs is a terrible but popular idea. It seems warm and fuzzy to think Sam, and the implicit company he keeps (the ultra rich)--who are "leveraging not only their abilities and luck" but already accrued wealth--can and will redistribute it. Anyone who wasn't born yesterday will simply laugh at this prospect. I'm not sure why Sam feels the need to call what most of the world is doing wor…

Unfortunately, many societies today incorporate too strongly one's official profession/title with self-worth. I don't believe sama was implying those who work those jobs are worthless. Instead, it seems like he's trying to say that there are far better ways to accomplish the same objective, and we shouldn't ignore them.

Because our systems of retraining and placing workers into a new profession are so terrible, it's common to assume that many or most displaced workers will remain unemployed. Breaking this status quo is essential to giving everyone a fair chance to work on what truly drives them while we automate more and more worthless jobs. That's why I'm so excited about free and widely available educational resources springing up online. It's not perfect yet, but we had to start somewhere. I have a deep respect for everyone (including sama) who've helped build or teach a mooc.

Re: The Software Revolution

#267

Post by tptacek »

Micaiah_Chang wrote: post #265
tptacek wrote: post #257

Earlier quoted context omitted.

Ok, to both you and 'Micaiah_Chang cross-thread: I do understand where the notion of hockey-stick increases in intellectual ability comes from. I do understand the concept that it's hard to predict what would come of "superintellectual" ability in some sort of synthetic intelligence. That we're in the dark about it, because we're intellectually limited. I don't understand the transition from synthetic superintellectu…

Er, sorry for giving the impression that it'd be a supervillain. My intention was to indicate that it'd be a weird intelligence, and that by default weird intelligences don't do what humans want. There are some other examples which I could have given to clarify (e.g. telling it to "make everyone happy" could just result in it giving everyone heroine forever, telling it to preserve people's smiles could result in it f…

So like it's clear to me why you wouldn't want to take a system based on AI-like technology and have it control air traffic or missile response.

But it doesn't take a deep appreciation for the dangers of artificial intelligence to see that. You can just understand the concept of a software bug to know why you want humans in the observe/decide/act loop of critical systems.

So there must be more to it than that, right? It can't just be "be careful about AI, you don't want it controlling all the airplanes at once".

Re: The Software Revolution

#269

Post by JoeAltmaier »

tptacek wrote: post #262
Micaiah_Chang wrote: post #258

Earlier quoted context omitted.

Smart is just a shorthand for a complicated series of lower level actions consisting of domain knowledge, raw computational speed and other things yes. I don't think we're really disagreeing about this. However, I do worry that you're confusing the existing constraints on the human brain (that people seem to have tradeoffs between, let's say, charisma and mathematical ability) and constraints that would apply to all…

I sort of am questioning that factor, yes. Stipulate, I don't know, several orders of magnitude of intellectual superiority. Stipulate that human intelligence can be captured in silico , so that when we talk about "intelligence", we are using the most charitable possible definition for the "fear AI" argument. What precise vectors would be available for this system to manipulate the world to harm humans?

I guess the same vectors available to a blogger, or pundit, or politician? To fear-monger; to mislead important decision makers; to spread lies and manipulate the outcomes of important processes.

Is it possible to say that such AIs are NOT at work right now, fomenting terrorism, gathering money through clever investment and spending it on potent schemes to upset economies and governments?

Re: The Software Revolution

#270

Post by eli_gottlieb »

joesmo wrote: post #194

"Trying to hold on to worthless jobs is a terrible but popular idea." It's terrible sure, but it's popular only because our economy requires it. That is the basis for the whole economy. It's not like people are clamoring to serve McDonalds for minimum wage or clean shit out bathrooms. They have no other choices in this economy. The economy demands it. While those jobs might be necessary, most middle management and of…

>It's terrible sure, but it's popular only because our economy requires it. That is the basis for the whole economy. It's not like people are clamoring to serve McDonalds for minimum wage or clean shit out bathrooms. They have no other choices in this economy. The economy demands it. While those jobs might be necessary, most middle management and office type jobs are incredibly redundant and frankly, pointless. They are there because people need to eat and we haven't figured out a better, more appropriate way of wealth transfer.

Then we should probably change the economy so that the people don't have to suffer so much.

>It's not astonishing considering that these things don't exist, pose no threat, and the people in power wouldn't understand them even if they did exist. There are many more pressing issues that hypotheticals.

I'll go tell friends, colleagues, and my professional idols that their work does not exist and is purely hypothetical, then.

Post reply on HN