Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Last Invention of Man

nautil.us

Post reply on HN
Print view View original

71–80 of 122 posts

Re: The Last Invention of Man

#71

Post by pdimitar »

crooked-v wrote: post #66
pdimitar wrote: post #48

Earlier quoted context omitted.

In the absolutely epic space opera "Hyperion" (and its sequel "Endymion") the author Dan Simmons theorizes -- through the plot, not through a monologue! -- that AI creatures would offer us tech that we could never invent -- impossibly minituarized personal computers (the size of a discrete and thin silver bracelet, an earring or even smaller implants under the skin, and which are basic AI by themselves), and real-tim…

That kind of falls apart in this context because a set of mysterious unreachable AIs bearing gifts are a de facto foreign nation and would be treated appropriately.

Well, I agree that I got a bit off-topic. The general analogy is still there but you're right about the details.

That being said, the author handled the problem you outlined with the simple premise of -- that humanity wouldn't be able to even survive as a whole if it delayed the decision how to treat the AI nation.

Without giving spoilers, the fledgling AI nation planned and executed things so well that humanity either had to embrace them and let its core be governed by black-box (and backdoored) tech like teleporters and miniscule personal assistants, or face literal extinction. Needless to say, the humans made a Faustian deal.

Co-evolution of species with wildly different values is the core of the conflict in these books, and even as a fanboy I dare saying it has great value outside of a sci-fi / space opera book.

Re: The Last Invention of Man

#72

Post by gertef »

kevin_thibedeau wrote: post #38
andrewla wrote: post #24

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets. However, once AIs can design and implement their own successors all bets are off. I would also be concerned about the development of non-theraputic neural implants and the ability of an AI…

We also have virus botnets

Re: The Last Invention of Man

#73

Post by kobeya »

sampo wrote: post #67
thaumaturgy wrote: post #30

Earlier quoted context omitted.

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being…

> Maybe any entity significantly smarter than a human being would be crippled by existential despair This was addressed in a Spider-Man story where a villain enhances his intelligence, then first becomes a very successful crime boss, then writes novels and music, and then falls into existential despair. https://en.wikipedia.org/wiki/Flowers_for_Rhino

I think it was done first, and probably better, in Flowers for Algernon.

Re: The Last Invention of Man

#74

Post by vonnik »

One of the most annoying things about Bostrom, Tegmark and other amateur sci-fi authors is that they have chosen parables as their medium. Bostrom likes his sparrows and owls[0], Tegmark likes thinly veiled references to DeepMind with Team Omega.[1] Parables are for children and religious flocks. They are also irrefutable. Which makes them a perfect tool for Bostrom/Tegmark, but an inappropriate interjection into more mature conversations about the state and possible futures of AI. In a sense, they are propaganda, meant to generate feelings of animosity much like an ugly picture of the vicious Hun.

The central, gaping hole in their fear-mongering, as Maciej has pointed out, is that they do not define or quantify intelligence, but they're very free with language when they talk about AI surpassing human intelligence. That is, their projections imply a measurement which they are incapable of making.

I do like the idea that Omega decided to launch a media company, though. Let's call it Interlace. Maybe the only thing David Foster Wallace was missing was a rogue AI capable of creating addictions. This is, in a sense, the same plot as Ex Machina: a machine that knows how to seduce us.

[0] https://blog.oup.com/2014/08/unfinished-fable-sparrows-super...

[1] https://deepmind.com/applied/deepmind-ethics-society/

Re: The Last Invention of Man

#75

Post by ericfrenkiel »

pdimitar wrote: post #48
kevin_thibedeau wrote: post #38

Earlier quoted context omitted.

The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets. However, once AIs can design and implement their own successors all bets are off. I would also be concerned about the development of non-theraputic neural implants and the ability of an AI…

In the absolutely epic space opera "Hyperion" (and its sequel "Endymion") the author Dan Simmons theorizes -- through the plot, not through a monologue! -- that AI creatures would offer us tech that we could never invent -- impossibly minituarized personal computers (the size of a discrete and thin silver bracelet, an earring or even smaller implants under the skin, and which are basic AI by themselves), and real-tim…

Echo the sentiments about "Hyperion" - one of my favorite sci-fi sagas of all time. The author is incredible because his prose are mellifluous regardless and across genre - sci-fi, drama, horror, and historical fiction are all in Dan Simmons' corpus of work.

Re: The Last Invention of Man

#76

Post by sampo »

elevensies wrote: post #12
leggomylibro wrote: post #3

Didn't the people at Los Alamos think that the atomic bomb's chain reaction might keep going and set the entire atmosphere on fire? And I seem to remember the large hadron collider failing to collapse into a black hole. Sure, AI might destroy civilization and/or upend the primacy of humankind. But if history has taught us anything, it's that we're gonna go ahead and do it anyways if it's possible. It's worked so far.

The atmosphere ignition thing was mentioned by Hamming via this anecdote: https://en.wikipedia.org/wiki/Richard_Hamming#Manhattan_Proj... Shortly before the first field test (you realize that no small scale experiment can be done—either you have a critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he sa…

According to this, they calculated it and concluded that it wouldn't be possible.

https://blogs.scientificamerican.com/cross-check/bethe-telle...

Re: The Last Invention of Man

#77

Post by markan »

apsec112 wrote: post #57
markan wrote: post #25

Earlier quoted context omitted.

Yes, this is the problem with AI risk---there's a community pushing hard to gather resources to the cause, but little or no scientific work to be done. This is a rather pathological situation---among other things, the AI risk community makes their own cause look silly, and they promote an unduly negative vision of AGI. I've written more about this here: http://www.basicai.org/blog/ai-risk-2017-08-08.html . On a posit…

"but little or no scientific work to be done." Quite a lot has been written about what scientific work needs to be done. These papers try to summarize possible research directions: https://arxiv.org/pdf/1606.06565.pdf https://intelligence.org/files/TechnicalAgenda.pdf

Yes and no. For safety of narrow AI systems, yeah, there's a lot of scope for research, and that's what your first link gets at.

But for AGI (which is what Tegmark talks about), there's no good way to get a handle on safety yet (other than working towards figuring out AGI).

As for MIRI's agenda, I don't buy that it will help with AGI safety at all. There are a variety of reasons for that, some of which are discussed in the piece I linked above.

Re: The Last Invention of Man

#78

Post by AgentME »

crooked-v wrote: post #69
AgentME wrote: post #56

Earlier quoted context omitted.

I feel like you could substitute AI for any existential threat in that to try to make people look silly. "Climate change ... really is a form of religion" ... add some analogies to Catholic punishment fantasies and doomsday prophecies to make it sound just like the type of thing humans have been talking about forever, criticize scientists for guilting people into giving them money to study their apocalypse fantasy, e…

The difference is that environmental protection evangelists never literally just reinvented Pascal's wager except with an all-powerful AI instead. https://rationalwiki.org/wiki/Roko's_basilisk

I thought the point of Roko's basilisk was a paradox to use to test different decision theories and to explore the nature of identity (Should the decision theory allow your choices to be influenced on possible outcomes to possible future copies of yourself?), and maybe a little about whether dangerous thoughts could exist in theory. Not an actual prediction or argument for or against AI research, any more than say the twin paradox is an argument for or against splitting up twins to send one on a rocket.

Re: The Last Invention of Man

#79

Post by pdimitar »

CoffeeDregs wrote: post #33
ifdefdebug wrote: post #29

"... the Omegas had pushed hard to make it extraordinary at one particular task: programming AI systems" This single line puts the whole article into the realm of pure science fiction: no technology capable of working such an ill-defined task like "programming AI systems" exists, not even as a work in progress, not even as an idea about how it could be done. And as science fiction, it's kind of a waste of time, but t…

This single line puts the whole article into the realm of pure science fiction: no technology capable of working such an ill-defined task like "programming AI systems" exists, I'll admit that they're not (currently) very good at it but there are interesting examples that contradict you: look up Genetic Programming; or Alistair Channon's work (15 years ago) on the generative production of neural networks. In 2003 and…

If you ever decide to publish the code, please tag me! I have my email in my profile here.

What you describe is of big interest to me but I've never done it due to never having to work with it commercially, which is a huge regret of mine.

In any case, any additional practical material and, ideally, your code, will be of immense educational value for me!

Re: The Last Invention of Man

#80

Post by felipemnoa »

thaumaturgy wrote: post #30
andrewla wrote: post #24

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being…

>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.The only intelligence that I believe is important is problem solving. If an AI is as capable as humans at problem solving then that is all you need.

>>crippled by existential despair

Sounds to me that you are talking about feelings. I really do not see the purpose of purposely giving the AI feelings. A strong AI should not have any feelings just like your google maps application does not have any feelings when giving you directions.

A general AI will simply be a general problem solver at the same level of a human but with a virtual infinite capacity to scale its computational abilities.

You could definitely give it free will so that it decides its own goals but why would anybody do something like that unless they want it to turn on them?

It would be like somebody launching a nuclear ballistic missile and letting it decide which city to land in. It is possible to program that type of free will but why? It could decide to destroy your city.

Post reply on HN