Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Last Invention of Man

nautil.us

Post reply on HN
Print view View original

31–40 of 122 posts

Re: The Last Invention of Man

#31

Post by carapace »

ryandvm wrote: post #19
leggomylibro wrote: post #3

Didn't the people at Los Alamos think that the atomic bomb's chain reaction might keep going and set the entire atmosphere on fire? And I seem to remember the large hadron collider failing to collapse into a black hole. Sure, AI might destroy civilization and/or upend the primacy of humankind. But if history has taught us anything, it's that we're gonna go ahead and do it anyways if it's possible. It's worked so far.

It's worked so far _for us_. But don't you ever wonder why after 14 billion years and with a hundred million star systems in the Milky Way, we still haven't heard a peep from anyone out there? The longer I live, the closer I get to concluding that sentience is an inherently unstable condition.

Quarantine.

Re: The Last Invention of Man

#32

Post by komali2 »

ifdefdebug wrote: post #29

"... the Omegas had pushed hard to make it extraordinary at one particular task: programming AI systems" This single line puts the whole article into the realm of pure science fiction: no technology capable of working such an ill-defined task like "programming AI systems" exists, not even as a work in progress, not even as an idea about how it could be done. And as science fiction, it's kind of a waste of time, but t…

I don't think it's a waste of time, it's bringing to the table a potential future that we need to begin to digest as a society immediately. Star Trek, for example, showed us an "obvious future" in which humanity was perfectly united, and then drew the scope in - what would your average office (or, naval vessel I guess) look like in this world? How would the people interact?

(I say obvious future because I don't see a future in which humanity has spread amongst the stars happening before it unites across cultural lines)

So this story says: "One day, we will have to deal with someone making a multi-purpose AI. This is one of the scenarios it might happen in - secretly, and for profit. Here's some speculation about how that would go down. How do you feel about this? What are the sort of preparations you think society should make, if this were to happen?"

If the only step is "AI happens," that's not nearly so bad or infeasible as "FTL happens" or "Time Travel happens."

I guess I am very stuck on your line "as science fiction, it's kind of a waste of time." How else can we imagine and design our futures but through things that don't currently exist (fiction)? Our entire world is literally science fiction in the eyes of Jules Verne - flying vehicles, submarines, traveling to the moon, etc.

Re: The Last Invention of Man

#33

Post by CoffeeDregs »

ifdefdebug wrote: post #29

"... the Omegas had pushed hard to make it extraordinary at one particular task: programming AI systems" This single line puts the whole article into the realm of pure science fiction: no technology capable of working such an ill-defined task like "programming AI systems" exists, not even as a work in progress, not even as an idea about how it could be done. And as science fiction, it's kind of a waste of time, but t…

    This single line puts the whole article into the realm of
    pure science fiction: no technology capable of working 
    such an ill-defined task like "programming AI systems"
    exists,
I'll admit that they're not (currently) very good at it but there are interesting examples that contradict you: look up Genetic Programming; or Alistair Channon's work (15 years ago) on the generative production of neural networks.

In 2003 and based on Channon's work, I built a system for generating and evolving neural networks using genetic algorithms which coded Lindenmeyer System to build the networks. Attached it to the Heat Bugs agent simulation (albeit with a well defined tasks) and it was startlingly effective. Given the current interest in AI, I've been meaning to resurrect and publish that code.

There was a good blog post on here the other day saying: we're doing lots of work and producing interesting results (the equivalent of the "dual slit experiment" for AI), but we haven't yet formalized the results into a framework. We're probably close.

Re: The Last Invention of Man

#34

Post by komali2 »

zardo wrote: post #18
AaronM wrote: post #7

Earlier quoted context omitted.

We probably would have found remnants of their society. Also if there was, they would have consumed all of the easy to reach resources, and we could have not have made it this far.

>We probably would have found remnants of their society. Not sure if that's true. Especially if technological civilizations tend to be short lived. The Jurassic was a very^very long time ago. What's the state of the paleocryptoxenocheology field any good over view papers?

If organic matter can survive (bones and even sometimes tissues), we can be very assured that hardened structures would.

Furthermore, there's nothing in the fossil record suggesting humans were around very^very long ago, so it'd have to have been a different intelligent species.

Re: The Last Invention of Man

#35

Post by komali2 »

satori99 wrote: post #23
Tenobrus wrote: post #10

Earlier quoted context omitted.

Poor Satoshi Nakamoto. One day it will come back.

...as Roko's basilisk. (Apologies if you hadn't heard of it before now)

Oh for fuck's sake, I had forgotten about this.

https://rationalwiki.org/wiki/Roko's_basilisk

Re: The Last Invention of Man

#36

Post by 5_minutes »

thaumaturgy wrote: post #30
andrewla wrote: post #24

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being…

Thanks for this

Re: The Last Invention of Man

#37

Post by ndkskshfr »

andrewla wrote: post #24

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence.

I’m not so sure about this, which is not so say i’m arguing FOR the leap. But, intuitively, there is an absolute massive leap in terms of perceived intelligence among humans, let alone with chimps. It seems some of the magic sauce for passing a turing test is, evolutionarily speaking, rather small.

Re: The Last Invention of Man

#38

Post by kevin_thibedeau »

andrewla wrote: post #24

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets.

However, once AIs can design and implement their own successors all bets are off.

I would also be concerned about the development of non-theraputic neural implants and the ability of an AI to surreptitiously influence human behavior through them. Thankfully that is likely to to be further off than when we start bumping into the AI singularity.

Re: The Last Invention of Man

#39

Post by jancsika »

I don't understand how in 2017 the author completely ignores the problem of avoiding detection in the Mechanical Turk phase of the story.

Let's handwave away the initial secret epiphany that creates the AI compiler in the first place. You've still got to get it to solve the following problems:

* route around or trick the various Narus devices located at unknown places on the internet to get the software up and running on AWS, and to somehow launder the money you earned on MTurk

* somehow avoid detection by the agencies leveraging the Narus devices and Amazon itself while a non-trivial number of MTurk tasks are being completed by new accounts which themselves are all on AWS.

There are of course variations on those themes (use botnets, ratchet up, etc.) and various associated chess moves. But I don't see any chess move that doesn't end up with both an NSL and a state-level actor taking over the means of producing those AIs for the chief tasks of code-breaking and stockpiling exploits (exploits both in the traditional sense as well as novel ones for the AI compiler itself).

With that in mind I don't think the author's plot arc would be able to mirror today's reality the way author imagines. Because the moment you get the AI equivalent of Stuxnet leaking out onto the internet, the resulting catastrophe would be so obvious and so dependent on AI for a solution that hiding AI would no longer be an option.

Edit: clarification

Re: The Last Invention of Man

#40

Post by dsacco »

ryandvm wrote: post #19
leggomylibro wrote: post #3

Didn't the people at Los Alamos think that the atomic bomb's chain reaction might keep going and set the entire atmosphere on fire? And I seem to remember the large hadron collider failing to collapse into a black hole. Sure, AI might destroy civilization and/or upend the primacy of humankind. But if history has taught us anything, it's that we're gonna go ahead and do it anyways if it's possible. It's worked so far.

It's worked so far _for us_. But don't you ever wonder why after 14 billion years and with a hundred million star systems in the Milky Way, we still haven't heard a peep from anyone out there? The longer I live, the closer I get to concluding that sentience is an inherently unstable condition.

I don't wonder about why we haven't heard from alien civilizations because it's statistically unlikely for us to ever perceive them (putting aside the assumption that we'd recognize their form of life or language in the first place). In a continually expanding universe, the likelihood of us ever interacting with an alien species technically diminishes all the time. For all we know, there could be a stable civilization of aliens so far away it's beyond our observable universe.

I don't think it's logical to use alien species (or the lack thereof) as an instructive point about sentience in general. There's simply not enough information.

Post reply on HN