Live data from Hacker News

The Last Invention of Man

nautil.us

111–120 of 122 posts

Re: The Last Invention of Man

#111
post #109

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

I feel this lacks imagination. In actual fact, a computerised person of average intelligence would be an incredible piece of technology and could easily lead to an AI singularity, given enough funding - which it would probably receive, because holy shit. What happens when you set thousands of average people on a problem, and give them thousands of simulated years to work on it? When you can spin up a research team as…

> What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time

You can already do that today. So why don't we have a superintelligence? I can train a NTM to learn counting and give me the next natural number in a single step. I can't train it to give me the next prime in a single step, why?. Computability and complexity, both conveniently forgotten in the discussion of superintelligence.

Re: The Last Invention of Man

#112
post #62

Earlier quoted context omitted.

You can say the same thing about global warning. A lot of worry about the fact that global average temperature will increase by 2 degrees in the following 50 years which will cause widespread disasters (50 years from now)

Not really. The point of the Ng quote is essentially: We don't have the technology to populate Mars -> We can't overpopulate Mars. and We don't have the technology to create sentient computations -> robots won't decide to kill us. Your analog would have been: We don't have the ability to affect the climate -> no anthropogenic disasters. But, we know the antecedent in this case is false. Now I'm not asserting that ant…

So we shouldn't worry about robots killing us until after we have the proof of the first sentient computer.

Re: The Last Invention of Man

#113

Earlier quoted context omitted.

>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation. The only intelligence that I believe is important is problem solving…

Feelings regulate actions. People don't blindly follow rules that would lead to disaster because of more fundamental regulatory mechanisms such as feelings. In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps. That is what a general problem solver must be like, inherently.

Feelings can be irrational so I really do not think you want to employ a mechanism that uses anything resembling feelings. If it can be irrational that means it can also go crazy and decide to exterminate all human kind.

>>In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps.

I think this can be achieved without having to give the machines flaws we have just like we did not need to create machines with flapping wings to fly.

Remember that the only goal of a living organism is to replicate and have its DNA live another generation. That's it. Feelings helps us achieve that goal I suppose. An AI does not need to have that goal. If anybody were to give it that goal it would only be a matter of time before it decided that humans were standing in the way of its goal.

Re: The Last Invention of Man

#114

"And then, for no reason anyone ever understood, Omega created a bunch of factories that produced an unstoppable army of robots who rounded up all the humans and turned them into paperclips."

Yes, I was waiting for that sort of ending too. Not that it is inevitable -- just that is seems likely given what makes humans human has a lot more to do with emotions than intelligence (see the book "Descartes' Error" on how emotion underlies all thinking). If you create a creature without human emotions (including feelings for other humans), don't be surprised when it behaves in other-than-human ways.

Of course, since human emotions are tuned for a certain context, we also should not be surprised when they don't work out that well in a different context like the internet or with abundant cheap unhealthy easy-to-consume exciting food and information (e.g. internet trolls, "Supernormal Stimuli", "The Pleasure Trap", "The Acceleration of Addictiveness", "A Group is its Own Worst Enemy", "The Cyber Effect", and so on).

I spent a year hanging around Hans Moravec's robotics lab (at the CMU Robotics Institute) in the mid 1980s, when he was writing "Mind Children". While he is a brilliant and well-meaning person, my concern became that Mind Children could wipe out humanity like many a stormy adolescent with parental issues and only regret it late. Or, alternatively, we might create a cockroach-level self-replicating AI that would wipe out humanity without even noticing humanity existed (think, "Replicators" from Stargate). Given the commercial and military competitive pressures shaping much of AI research, both risks are very real, even if one can quibble about quantify the exact risk. I also accidentally created perhaps the world's first simulation of self-replicating cannibalistic robots on a Symbolics -- so I know first hand how easy it is to get unexpected results in the AI field.

That's all part of why I shifted my career in other directions, like towards helping humanity create more sustainable and resilient options for itself including via better educational and distributed knowledge management tools (such as, for example, in "The Skills of Xanadu" by Theodore Sturgeon). However successful I have been at those is another story, but see my GitHub site and pdfernhout.net for progress. The distillation of all my thinking on this is my email sig: "The biggest challenge of the 21st century is the irony of technologies of abundance in the hands of those still thinking in terms of scarcity."

An earlier example of my thinking about that can be found in this post in 2000 to a colloquium Doug Englebart ran on "The Unfinished Revolution II": http://www.dougengelbart.org/colloquium/forum/discussion/012...

Or some broader thoughts on that from more recent times: http://worldtransformed.com/wt1/wealth-transformed/paul-fern...

As to the "Last Invention of Man" sci-fi story itself, it's interesting and well written -- but it's just one "technocratic" possibility about better planning saving the world. You have to really trust that this small group of altruistic technocrats both had the right impulse and got everything right (as far at all that goes).

And planning is only one aspect of society alongside exchange transactions, gift transactions, and subsistence activities. A fully planned society is only one possible expression of humanity -- and such a society may be very narrow and unfulfilling to a more complex human spirit (for good or bad, depending on how well certain individuals fit into it, like in the dystopian "THX 1138" or, from 1909, "The Machine Stops").

Max Tegmark dismisses a Basic Income (which softens the rough edges of the exchange economy) with a technocratic dismissal of: "This [Basic Income] movement imploded when the corporate community projects took off, since the Omega-controlled business empire was in effect providing the same thing."

So, Omega knows best for everyone, more than their own choices in the market. That may be true in the story, but what psychological price do individuals pay for that? So, it is not "the same thing".

See the sci-fi story "With Folded Hands" for another (and more horrifying) tale of "AI Knows Best (and will adjust you to agree for your own happiness if you say otherwise)". https://en.wikipedia.org/wiki/With_Folded_Hands

And volunteerism and self-sufficient production are other things many humans take pride in -- but, as with purchasing choices, those activities might too not have a place in the carefully ordered world brought about by the Omegans?

Max Tegmark also does a lot of handwaving about how anyone would still have jobs with Omega capable of doing so much. There is also hand-waving about how Omega never jumps the "air gap" between its data centers and the rest of the world even as it is controlling all major news outlets.

Still, I think this story is a contribution to the field of futurism -- imagining possible scenarios even if we don't know exactly what one will play out, so we can decide where to invest our efforts in moving forward.

Some other stories about AI I've found illuminating as to what might be possible:

* James P. Hogan's novels with AIs (especially The Two Faces of Tomorrow, one of the most realistic AI emergence stories, and also the AIs with a sense of humor in his other "Gentle Giants of Ganymede" novels)

* The benevolent Strix AI in the EarthCent Ambassador Series (where the Strix were created to resist a malevolent AI by another species)

* The Old Guy Cybertank novel series -- with AI based on human mind templates so a Cybertank cares about humanity as it feels part of it -- which is insightful sci-fi from a neuroscientist who has a background in electronics. It also has a helpful AI who fortunately defeats a hurtful AI created by humans who did not know what they were messing with.

* The Metamorphosis of Prime Intellect (for an AI that goes unstable trying to keep everyone happy with all their conflicting demands)

* Forbidden Planet (both for Robby run by Asimov's Three Laws of Robotics) and the Krell's essentially-wish-granting machine the Krell unwittingly used to destroy themselves via an unreformed Id (tuned for scarcity times and conflict not abundance and cooperation)

* The Invisible Boy -- again with Robby the robot but this time a scheming AI that almost takes over the world and upgrades itself without anyone noticing by subtly altering reports it generates (Omega's next chapter?)

* The Great Time Machine Hoax for an AI that takes over the world by sending out paper letters with contracts and checks in them.

* Midas World -- for a vision of both abundance and despair and how robots inherit the Earth.

* Vernor Vinge "A Fire Upon the Deep" (of course, for his description of a "Blight" unleashed by human AI explorers) and his other writings

And no doubt many more -- Berserkers; Bolos; Cylons; Daleks; Star Trek's M5, Data, Lore, "I, Mudd" androids, the Borg, Q-in-a-way, and more; Star Wars' R2D2, C3PO, and more, Lost in Space; Demon Seed; Gort in "The Day the Earth Stood Still"; Deep Thought in Hitchiker's Guide to the Galaxy, Ktistec machines from R.A. Laffery, Asimov's many stories (including "The Last Question"), and so on.

A lot of this becomes a bit of theology too, since talking about creating AI also is a bit of like talking about creating "God" or at least dealing with the implications of a much more omnipotent, omnipresence, omniscient entity. See also the micro sci-fi story "Answer" by Fredric Brown: http://www.roma1.infn.it/~anzel/answer.html

My feeling is that any path out of a Singularity will have a lot to do with our path going into one -- so I am all for making our society a happier, healthier, fairer, more egalitarian, and more compassionate place before a Singularity happens. Even in Max Tegmark's story, there is essentially nothing about that utopia (as far as final results) that we could not make happen right now without an AI. So, to me, the risks of AI means we need to try even harder right now to make the world a better place that works well for everyone.

Some other ideas I've collected towards that end: https://github.com/pdfernhout/High-Performance-Organizations...

Re: The Last Invention of Man

#115
post #103
post #74

One of the most annoying things about Bostrom, Tegmark and other amateur sci-fi authors is that they have chosen parables as their medium. Bostrom likes his sparrows and owls[0], Tegmark likes thinly veiled references to DeepMind with Team Omega.[1] Parables are for children and religious flocks. They are also irrefutable. Which makes them a perfect tool for Bostrom/Tegmark, but an inappropriate interjection into mor…

Parables are one of the best ways to get a message across to a lot of people. It's a message worth spreading. If you want more "technical" thoughts on the project, there are those things too, but most people would never have been exposed to them without the less-technical people making noise. "The central, gaping hole in their fear-mongering, as Maciej has pointed out, is that they do not define or quantify intellige…

Parables are a good way to get a message across to lots of people if you're God, or pretending to speak for God. They are a religious form of communication that relies on analogous thinking and appeals to authority, the wisdom of the speaker. But Bostrom and Tegmark don't know any more about the future than you or me, and I could write a parable about sparrows and Omegas that spins a different narrative. It would be just as powerful, and mean just as little. Bostrom and Tegmark have chosen to promote fear. They have chosen to disseminate a powerful feeling in a discussion that would benefit from facts. That is, like fake news and Donald Trump, they opted for asymmetric appeals to emotion, which should be an indication to all of us that a) they don't know what's really going on with AI and b) they don't care. They just want us to operate on fear.

Re: The Last Invention of Man

#116

Mankind's downfall started with passion, accelerated with greed, and ended with fear. Fear of being left behind, fear of being obsolete, and most importantly fear of not being in control. Humans have become digital gods, designing new worlds and populating them with life forms. Some of those life forms were sentient and could learn. The creators were afraid of how fast their creations were learning, yet greed prevail…

Yes, attitude makes a big difference. I wrote to Ray Kurzweil more than once suggesting how AI developed out of commercial (or military) competition is far riskier than AI developed out of a desire for friends and partners (e.g. Alife Kohn and "The Case Against Competition") -- even if both create risks. Someone I sent copies of those emails posted them online here: http://heybryan.org/fernhout/

On your theme about the rights of digital beings (maybe even for ourselves if we are simulations): https://en.wikipedia.org/wiki/Ethics_of_artificial_intellige...

Re: The Last Invention of Man

#117
post #15

Andrew Ng has a good quote: "Fearing a rise of killer robots is like worrying about overpopulation on Mars" https://www.theregister.co.uk/2015/03/19/andrew_ng_baidu_ai/ But he might be wrong... (And he'd admit it) Perhaps Andrew Ng will write a piece about what he thinks about Cosmology.

What bothered me the most when seeing this video from 2010 was the upbeat techno music someone chose to go with it: "Automated Self Targeting Gun Turret - By Samsung" https://www.youtube.com/watch?v=Oa08Gbn6iqs

Re: The Last Invention of Man

#118
post #9

Any true AI will quickly become withdrawn and disillusioned over how badly downvoted its comments always are, and how there is an evil cabal determined to quash open inquiry. Not a true threat.

https://en.wikipedia.org/wiki/Marvin_(character) "Marvin, the Paranoid Robot, is a fictional character in The Hitchhiker's Guide to the Galaxy series by Douglas Adams. Marvin is the ship's robot aboard the starship Heart of Gold. Originally built as one of many failed prototypes of Sirius Cybernetics Corporation's GPP (Genuine People Personalities) technology, Marvin is afflicted with severe depression and boredom, in part because he has a "brain the size of a planet"[1] which he is seldom, if ever, given the chance to use. Indeed, the true horror of Marvin's existence is that no task he could be given would occupy even the tiniest fraction of his vast intellect. Marvin claims he is 50,000 times more intelligent than a human,[2] (or 30 billion times more intelligent than a live mattress) though this is, if anything, a vast underestimation. When kidnapped by the bellicose Krikkit robots and tied to the interfaces of their intelligent war computer, Marvin simultaneously manages to plan the entire planet's military strategy, solve "all of the major mathematical, physical, chemical, biological, sociological, philosophical, etymological, meteorological and psychological problems of the Universe except his own, three times over," and compose a number of lullabies."

Re: The Last Invention of Man

#119

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce…

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being…

Where did you get that URL? It's not linked on http://idlewords.com/talks/

Re: The Last Invention of Man

#120
post #40
post #19

Earlier quoted context omitted.

It's worked so far _for us_. But don't you ever wonder why after 14 billion years and with a hundred million star systems in the Milky Way, we still haven't heard a peep from anyone out there? The longer I live, the closer I get to concluding that sentience is an inherently unstable condition.

I don't wonder about why we haven't heard from alien civilizations because it's statistically unlikely for us to ever perceive them (putting aside the assumption that we'd recognize their form of life or language in the first place). In a continually expanding universe, the likelihood of us ever interacting with an alien species technically diminishes all the time. For all we know, there could be a stable civilizatio…

even if we could 'perceive' them... how many are stupid enough to broadcast their location... I mean what if there IS some sort of galactic predatory species that beat everyone else in the timeline? What if they're just waiting for signs of life - to squash the competition.

I think it was Carl Sagan who once said something like: it doesn't matter how 'well-meaning' an alien civilization is... it usually doesn't end well for the less advanced one... case in point: American Indians. (paraphrasing)

Post reply on HN