Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

Post reply on HN
Print view View original

31–40 of 181 posts

Re: The Coming Technological Singularity (1993)

#31

Post by jodrellblank »

WillAdams wrote: post #6

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

Skip a few generations and the machine will build itself. There’s no need for it to take lasers exploding tin to generate ultra Violet to etch patterns to make intelligence, humans don’t grow brains that way or spend billions on fabs and power plants to produce children.

How it gets from here to there is a handwave, though.

Re: The Coming Technological Singularity (1993)

#32

Post by nradov »

WillAdams wrote: post #6

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

Right. In order to design a significantly better computer system, you first need to design a better (smaller feature size) EUV lithography process which can produce decent yield at scale.

Re: The Coming Technological Singularity (1993)

#33

Post by rsanheim »

jodrellblank wrote: post #31
WillAdams wrote: post #6

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

Skip a few generations and the machine will build itself. There’s no need for it to take lasers exploding tin to generate ultra Violet to etch patterns to make intelligence, humans don’t grow brains that way or spend billions on fabs and power plants to produce children. How it gets from here to there is a handwave, though.

That’s a pretty enormous handwave.

Re: The Coming Technological Singularity (1993)

#34

Post by aithrowawaycomm »

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter.

I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now.

1) A nitpick I've had since grad school: the answer to "can we create a machine equivalent to a human mind [assuming arbitrary resources]?" is "yes, of course." The atoms in a human body can be described by a hideously ugly system of Schrödinger equations and a Turing machine can solve that to arbitrary numerical precision. Even Penrose's loopy stuff about consciousness doesn't change this. QED.

2) The more serious issue: I sincerely have no idea why people believe so strongly that a human-level AI can build a superhuman AI. It is bizarre that this claim is accepted with "little doubt" when there are very good reasons to doubt it: how on earth would such an AI even know it succeeded? How would it define the goal? This idea makes sense for improving Steven Tyler-level AI to Thelonious Monk-level; it makes no sense for a transition like chimp->human. Yet that is precisely the magnitude of transition envisioned with these singularity stories.

You might defend the first point by emphasizing "can we create a human-level AI?" i.e. not whether it's theoretically possible, but humanly feasible. This just makes the second point even more incoherent! If humans are too stoopid to build a human-level AI, why would a human-level AI be...smarter than us?

I just don't understand how anyone can rationally accept this stuff! It's so dumb! Tech folks (and too many philosophers) are hopped up on science fiction: the reason these things are accepted with "little doubt" is that this is religious faith dressed up in the language of science.

Re: The Coming Technological Singularity (1993)

#35

Post by bbor »

trescenzi wrote: post #28
samsartor wrote: post #15

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting…

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

There’s a very big difference between knowing “how” to solve a problem in a broad sense, eg “if we shared more we could solve hunger”, and “how” to solve it in terms of developing discrete, detailed procedures that can be passed to actuators (human, machines, institutions) and account for any problems that may come up along the way.

Sure, there are some political problems where you have to convince people to comply. But consider a rich corporation building a building, which will only contract with other AI-driven corporations whenever possible; they could trivially surpass anyone doing it the old way by working out every non-physical task in a matter of minutes instead of hours/days/weeks, thanks to silicon’s superior compute and networking capabilities.

Even if we drop everything I’ve said above as hogwash, I think Vinge was talking about something a bit more directly intellectual, anyway: technological development. Sure, there’s some empirical steps that inevitably take time, but I think it’s obvious why having 100,000 Einsteins in your basement would change the world.

Re: The Coming Technological Singularity (1993)

#36

Post by nradov »

bbor wrote: post #35
trescenzi wrote: post #28

Earlier quoted context omitted.

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

There’s a very big difference between knowing “how” to solve a problem in a broad sense, eg “if we shared more we could solve hunger”, and “how” to solve it in terms of developing discrete, detailed procedures that can be passed to actuators (human, machines, institutions) and account for any problems that may come up along the way. Sure, there are some political problems where you have to convince people to comply.…

An AI-driven corporation wouldn't be able to surpass anyone doing it the old way because they'd still have to wait for building permits and inspections.

Re: The Coming Technological Singularity (1993)

#37

Post by nradov »

cloudking wrote: post #4
crackalamoo wrote: post #3

> I'll be surprised if this event occurs before 2005 or after 2030. I'm not truly confident AGI will be achieved before 2030, and less so for ASI. But I do think it is quite plausible that we will achieve at least AGI by 2030. 6 years is a long time in AI, especially with the current scale of investment.

What is AGI and ASI? I think a fundamental issue here is both are sci-fi concepts without a clear agreement on the definitions. Each company claiming to work towards "AGI" has their own definition. How will someone claim they've achieved either, if we can't agree on the definitions?

I propose we define AGI as a "strong" form of the Turing test. It must be able to convince a jury of 12 tenured college professors drawn from a variety of academic disciplines that it's as intelligent as an average college freshman over a period of several days. So it need not be an expert in any subject but must be able to converse, pursue independent goals, reason, and learn — all in real time.

Re: The Coming Technological Singularity (1993)

#38

Post by bbor »

aithrowawaycomm wrote: post #34

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter. I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now. 1) A nitpick I've had since grad school: the answer to "can we cre…

If you're ever stuck wondering why a bunch of smart, motivated people with no clear corrupting motivations are being idiotic, that's a strong heuristic that you should spend a bit more time analyzing the issue, IMO ;). "Ugh, why is everyone else so stupid" is a common take for undergrad engineers, but I'm sure you've grown out of it in other ways. Anyway, more substantively:

The simple answer is that people have thought about it in depth, most famously noted doomer Eliezer Yudkowsky in Intelligence Explosion Microeconomics (2013)[1] and its main citation, Irving John Good's Speculations Concerning the First Ultraintelligent Machine (1965)[2]. Another common citation that drops a bit of rigour in the name of approachability is Nick Bostrom's 2014 Superintelligence: Paths, Dangers, Strategies[3].

[ETA: to put it even simpler: a system that improves itself is a (the?) quintessential setup for exponential growth. E.g. compound interest]

For the time-bound, the most rigorous treatment of your concern among those three is in Section 3 of Yudkowsky's paper, "From AI to Machine Superintelligence". To list the headings briefly:

- Increased computational resources

- Communication speed

- Increased serial depth (i.e. working memory capacity)

- Duplicability (i.e. reliability)

- Editability (i.e. we know how computers work)

- Goal coordination (this is really just communication speed, again)

- Improved rationality (i.e. fewer emotions/accidental instincts getting in the way)

Let's drop "human" and "superhuman" for a minute, and just talk about "better computers". I'm assuming you're a software engineer. Don't you see how a real software dev replacement program could be an unimaginable gamechanger for software? Working 24/7, enhancing itself, following TDD perfectly every time, and never ever submitting a PR that isn't rigorously documented and reviewed? All of which only gets better over time, as it develops itself?

[1] https://intelligence.org/files/IEM.pdf

[2] https://vtechworks.lib.vt.edu/server/api/core/bitstreams/a5e...

[3] https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...

TL;DR May God have mercy on us all.

Re: The Coming Technological Singularity (1993)

#39

Post by nradov »

Mistletoe wrote: post #18
ta93754829 wrote: post #8

Earlier quoted context omitted.

we keep moving the goalposts, and that's not a bad thing. remember when Doom came out? How amazing and "realistic" we thought the graphics were? How ridiculous that seems now? We'll look back at ChatGPT4 the same way.

Or is ChatGPt4 4k TV, which is good enough for almost all of us and we are plateauing already? https://www.reddit.com/r/OLED/comments/fdc50f/8k_vs_4k_tvs_d...

For the work that I do, ChatGPT accuracy is still garbage. Like it makes obvious factual errors on very simple technical issues which are clearly documented in public specifications. I still use it occasionally as it does sometimes suggest things that I missed, or catch errors that I made. But it's far from "good enough" to send the output to co-workers or customers without careful review and correction.

I do think that ChatGPT is close to good enough for replacing Google search. This is, ironically, because Google search results have deteriorated so badly due to falling behind the SEO spammers and much of the good content moving off the public Internet.

Re: The Coming Technological Singularity (1993)

#40

Post by rnk »

Vernor Vinge introduces many fantastic ideas in his really excellent scifi book A Fire Upon the Deep. He has many fascinating concepts like what if somehow there are parts of the universe where you can go faster than the speed of light, and you would be smarter there, that's where the super intelligent beings go. Guess what, we humans live in the slow zone, you morons. Also there it's a ftl communication method that is like good old Usenet. There is (what looked credible to me) a fascinating set of multiple brain beings, thing like dogs where together 5 of them form one "intelligence" where the different personalities combine in interesting ways.

And I was sad to notice he died this year, aged 79. A real cs prof who wrote sci fi.

Post reply on HN