Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

Post reply on HN
Print view View original

71–80 of 181 posts

Re: The Coming Technological Singularity (1993)

#71

Post by ben_w »

amanaplanacanal wrote: post #61
auggierose wrote: post #55

Earlier quoted context omitted.

I don't think it makes sense to include all of that in the calculation. A human doesn't need all of that, they just need 900 calories. You can just eat berries, no need to cook anything, for example.

A human cannot live on just berries. Even if they could, where are those berries gonna come from?

"Where" is easy.

I think more importantly is that uncooked food is harder to digest, so we need more of it.

Most years I pick free blackberries growing wild in the city. Won't scale to all of us, not seasonal, and I'd need to eat 6kg of them a day for my RDA of calories, and 12x my RDA of dietary fibre sounds unwise, but that kind of thing is how we existed before farming.

http://www.wolframalpha.com/input/?i=6kg%20blackberries

Re: The Coming Technological Singularity (1993)

#72

Post by Terr_ »

Animats wrote: post #20
WillAdams wrote: post #6

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

- could actually do useful work reliably with minimal supervision That's the big problem. LLMs can't be allowed to do anything important without supervision. We're still at 5-10% totally bogus results.

I think a deeper issue is that they are essentially "attempt to extend this document based on patterns in documents you've already seen" engines.

Sometimes that's valuable and exactly what you need, but problems arise when people try to treat them as some sort of magical oracle that just needs to be primed with the right text.

Even "conversational LLMs are just updating a theater-style script where one of the characters happens to be described as a computer.

Re: The Coming Technological Singularity (1993)

#73

Post by ben_w »

DiscourseFan wrote: post #41
p1esk wrote: post #25

Earlier quoted context omitted.

fairly clear what they can and can't do It’s not at all clear what the next gen models will do (e.g. gpt5). Might be enough to trigger mass unemployment. Or not.

Didn't OpenAI just cut its AGI department?

AGI readiness team. With the department leader giving the statement that nobody is ready.

Unless there's been another one since then? It's getting to be a bit of a blur.

Re: The Coming Technological Singularity (1993)

#74

Post by Terr_ »

Teever wrote: post #52
JSteph22 wrote: post #49

Earlier quoted context omitted.

>There are still some big pieces missing. The most glaring one is that current LLMs are many, many orders of magnitude away from working on the equivalent of 900 calories per day of energy.

I think you're correct that the energy efficiency of a human exceeds that of current computers, but I think it's a bit more complicated than a first order calorie count. How many joules go into producing those 900 calories? Like in terms of growing the food, from fertilizer production to tractor fuel, to feeding the farmer, to shipping the food, packaging it, storing it at the appropriate temperature, the ratio of sp…

> How many joules go into producing those 900 calories?

I think what you're getting at is question about conversion efficiencies, from solar radiation to whatever the computing-machine needs.

However, your description seems to risk double-counting things: You can't just sum up the inputs of each step, because (most of) the same energy is flowing onwards.

Re: The Coming Technological Singularity (1993)

#75

Post by ben_w »

cloudking wrote: post #4
crackalamoo wrote: post #3

> I'll be surprised if this event occurs before 2005 or after 2030. I'm not truly confident AGI will be achieved before 2030, and less so for ASI. But I do think it is quite plausible that we will achieve at least AGI by 2030. 6 years is a long time in AI, especially with the current scale of investment.

What is AGI and ASI? I think a fundamental issue here is both are sci-fi concepts without a clear agreement on the definitions. Each company claiming to work towards "AGI" has their own definition. How will someone claim they've achieved either, if we can't agree on the definitions?

Indeed.

The definition of AGI that OpenAI uses (or used) was of economic relevance. The one I use would encompass the original ChatGPT (3.5)*. I've seen threads here that (by my reading) opine that AGI is impossible because the commentor thinks humans can violate Gödel's incompleteness theorem (or equivalent) and obviously computers can't.

ASI is easier to discuss (for now), because it's always beyond the best human.

* weakly intelligent, but still an AI system, and it's much more general than anything we had before transformers were invented.

Re: The Coming Technological Singularity (1993)

#76

Post by gary_0 »

aithrowawaycomm wrote: post #34

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter. I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now. 1) A nitpick I've had since grad school: the answer to "can we cre…

> I sincerely have no idea why people believe so strongly that a human-level AI can build a superhuman AI. ...how on earth would such an AI even know it succeeded?

This touches on one of the few good reasons to be less ardent about AI/AGI: "intelligence" is not very well-defined and we don't have very good ways of measuring it. I don't think this is a total blocker, but it might present difficulties. What if our current approach ends up creating super-autism instead of super-intelligence? There's a long history (starting with Asimov) of drilling down into how vague things get when you start trying to draw clean lines around AI and its implications and those questions are yet to be definitively answered.

However, your broader point seems to imply that you can't "bootstrap" intelligence, which I don't find convincing. Humans, after all, could barely master fire a few hundred thousand years ago, and now we have an understanding of the universe that the earliest humans were incapable of comprehending on even a basic level. It's obvious to me that simpler things are capable of building more complex things; blind evolution can do it, so there's nothing in physics preventing intelligence bootstrapping. We also have the ability to use intellectual division of labor to build tools that vastly enhance our abilities as a species. The human brain as hardware is far from some impassable apex; hardware can always be used to build better hardware, much like the earliest CPUs were themselves used to design better CPUs.

Re: The Coming Technological Singularity (1993)

#77

Post by keiferski »

rsaarelm wrote: post #51
trescenzi wrote: post #28

Earlier quoted context omitted.

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

So you're saying that it's naive to suppose that everybody being much smarter than they are now would transform society, because any wide-scale societal change requires ongoing social cooperation between the many average-intelligence people society currently consists of?

Here’s a simpler way to put it: intelligence and social cooperation are not the same thing. Being good at math or science doesn’t mean you understand how to organize complex political groups, and never has.

People tend to think their special gift is what the world needs, and academically-minded smart people (by that I mean people that define their self-worth by intelligence level) are no different.

Re: The Coming Technological Singularity (1993)

#78

Post by rsaarelm »

keiferski wrote: post #77
rsaarelm wrote: post #51

Earlier quoted context omitted.

So you're saying that it's naive to suppose that everybody being much smarter than they are now would transform society, because any wide-scale societal change requires ongoing social cooperation between the many average-intelligence people society currently consists of?

Here’s a simpler way to put it: intelligence and social cooperation are not the same thing. Being good at math or science doesn’t mean you understand how to organize complex political groups, and never has. People tend to think their special gift is what the world needs, and academically-minded smart people (by that I mean people that define their self-worth by intelligence level) are no different.

Yes, because you need to spend a lot of time doing social organization and thinking about it to get very good at it, just like you need to spend a lot of time doing math or science and thinking about it to get very good at it. And then you need to pick up patterns, respond well to unexpected situations and come up with creative solutions on top of that, which requires intelligence. If you look at the people who are the best at doing complex political organization, they'll probably all have above-average intelligence.

Re: The Coming Technological Singularity (1993)

#79

Post by epcoa »

ben_w wrote: post #67
epcoa wrote: post #62

Earlier quoted context omitted.

“An aligned AI would not want to do these things (and if we have an unaligned AI, we’re back to talking about risks).” An aligned AI is not AGI, or whatever they want to call it.

> An aligned AI is not AGI, or whatever they want to call it. There's a few ways I can interpret that. If you mean "alignment and competence are separate axies" then yes. That's well understood by the people running most of these labs. (Or at least, they know how to parrot the clichés stochastically :P) If you mean "alignment precludes intelligence", then no. Consider a divisive presidential election between Alice an…

Campaign team members and even candidates switch teams often enough. Weak analogy. What is the alignment of human GI, completely generalized?

Re: The Coming Technological Singularity (1993)

#80

Post by corimaith »

rsaarelm wrote: post #51
trescenzi wrote: post #28

Earlier quoted context omitted.

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

So you're saying that it's naive to suppose that everybody being much smarter than they are now would transform society, because any wide-scale societal change requires ongoing social cooperation between the many average-intelligence people society currently consists of?

The prisoner's dilemma is a well known example of how rationality fails. To overcome requires something more than intelligence, it requires a predisposition to cooperation, to trust, in faith. Some might say that is what seperates Wisdon from Knowledge.
Post reply on HN