Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

Post reply on HN
Print view View original

51–60 of 181 posts

Re: The Coming Technological Singularity (1993)

#51

Post by rsaarelm »

trescenzi wrote: post #28
samsartor wrote: post #15

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting…

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

So you're saying that it's naive to suppose that everybody being much smarter than they are now would transform society, because any wide-scale societal change requires ongoing social cooperation between the many average-intelligence people society currently consists of?

Re: The Coming Technological Singularity (1993)

#52

Post by Teever »

JSteph22 wrote: post #49
Animats wrote: post #19

We still don't have squirrel-level AI. This is embarrassing. Now that LLMs have been around for a while, it's fairly clear what they can and can't do. There are still some big pieces missing. Like some kind of world model.

>There are still some big pieces missing. The most glaring one is that current LLMs are many, many orders of magnitude away from working on the equivalent of 900 calories per day of energy.

I think you're correct that the energy efficiency of a human exceeds that of current computers, but I think it's a bit more complicated than a first order calorie count.

How many joules go into producing those 900 calories? Like in terms of growing the food, from fertilizer production to tractor fuel, to feeding the farmer, to shipping the food, packaging it, storing it at the appropriate temperature, the ratio of spoiled food to actually consumed, the energy to cook it, all of that isn't counted in that simple 900 calorie measurement.

I've been thinking about this for a while now but I haven't been able to quantify it so maybe someone reading this comment can help.

Re: The Coming Technological Singularity (1993)

#53

Post by ben_w »

WillAdams wrote: post #6

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

> if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer?

Most of the computers we use today were designed by software: Feature sizes are (and have been for some time) in the realm where the Schrödinger equation matters, and more compute makes it easier to design smaller feature sizes.

Similar points apply to the question of cost: it has not been constant, the power to keep x-teraflops running has decreased* while the cost to develop the successor has increased.

Regarding LLMs in particular, I believe there are already models meeting all but one of your criteria — though I would argue that the missing one, "could actually do useful work reliably with minimal supervision", is by far the most important.

* If I read this chart right, my phone beats the combined top 500 supercomputers when the linked article was written by a factor of ten or so: https://commons.m.wikimedia.org/wiki/File:Supercomputers-his...

Re: The Coming Technological Singularity (1993)

#54

Post by bryanrasmussen »

rsaarelm wrote: post #51
trescenzi wrote: post #28

Earlier quoted context omitted.

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

So you're saying that it's naive to suppose that everybody being much smarter than they are now would transform society, because any wide-scale societal change requires ongoing social cooperation between the many average-intelligence people society currently consists of?

I think they're saying adequate intelligence to solve all problems is already here, it just isn't evenly distributed yet - and never will be.

Re: The Coming Technological Singularity (1993)

#55

Post by auggierose »

Teever wrote: post #52
JSteph22 wrote: post #49

Earlier quoted context omitted.

>There are still some big pieces missing. The most glaring one is that current LLMs are many, many orders of magnitude away from working on the equivalent of 900 calories per day of energy.

I think you're correct that the energy efficiency of a human exceeds that of current computers, but I think it's a bit more complicated than a first order calorie count. How many joules go into producing those 900 calories? Like in terms of growing the food, from fertilizer production to tractor fuel, to feeding the farmer, to shipping the food, packaging it, storing it at the appropriate temperature, the ratio of sp…

I don't think it makes sense to include all of that in the calculation. A human doesn't need all of that, they just need 900 calories. You can just eat berries, no need to cook anything, for example.

Re: The Coming Technological Singularity (1993)

#56

Post by Bengalilol »

dartos wrote: post #23
crackalamoo wrote: post #21

Earlier quoted context omitted.

I don't think we're at a plateau. There's still a lot GPT-4 can't do. Given the progress we've seen so far with scaling, I think the next iterations will be a lot better. It might even take 10 or even 100x scale, but with increased investment and better hardware, that's not out of the question.

I thought we’ve seen diminishing returns on benchmarks with the last wave of foundation models. I doubt we’ll see a linear improvement curve with regards to parameter scaling.

And now we have the LLMs self feeding their models (which may be either good or bad). This shouldn’t lead to short-term wide (as in AGI) efficiency. I bet this is a challenge.

Re: The Coming Technological Singularity (1993)

#57

Post by ascorbic »

trescenzi wrote: post #28
samsartor wrote: post #15

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting…

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

Dario Amodei's recent post had a good analysis about which fields are and are not limited by intelligence.

https://darioamodei.com/machines-of-loving-grace

Re: The Coming Technological Singularity (1993)

#58

Post by szundi »

samsartor wrote: post #15

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting…

Skynet will build consensus killing people until only one remains. It agrees with himself isn’t it? Oh, he expressed some concerns about the result? Sadly the instance is faulty recognizing the obvious, terminated.

Re: The Coming Technological Singularity (1993)

#59

Post by trashtester »

samsartor wrote: post #15

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection. I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting…

AlphaFold/AlphaProteo are direct examples of how AI can allow us to bypass experiments when doing science.

More importantly, though, is that insufficiently powerful AI has been a limiting factor in robotics. That seems to be coming to an end now. And once we have humanoid robots powered by superhuman intelligence entering the workforce, the impact will be massive.

Quite possibly mostly for the bad for most people.

Re: The Coming Technological Singularity (1993)

#60

Post by oezi »

JSteph22 wrote: post #49
Animats wrote: post #19

We still don't have squirrel-level AI. This is embarrassing. Now that LLMs have been around for a while, it's fairly clear what they can and can't do. There are still some big pieces missing. Like some kind of world model.

>There are still some big pieces missing. The most glaring one is that current LLMs are many, many orders of magnitude away from working on the equivalent of 900 calories per day of energy.

900 kcal/day ~ 50 Watts

It is more than phones/laptops consume.

Certainly, we can only run small LLMs on such edge devices, but we getting to the level of compute efficiency that output indeed is comparable.

Post reply on HN