Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

41–50 of 181 posts

Re: The Coming Technological Singularity (1993)

#41
post #25
post #19

We still don't have squirrel-level AI. This is embarrassing. Now that LLMs have been around for a while, it's fairly clear what they can and can't do. There are still some big pieces missing. Like some kind of world model.

fairly clear what they can and can't do It’s not at all clear what the next gen models will do (e.g. gpt5). Might be enough to trigger mass unemployment. Or not.

Didn't OpenAI just cut its AGI department?

Re: The Coming Technological Singularity (1993)

#42

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

  if a computer system were able to design a better computer system, how much would it cost to then manufacture said system?
I think the implication is that the primary advancements would come in the form of software. IMO it's trivially true that we're not taking full advantage of the hardware we have from a software PoV -- if we were, we wouldn't need SWEs, right? From that it should follow that self-improving software is dangerously effective.

  once this new computer is running, how much power does it require? What are the on-going costs to keep it running? 
I mean, lots, sure. But we allocate immense resources to relatively trivial luxuries in this world; I don't think there's any reason to think we can't spare some giant computers to rapidly advance our technology. In a capitalist society, it's happily/sadly pretty much guaranteed that people will figure out how to get the resources there if scientists tell them the RoI is infinity+1.

  I'd be satisfied with a Large-Language-Model which
Those are great asks and I agree, but just to be super clear in case its not: Vinge isn't talking about chatbots, he's talking about systems with many smaller specialized subsystems. In today's parlance, gaggle of "LLMs" equipped with "tool use", or in yesterday's parlance, a "Society of Mind".

Re: The Coming Technological Singularity (1993)

#43
post #30
post #24

Earlier quoted context omitted.

Yup things look bleak but its not a static world. For everything that happens there is a reaction. It builds with time. But to find the right reaction also takes time. This is the Explore part in the Tradeoff. AI will be applied there not just on the Exploit front. What you are alluding too is Media/Social Medias current architecture and how it captures and steals peoples attention. Totally on the Exploit end of the…

Eh, I’m not talking about people’s opinions. I’m talking about evolutionary functions, and how much more likely it is to prefer something that has fun and just looks like it’s doing something, instead of actually doing something. Aka manipulation vs actual hard work. Do you have any concrete proposals, besides ‘it will get better’? Actual competency is hard. Faking it is usually way easier. It’s the same reason the ‘…

You could ask FDR and Churchill that after the fall of France and it wouldnt be too useful what they said cuz it took them almost 3 years before they openly said victory = end of the nazis and nothing else.

So dont just sweep the fact that things take Time under the carpet. Its not healthy cause its like looking at tree shoots in the ground and saying but why does that not look like a tree yet.

Finding gold in an unexplored jungle takes much longer than extracting gold from an existing mine. This is the Explore Exploit tradeoff. Exploit is easy. More ppl do it. Explore is hard. And takes more time. If AI shifts the balance on explore the story changes.

If you want to talk about Explore in Media/attention (mis)allocation you can already see the appearance of green shoots in the ground. There are multiple things going on parallely.

First there is a realization that Attention is finite and doesnt grow while Content keeps exploding. Totally unsustainable to the point the UN has published a report about the Attention Economy. This doesnt happen without people reacting and going into explore mode for solutions.

They are already talking about how to shift these algos/architectures based on Units of Time spent consuming(Exploit) to Value derived from time spent.

Giving people feedback on how their time is being divided between consumption(entertaimment) and value. Then allowing then to create schedules. What you now start seeing as digital wellbeing tech.

There are now time based economic models where platform doesnt just assume time spent is free but something the platform needs to pay for. People are experimenting with rewards micropayments. All these are examples of explore mode being activated.

There is also realization that content discovery on centralized platforms like youtube tiktok insta cause homogenity in what eveeyone upvotes. So you see people reacting and decentralizing to protect and preseeve niches. AI(curator of curators) will play a big role in finding such niche that fit your needs.

Will just end with people are also realizing there is huge misallocation of Ambition/Drive problem. Anthony Bourdain says Life is Good in every show od his and then kills himself. Shaq says he has 40 cars but doesnt know why. Since media(society's attention allocator) has tied success to wealth/status accumulation, conspicuous consumption/luxury/leisure etc. People end up in these kind of traps. So now we are seeing reactions, esp with climate change/sustainability that ambition and energy have to be shown other paths. Lot of changes in advertiaing and media companies around it. All are explore mode functions.

Re: The Coming Technological Singularity (1993)

#44
post #35

Earlier quoted context omitted.

I do think one of the major weaknesses of “smart people” is they tend to think of intelligence as the key aspect of basically everything. Reality is though we have plenty of intelligence already. We know how to solve most of our problems. The challenges are much more social and our will as a society to make things happen.

There’s a very big difference between knowing “how” to solve a problem in a broad sense, eg “if we shared more we could solve hunger”, and “how” to solve it in terms of developing discrete, detailed procedures that can be passed to actuators (human, machines, institutions) and account for any problems that may come up along the way. Sure, there are some political problems where you have to convince people to comply.…

100,000 Einsteins in your basement would be amazing. You'd have major breakthroughs in many fields. But at some point the gains will be marginal. All the problems solvable by shear intellectual labor will run dry, and you'll be blocked on everything else.

Re: The Coming Technological Singularity (1993)

#45

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter. I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now. 1) A nitpick I've had since grad school: the answer to "can we cre…

My dumb guy take on it: suppose we build a human-level AGI and it turns out to be limited by compute and memory. Those being limiting factors don’t seem at all far-fetched to me; it seems unlikely that the first real-time AGI will be mostly idling its CPUs. So then wait 18 months and run that same program on a machine that’s this year’s model plus a Moore’s Law doubling. You’ve probably got ASI. Right?

Re: The Coming Technological Singularity (1993)

#46
post #25
post #19

We still don't have squirrel-level AI. This is embarrassing. Now that LLMs have been around for a while, it's fairly clear what they can and can't do. There are still some big pieces missing. Like some kind of world model.

fairly clear what they can and can't do It’s not at all clear what the next gen models will do (e.g. gpt5). Might be enough to trigger mass unemployment. Or not.

Bigger LLM models probably won't fix the underlying problems of hallucinations, lack of a confidence metric, and lack of a world model. They just do better at finding something relevant on already-solved problems.

Re: The Coming Technological Singularity (1993)

#47
Related. Others?

The Coming Technological Singularity (1993) - https://news.ycombinator.com/item?id=35617100 - April 2023 (169 comments)

The coming technological singularity: How to survive in the post-human era [pdf] - https://news.ycombinator.com/item?id=35184764 - March 2023 (2 comments)

The Coming Technological Singularity: How to Survive in the PostHuman Era (1993) - https://news.ycombinator.com/item?id=34456861 - Jan 2023 (1 comment)

The Coming Technological Singularity (1993) - https://news.ycombinator.com/item?id=11278248 - March 2016 (8 comments)

The Coming Technological Singularity (original essay on the Singularity, 1993) - https://news.ycombinator.com/item?id=823202 - Sept 2009 (1 comment)

The original singularity paper - https://news.ycombinator.com/item?id=624573 - May 2009 (17 comments)

Re: The Coming Technological Singularity (1993)

#48
post #36
post #35

Earlier quoted context omitted.

There’s a very big difference between knowing “how” to solve a problem in a broad sense, eg “if we shared more we could solve hunger”, and “how” to solve it in terms of developing discrete, detailed procedures that can be passed to actuators (human, machines, institutions) and account for any problems that may come up along the way. Sure, there are some political problems where you have to convince people to comply.…

An AI-driven corporation wouldn't be able to surpass anyone doing it the old way because they'd still have to wait for building permits and inspections.

Permits and inspections might be the the reason for humanities downfall then, at what point does war become the more efficient option?

Re: The Coming Technological Singularity (1993)

#49
post #19

We still don't have squirrel-level AI. This is embarrassing. Now that LLMs have been around for a while, it's fairly clear what they can and can't do. There are still some big pieces missing. Like some kind of world model.

>There are still some big pieces missing.

The most glaring one is that current LLMs are many, many orders of magnitude away from working on the equivalent of 900 calories per day of energy.

Re: The Coming Technological Singularity (1993)

#50

The thing which these discussions leave out are the physical aspects: - if a computer system were able to design a better computer system, how much would it cost to then manufacture said system? How much would it cost to build the fabrication facilities necessary to create this hypothetical better computer? - once this new computer is running, how much power does it require? What are the on-going costs to keep it run…

[deleted]
Post reply on HN