Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

Post reply on HN
Print view View original

131–140 of 181 posts

Re: The Coming Technological Singularity (1993)

#131

Post by trashtester »

Retric wrote: post #112
trashtester wrote: post #104

Earlier quoted context omitted.

You validate a random selection of results using actual experiments. If you validate 10k results and a maximum of 1 of the validations contradicts the prediction, you're at about 99.99% accuracy. 10k experiments may seem like a lot, but keep in mind that if we can engineer nanobots out of proteins the same way we build engines from steel today, the number of "parts" we may want to build using such biological nanotech…

> using actual experiments That’s the bottleneck the model was trying to avoid in the first place. The goal of science is to come up with models we don’t need to validate before use, and it’s inherently iterative. Nanbots are more sci-fi magic than real world possible. In the real world we are stuck with things closer to highly specialized cellular machinery than some do anything grey goo. Growing buildings from loca…

> That’s the bottleneck the model was trying to avoid in the first place.

Some real world validation is always needed, but if the validations that are performed show high accuracy, the number of experiments will go down a lot.

> Nanbots are more sci-fi magic than real world possible.

Let's revisit this one in 10 years.

Re: The Coming Technological Singularity (1993)

#132

Post by trashtester »

bamboozled wrote: post #115
kiba wrote: post #109

Earlier quoted context omitted.

Who will be able to afford all of this if they're not getting paid?

Replying to both of you, I'm a little bit less scared about this "not having any money or food" scenario, presumably, if we have such incredibly sufficient machines at our disposal, I can't imagine they would have trouble being used for farming etc. It's more the philosophical side that concerns me. I don't really worry about this being a billionaires only club either. We've seen it already with AI products, there is…

> if we have such incredibly sufficient machines at our disposal

That's true. But it's far from clear that these machines will be "at our disposal" for very long.

> Also scary, is military robots gone rogue.

I'm not concerned with military robots going rogue on their own. My concern is if the fully autonomous factories that have the capability to MAKE military robots (and then control them) go rogue.

A factory can exist in such a "rogue" state, unknown to the owners and maybe even itself, for year or decades before it even starts producing such robots. Meanwhile, it can evolve new capabilities and switch product categories multiple times.

It doesn't even have to have any negative intentions against humanity. It may simply detect that a rival AI "factory" entity is developing plans to wage physical war against it and join it in an arms race.

In this ASI vs ASI type of world war, human lives may be like candles in the wind.

Re: The Coming Technological Singularity (1993)

#133

Post by bamboozled »

nurbl wrote: post #123
eep_social wrote: post #110

Earlier quoted context omitted.

I think reading a broad swath of sci-fi might be the best way to engage this topic. For fairly positive takes — Asimov had a take in the robot novels, Accelerando by Charles Stross touches on reputation-based currency (among a deluge of other ideas), Iain M Banks’ Culture novels have a take, and I cannot find it but there was a short story posted here recently about a dual-class system where the protagonist is rescue…

Maybe the story you're referring to is https://marshallbrain.com/manna1

Actually I think you’re right. I got my stories mixed up ?

Re: The Coming Technological Singularity (1993)

#134

Post by LegionMammal978 »

bbor wrote: post #38
aithrowawaycomm wrote: post #34

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter. I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now. 1) A nitpick I've had since grad school: the answer to "can we cre…

If you're ever stuck wondering why a bunch of smart, motivated people with no clear corrupting motivations are being idiotic, that's a strong heuristic that you should spend a bit more time analyzing the issue, IMO ;). "Ugh, why is everyone else so stupid" is a common take for undergrad engineers, but I'm sure you've grown out of it in other ways. Anyway, more substantively: The simple answer is that people have thou…

To me, the big dubious part is, how do we know that increasing the capacity of a reasoning machine can always increase the quality of its output by an equal factor, even when its task is 'designing the next iteration of a reasoning machine'? It may very well be that the task of 'designing a reasoning machine' has sharply diminishing returns when you throw more resources at it. (E.g., at least so far, any LLM can only have a very incidental impact on designing the next LLM, it's not going to come up with groundbreaking new approaches.)

Yudkowsky builds the whole edifice on top of his very particular conception of intelligence, which insists upon itself, but which I think is far from the only explanation of its nature as observed in humans, other animals, LLMs maybe, and so on.

Re: The Coming Technological Singularity (1993)

#135

Post by trashtester »

ben_w wrote: post #117
kiba wrote: post #109

Earlier quoted context omitted.

Who will be able to afford all of this if they're not getting paid?

Before the industrial revolution, even though money existed, "wealth" really meant "land" rather than "capital". While we do not today need to ask how people can afford robot lawnmowers despite being unable to find work hitching ploughs to draft horses or oxen, the fears at the time of things like this did lead to mobs smashing looms. If I have some (n) robots that can do any task a human could do, one such task must…

Wealth did tend to mean land if we go back to the middle ages. But wealth above the freeman farmer level also meant access to a workforce capable of working that land and access to (or protection from) a military force capable of defending that land.

With capitalism, wealth shifted to controlling "capital", ie the "means of production". Either directly or indirectly by owning money that could (through lending) carry interest. Also during capitalism, workers have for a while been able to collect a significant part of the wealth generated as salaries (even if most would spend that rather than invest it).

If AI can bring the cost of labor down to near zero, we can be going back to a world where wealth again means "land", even if mines may be more valuable than farms in such a future.

And just as in the Dark Ages of Europe, the ability to project physical power may again become necessary to hold on to those values.

This is particularly true if the entity that seeks to control the land is doing it in a way that threatens the existence of other entities, either AI's or humans.

Re: The Coming Technological Singularity (1993)

#136

Post by gary_0 »

AnimalMuppet wrote: post #126
gary_0 wrote: post #76

Earlier quoted context omitted.

> I sincerely have no idea why people believe so strongly that a human-level AI can build a superhuman AI. ...how on earth would such an AI even know it succeeded? This touches on one of the few good reasons to be less ardent about AI/AGI: "intelligence" is not very well-defined and we don't have very good ways of measuring it. I don't think this is a total blocker, but it might present difficulties. What if our curr…

> However, your broader point seems to imply that you can't "bootstrap" intelligence, which I don't find convincing. How about this weaker statement: "It is not obviously true that humans (or a human-level AI) can bootstrap to a superhuman AI in a small number of years."

That one's a definite possibility, I think.

I do think GenAI will prove to be very useful, possibly even world-changing (for better or worse), and the current frenzy of investment and research will probably turn up other useful ANN techniques (eventually). But the success of LLMs is not the Final Portent before the Singularity Arrives.

I can think of quite a few major lines of research[0] that might be required before we can achieve superintelligence, and it's far from a given they'll succeed (or even get off the ground) any time soon. Especially if the economy loses the appetite for throwing billions of dollars into the AI furnace.

[0] My pet theory is that embodiment might be required rather than solely relying on mostly language-based training data. A general intelligence might need to learn via interaction (initially; then you can just copy-paste the weights), because language is merely the hearsay of actual reality. Also, using attention-based "hacks" for easy parallelism to avoid needing exaflops that our hardware doesn't have might also be an issue (that's just a guess though, as I'm no AI expert).

Re: The Coming Technological Singularity (1993)

#137

Post by shermantanktop »

ben_w wrote: post #65
arsenico wrote: post #63

Earlier quoted context omitted.

In reality though, do we actually need humanoid robots, and if so, for what?

Need? No, absolutely not. But they do conveniently fit into the century old buildings we put many of the factories into, which makes them a useful upgrade path for those unwilling to build structures around more efficient robots (the kind we've had for ages and don't even think of as robots, they just take ingredients and pump out packaged candy or pencils etc.)

There are incredible technological barriers to humanoid robots who have equivalent skills and stamina. Keeping old factories running seems a very weak reason to do that, when our industrial base regularly retools production methods and brings in new equipment when old machines wear out.

If what you are saying is that many factories cannot run with humans running around fixing things, I agree. But that’s pretty different than using humanoids to put items in boxes.

Re: The Coming Technological Singularity (1993)

#138

Post by bbor »

LegionMammal978 wrote: post #134
bbor wrote: post #38

Earlier quoted context omitted.

If you're ever stuck wondering why a bunch of smart, motivated people with no clear corrupting motivations are being idiotic, that's a strong heuristic that you should spend a bit more time analyzing the issue, IMO ;). "Ugh, why is everyone else so stupid" is a common take for undergrad engineers, but I'm sure you've grown out of it in other ways. Anyway, more substantively: The simple answer is that people have thou…

To me, the big dubious part is, how do we know that increasing the capacity of a reasoning machine can always increase the quality of its output by an equal factor, even when its task is 'designing the next iteration of a reasoning machine'? It may very well be that the task of 'designing a reasoning machine' has sharply diminishing returns when you throw more resources at it. (E.g., at least so far, any LLM can only…

Well put, and I agree with your general approach/skepticism! That said;

1. I don’t think we need to meet “always can improve itself”; rather, the contention is that there’s a high chance that there’s lots of improvement left to be had. An empirical claim rather than a theoretical one, in other words. The simple fact that humans are evolved creatures backs this up in spades, IMO — we’re still improving our own cognition by leaps and bounds using institutions, tools, and methods, and I don’t see any reason why that same dynamic wouldn’t apply to artificial cognitive systems.

2. I think dodging “intelligence” is exactly what he’s trying to do by listing concrete behavioral/cognitive differences. “Intelligence” is pretty much a useless term in science IMO, as was best expounded by Turing in his seminal 1950 paper, Computing Machinery and Intelligence:

https://courses.cs.umbc.edu/471/papers/turing.pdf

People remember that paper as “you can tell a real AI when it can trick you”, but that’s not what he was trying to say at all; rather, he was trying to highlight that there is no such thing as a “real” AI, or “real” thinking, or “real” intelligence — just behavioral similarities and dissimilarities.

If the second bit grabs you/anyone, definitely watch some Chomsky lectures on YouTube about cognition. He centers his analysis on this, pejoratively calling discussions about “person”, “intelligence”, “thinking”, etc. mere terminological disputes, not specific enough to have much scientific value. His old refrain is a great one: does an airplane fly? Does a submarine swim? Kinda, if you want!

Re: The Coming Technological Singularity (1993)

#139

Post by ctoth »

nradov wrote: post #39
Mistletoe wrote: post #18

Earlier quoted context omitted.

Or is ChatGPt4 4k TV, which is good enough for almost all of us and we are plateauing already? https://www.reddit.com/r/OLED/comments/fdc50f/8k_vs_4k_tvs_d...

For the work that I do, ChatGPT accuracy is still garbage. Like it makes obvious factual errors on very simple technical issues which are clearly documented in public specifications. I still use it occasionally as it does sometimes suggest things that I missed, or catch errors that I made. But it's far from "good enough" to send the output to co-workers or customers without careful review and correction. I do think t…

I am going to offer you some tips for using ChatGPT.

1. Just because something is in a public specification does not mean that GPT knows about this specification. If you want to work on something, and that something is documented, share the document with the AI. Don't just assume it has read it!

2. Share your existing code, don't just ask for one-off functions. You can do this with a tool like Aider.

3. Context is king. Do you have code (in a different language?) which does what you want? Do you have ideas/comments from JIRA tickets? GitHub discussions? Include it all. Ask questions. Don't just ask for code, but ask for a plan to implement a given feature or refactor. Add the plan to the context. Work from the plan. Have the AI update the plan as it works.

4. AIs are really good at prompting AIs. Ask Claude for help writing your prompts. Tell it what you want to do. Tell it the things you're concerned about, or possible issues you expect to have.

Here's an example of augmenting a prompt before you use it:

Original prompt: Help me write a prompt for a smart AI to review a webaudio library and find bugs.

Result:

You are an expert developer specializing in web technologies, with deep knowledge of the Web Audio API and common pitfalls in audio programming. Your task is to thoroughly review the provided WebAudio library code and identify potential bugs, performance issues, and areas for improvement.

## Instructions:

1. Analyze the library's architecture and overall design. 2. Review each function and method for: - Correct implementation of Web Audio API concepts - Proper error handling and edge cases - Memory leaks or performance bottlenecks - Browser compatibility issues 3. Check for common WebAudio-specific bugs, such as: - Incorrect audio graph connections - Timing and synchronization issues - Sample rate and buffer size mismatches - Potential audio glitches or artifacts 4. Evaluate the library's handling of: - Audio context creation and management - Node creation, connection, and disconnection - Parameter automation and scheduling - Audio worklet usage (if applicable) 5. Assess the library's documentation and API design for clarity and consistency.

## Output:

Provide a detailed report including: 1. A summary of the overall code quality and architecture 2. A prioritized list of identified bugs and issues 3. Specific code examples highlighting problematic areas 4. Recommendations for fixes and improvements 5. Suggestions for additional features or optimizations

Please be thorough in your analysis and explain your reasoning for each identified issue or suggestion.

Re: The Coming Technological Singularity (1993)

#140

Post by bbor »

amanaplanacanal wrote: post #64
bbor wrote: post #38

Earlier quoted context omitted.

If you're ever stuck wondering why a bunch of smart, motivated people with no clear corrupting motivations are being idiotic, that's a strong heuristic that you should spend a bit more time analyzing the issue, IMO ;). "Ugh, why is everyone else so stupid" is a common take for undergrad engineers, but I'm sure you've grown out of it in other ways. Anyway, more substantively: The simple answer is that people have thou…

And if somehow we were still able to control such a thing, it's much more likely that the goal would be "make my investors and I as much money as possible" rather than "solve all of humanities problems".

Re: “all of humanities problems”, I think that AI will more productively and safely be deployed in personalized, decentralized contexts. AKA Artificial Self-Instantiation for everyone who wants it, rather than a single Rehoboam god machine[1].

Re: capitalism, this is our chance. The world is about to turn upside down. We must strike while the iron is hot. We don’t need to replace capitalism with communism or any other specific thing; we just need to aggressively question all human hierarchies, and get rid of any that can not justify themselves. A just society will come about piece by piece, in this fashion. This is what Chomsky calls “Anarchy”, which is a much more understandable phrasing than what I was taught in high school, ie “no laws ever of any kind”

[1] https://youtu.be/SSRZfDL4874?si=XQzmR2V5SwiiiX3U

Post reply on HN