We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.
Humanity isn't ready for the coming intelligence explosion
211–220 of 524 posts
Re: Humanity isn't ready for the coming intelligence explosion
#212We don't need to live "side by side" with AI. AI is not alive, it's a technology we use. This is like talking about living side by side with your toaster.
Re: Humanity isn't ready for the coming intelligence explosion
#213Earlier quoted context omitted.
Here is an issue I think about often, but I am not quite sure how to put it into words. We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this? I don't just mean climate, but business planning, health, risk assessment…
The free intelligence may help to steer the society and politicians to one or the other side. The only question is who decides which.
But this time it's absolutely different, right? Riiight?..
Re: Humanity isn't ready for the coming intelligence explosion
#214How far are we from reliable AI? One that could, for example, handle 80% of office jobs without screwing up more than humans?
Re: Humanity isn't ready for the coming intelligence explosion
#215Earlier quoted context omitted.
Why is AI not alive
You might be trolling, but I like Schrödingers definition of life: living things avoid decay into equilibrium. Inference engines are trying to converge any input to a certain limit, and I presume that's close enough to an equilibrium to not count as living thing.
Re: Humanity isn't ready for the coming intelligence explosion
#216Earlier quoted context omitted.
"but nobody would argue that an asteroid is alive" Many animists would disagree: https://en.wikipedia.org/wiki/Animism
TIL, thanks. Holy fuck we humans are crazy sometimes.
Re: Humanity isn't ready for the coming intelligence explosion
#217Earlier quoted context omitted.
"Alive" is irrelevant. Bullshit like "alive", "sentient" or "conscious" doesn't decide much, at the day's end. It just distracts from the thing that matters. And the thing that matters is: capability. Even today's AIs are capable of autonomous goal-oriented agentic behavior - and growing more and more so with every release. At sufficient capability, AI stops being "a technology we use", and becomes a force in itself…
DNNs/LLMs can only predict next tokens based on training data. They often make big direction mistakes as they are particularly bad at common sense. Kind of like the Paperclip Maximizer scenario. They need a human with deep knowledge to drive them and to catch them when they go off the rails.
And with what we know of LLMs? Autoregressive transformers are Turing complete in theory, and we are yet to find anything that LLMs are "fundamentally incapable" of in practice. Even continuous learning is already approximated with in-context learning - both allow a system to learn from prior experience, both have practical limits on how far they go. That's what powers "trial and error" in today's agentic LLMs.
"LLMs can only predict next tokens based on training data" is comforting but misleading. It just isn't the saving grace you want it to be. It describes an interface, not a ceiling. And if there is some sort of fundamental "capability ceiling" that LLMs are heading towards, we are yet to see it. We know plenty of things LLMs are bad at, but they keep getting less bad at them release to release.
If there is none, then, simply improving over the current recipes iteratively might yield systems that only "need a human" in the same way you "need" to have a boss. Maybe less so.
Re: Humanity isn't ready for the coming intelligence explosion
#218Earlier quoted context omitted.
Why is AI not alive
It has no body, metabolism or reproductive capability. It's a process, not an entity.
Re: Humanity isn't ready for the coming intelligence explosion
#219Earlier quoted context omitted.
No, that’s not accurate. I’m not sure what to tell you. It’s like hearing that no serious programmer uses Python, it’s so far from my experience and that of everyone I know in the field that I don’t know how to engage.
Do you think that everyone in the Linux kernel team write their code using AI only? What about Emacs? And cURL?
> I used Claude Code and Codex for the translation. This was human-directed, not autonomous code generation. I decided what to port, in what order, and what the Rust code should look like. It was hundreds of small prompts, steering the agents where things needed to go. After the initial translation, I ran multiple passes of adversarial review, asking different models to analyze the code for mistakes and bad patterns.
Sounds to me like "wrote their code using AI only".
And from the creator of Redis, antirez: https://antirez.com/news/158
> It is simply impossible not to see the reality of what is happening. Writing code is no longer needed for the most part.
Re: Humanity isn't ready for the coming intelligence explosion
#220I couldn't read the whole article, but just from the part I could read: > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories. I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a m…
> an incentive to make their technology seem transformative and powerful I have no frame of reference to process this. Humans species perhaps 300,000 years, we are essentially the same. Transistor, 79 years, explosive growth in numbers and power. Integrated circuit, 68 years, explosive growth in numbers and power. "Attention is all you need", 9 years. ChatGPT, 4 years, explosive growth in instances and power. Humans…
However - 'Humans species perhaps 300,000 years, we are essentially the same' is an enormous misconception. That fact that we lack gross anatomical changes during this period, ignores everything we now know about punctuated equilibrium and rapid evolution. It's highly probably we've had an enormous number of evolved psychological changes during the last few hundred, and even tens of thousands of years. Changes that relate to our capacity to live in large groups, adapt to urban environments, resist disease and so on. We know that's the case simply because acute pandemics become epidemics through herd immunity, and through the acquisition of lactose tolerance etc.
It seems highly unlikely that adaptations stop there. Altering the environment (in the last 10K years that means the built environment) alters the environment of evolutionary adaptiveness. It seems likely that we've essentially domesticated out much of our propensity for violence and increased our capacity for mood regulation.
Obviously it's incredibly tricky to pair these specific behavioural changes to genetic changes -> protein synthesis -> behaviour. Bearing in mind though we're only 20 years out from the first study to link allele variant to behaviour (the COMT Val/Met polymorphism), and the potential controversy around such research, this shouldn't be surprising.