My biggest worry is that the development of AI will stop once people can no longer easily tell when the AI is wrong. Following its advice may become mandatory in certain aspects of life. But it will be quite not good enough, and give catastrophic advice, but the failures will be blamed on people who don't follow it correctly.
The Adolescence of Technology
131–140 of 159 posts
Re: The Adolescence of Technology
#132Some people say that human jobs will move to the physical world, which avoids the whole category of “cognitive labor” where AI is progressing so rapidly. I am not sure how safe this is, either. A lot of physical labor is already being done by machines (e.g., manufacturing) or will soon be done by machines (e.g., driving). Also, sufficiently powerful AI will be able to accelerate the development of robots, and then co…
> But in 2026 in the US only Waymo has highly autonomous driving systems, in only a few markets
10 years ago I predicted that the uptake of autonomous vehicles would be slow but that it would be because of labor protections. While those have had some impact, that isn't really the issue: it's that the cars just don't quite work well enough yet and that last ~20% of function turns out to be both incredibly difficult and incredibly important.
Re: The Adolescence of Technology
#133One of my formative impressions of AI came from the depiction of the Colligatarch from Alan Dean Foster's The I Inside. The AI in the book is almost feels like it is the main message masquerading as a subplot. Asimov knew the risks, and I had assumed until fairly recently that the lessons and explorations that he had imparted into the Robot books had provided a level of cultural knowledge of what we were about to fac…
My interpretation is that Asimov assumed that humans would require understanding at the deepest levels of artificial intelligence before it could be created. He built the robot concepts rooted in the mechanical world rather than the world of the integrated circuit. He never imagined, I suppose, that we would have the computing power necessary to just YOLO-dump the sum of all human knowledge into a few math problems a…
> He built the robot concepts rooted in the mechanical world
He was idealistic even at the time. The 3 Laws were written 30 years after some of the earliest robots were aiming artillery barrages at human beings.
Re: The Adolescence of Technology
#134Earlier quoted context omitted.
In the large, ideas can have a massive influence on what happens. This inevitability that you're expressing is itself one of those ideas. Shifts of dominant ideas can only come about through discussions. And sure, individuals can't control what happens. That's unrealistic in a world of billions. But each of us is invariably putting a little but of pressure in some direction. Ironically, you are doing that with your c…
How will this pressure add up and bubble to the sociopaths which we collectively allow to control most of the world's resources? It would need for all these billions to collectively understand the problem and align towards a common goal. I don't think this was a design feature, but globalising the economy created hard dependencies and the internet global village created a common mind share. It's now harder than ever…
> How will this pressure add up and bubble to the sociopaths which we collectively allow to control most of the world's resources?
By things like: https://en.wikipedia.org/wiki/Artificial_Intelligence_Act
and: https://www.scstatehouse.gov/sess126_2025-2026/bills/4583.ht... (I know nothing about South Carolina, this was just the first clear result from the search)
Re: The Adolescence of Technology
#135Earlier quoted context omitted.
Maybe I'm missing the point as well, but what did it do wrong? It seemed like you wanted to see if a search tool was working. It looked to see. It tried one search using on data source KJ and found no matches. Next question would be is the quote not in there, is there a mis-remembering of the quote or is their something wrong with the data source. It tries an easier to match quote and finds nothing, which it finds od…
What I was expecting is that it would pull up the KJV using the results returned from the wiki_source_search tool instead of going for a totally different translation and then doing a text match for a KJV quote
> I was expecting is that it would pull up the KJV using the results returned from the wiki_source_search
Did you tell it to do that?
Re: The Adolescence of Technology
#136Occasionally, I read these types of essays and get awfully depressed. As someone just starting out in the technology field (and I guess white-collar work in general), it feels as if I suddenly have no hope of ever living a fruitful and meaningful life. The means by which I could ever potentially earn a living are slowly destroyed. I do wonder if others in my age group ever feel the same, if basically everyone under 3…
Do you think people were more optimistic about the future during World War 2? Or after WW2 when everyone was worried about nuclear annihilation?
How about before that when your new baby had a 30% chance of death before age 5?
Before that, starvation, plague and war were always real things to worry about for the entirety of human history.
I think everyone reading this has the same problem of needing to figure out hedonics in order to appreciate what you do have instead of focusing on minuscule bullshit that you don't have.
Re: The Adolescence of Technology
#137Earlier quoted context omitted.
It's because few realize how downstream most of this AI industry is of Thiel, Eliezer Yudkowsky and LessWrong.com. Early "rationalist" community was concerned with AI in this way 20 years ago. Eliezer inspired and introduced the founders of Google DeepMind to Peter Thiel to get their funding. Altman acknowledged how influential Eliezer was by saying how he is most deserving of a Nobel Peace prize when AGI goes well (…
It is very weird to wonder, what if they're all wrong. Sam Bankman-Fried was clearly as committed to these ideas, and crashed his company into the ground. But clearly if out of context someone said something like this: "Clearly, the most obvious effect will be to greatly increase economic growth. The pace of advances in scientific research, biomedical innovation, manufacturing, supply chains, the efficiency of the fi…
(they are all wrong)
A fun property of S-curves is that they look exactly like exponential curves until the midpoint. Projecting exponentials is definitionally absurd because exponential growth is impossible in the long term. It is far more important to study the carrying capacity limits that curtail exponential growth.
Re: The Adolescence of Technology
#138It's interesting just how many opinions Amodei shares with AI 2027's authors despite coming from a pretty different context. - Prediction of exponential AI research feedback loops (AI coding speeding up AI R&D) which Amodei says is already starting today - AI being a race between democracies and autocracies with winner-takes-all dynamics, with compute being crucial in this race and global slowdown being infeasible -…
It's because few realize how downstream most of this AI industry is of Thiel, Eliezer Yudkowsky and LessWrong.com. Early "rationalist" community was concerned with AI in this way 20 years ago. Eliezer inspired and introduced the founders of Google DeepMind to Peter Thiel to get their funding. Altman acknowledged how influential Eliezer was by saying how he is most deserving of a Nobel Peace prize when AGI goes well (…
Yeah, it's a pretty blatant cult masquerading as a consensus - but they're all singing from the same hymn sheet in lieu of any actual evidence to support their claims. A lot of it is heavily quasi-religious and falls apart under examination from external perspectives.
We're gonna die, but it's not going to be AI that does it: it'll be the oceans boiling and C3 carbon fixation flatlining that does it.
Re: The Adolescence of Technology
#139Earlier quoted context omitted.
>So what can you and I do? Engage respectfully, Try and see other points of view, Try and express your point of view. I decided some time ago that I would attempt to continue conversations on here to try and at least get people to understand that other points of view could be held by rational people. It has certainly cost me Karma, but I hope there has been a small amount of influence. Quite often people do not chang…
Hypothetically, however, if your adversary is indeed without morality, then failing to acknowledge that means working with invalid assumptions. Laboring under a falsehood will not help you. Truth gives you clear eyed access to all of your options. You may prefer to assume that your opponent is fundamentally virtuous. It's valid to prefer failing under your own values than giving them up in the hopes of winning. Still…
This lacks any middle ground. I mean, is world divided between adversaries and allies? Are assumptions either valid or invalid? Are humans either "fundamentally virtuous" or "without morality"?
Such crude model doesn't help in navigating the reality at all.
Re: The Adolescence of Technology
#140I am continually surprised by the reference to "voluntary actions taken by companies" being brought up in discussion of the risks of AI, without some nuance given to why they would do that. The paragraph on surgical action goes in to about 5-10 times more detail on the potential issues with gov't regulation, implying to me that voluntary action is better. Even for someone at anthropic, i would hope that they would discuss it further.
I am genuinely curious to understand the incentives for companies who have the power to mitigate risk to actually do so. Are there good examples in the past of companies taking action that is harmful to their bottom line to mitigate societal risk of harm their products on society? My premise being that their primary motive is profit/growth, and that is revenue or investment dictated for mature and growth companies respectively (collectively "bottom line").
Im only in my mid 30s so dont have as much perspective on past examples of voluntary action of this sort with respect to tech or pre-tech corporates where there was concern of harm. Probably too late to this thread for replies, but ill think about it for the next time this comes up.