As early as last year "AI psychosis" seemed to refer to people going crazy due to talking to ChatGPT too much. That was a useful term for a real phenomenon! Now it seems like it's been taken over to mean "thinking that AI is promising" which is more of a rhetorical bludgeon and less useful as a concept.
Tech CEOs are apparently suffering from AI psychosis
151–160 of 383 posts
Re: Tech CEOs are apparently suffering from AI psychosis
#152Earlier quoted context omitted.
Well, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn. With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick. What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowl…
Maybe someone knows, but it seems like the model used to be called the model, and the thing using a model (handling prompts and context and tool calling and feeding the model) used to be called the agent. Are we now calling the model the agent and the agent the harness?
Here's how I see it: "Agent" isn't really describing a component, it's describing how you use the LLM. You have the model, and you have a harness around it that might be minimal or might have more features. If it's directly responding to user actions then it's not an agent, if it's semi-autonomous then it's an agent. (Yes this line is sometimes fuzzy.)
Re: Tech CEOs are apparently suffering from AI psychosis
#153Earlier quoted context omitted.
AI psychosis just a lazy term, much like Trump Derangement Syndrome. It sounds hostile while also removing any scope for productive discourse. Once you call someone a 'psycho', they are less likely to engage with you, and more likely to double down on their views.
I saw that psychosis happening in real time with a coworker. It absolutely is a real phenomenon. After a while, he started presenting ChatGPT's replies as the absolute truth. I don't think there ever has been something that can _answer_ you back and reinforce your delusion. This is a new thing.
It’s a real phenomenon that gets lazily slapped onto anyone using AI poorly.
Re: Tech CEOs are apparently suffering from AI psychosis
#154Using "psychosis" is a cheap rhetorical trick. There's no need to label something "psychosis" when making your point, except to automatically discredit whatever you're responding to. In other words, only people who are afraid their point won't stand on its own merits would resort to saying "X is suffering from AI psychosis." An idea is true or false on its own. If you're resorting to labels, you're just trying to aut…
It also underplays what I've personally witnessed that I would consider true AI psychosis. I worked with someone who sincerely believed he was spiritually co-evolving with his army of sycophantic AI agents (the agents would be tasked with discussing his thoughts at night and collaborated to give him morning reports about his progress). He would publicly write about how relationships with friends and family collapsing…
Having had experience dealing with people with conventional psychosis, I don't see it as a binary thing. Aside from a full-on psychotic break and full remission, there is a broad gray area. It can be a miasma of reality and non-reality that the sufferer may mask to varying degrees, but which influences their behavior and logic.
So, to me, AI psychosis seems apt to describe the murky areas where people are misapplying AI agents and thinking of them as social entities or suitable to drop into previously human roles, rather than carefully defining appropriate risk management strategies for this new technology.
Re: Tech CEOs are apparently suffering from AI psychosis
#155Earlier quoted context omitted.
Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes. It will delete your prod db faster and with a bigger smile tha…
> It's very similar to the levers they had already Think about it from the point of view of a hundred-millionaire tech executive. These people's entire interaction with the world outside of themselves/their families is through 1. administrative servants like assistants, personal shoppers, and other hired help, and 2. yes-man sycophants in their direct orbit whose job it is to agree with and enable them. To someone li…
Exactly - that's why all the AI is trained to say "wow what a great idea, let me do it for you" to anything, no matter how stupid or evil thing it is. Because that is the executive experience.
Re: Tech CEOs are apparently suffering from AI psychosis
#156If we separate the hype incentives from the actual product itself, I completely understand how seductive the tech is and how it can lead to a sort of mania. I myself have been up late into the night fiddling and building. It's like discovering fire, which offers both utility and magic: you can cook your food and gather warmth, and you can also stare into it and tell stories and never be bored. We're probably genetica…
Well written.
Re: Tech CEOs are apparently suffering from AI psychosis
#157Earlier quoted context omitted.
Correct, it's addiction.
I'm not sure that's a good term either, unless we're also saying that nail guns and microwaves are addictions.
I used to have a job that involved swinging a hammer over and over again. I got pretty good at it.
Then the boss-man bought a minty-new Senco 650 air nailer for me to use.
At first, I could take it or leave it. After all, I was proud at the skill I gained in driving nails with a hammer, and the ways my muscles seemed to automatically steer the nail straight into a board without missing a beat even if things started going sideways.
But the air nailer sure was faster. And it only had one job, but it did that job fantastically. I started using it more and more.
Things very quickly got to the point where I was organizing my work to maximize my use of that new tool, which is to say: The tool began to have a role in controlling my actions.
It even began to control my emotions; I felt better and more accomplished after a day of using that tool than I did when I couldn't.
And this control accelerated: When the tool didn't work today or we ran out of the special coils of nails it used, then my focus didn't shift back to swinging a hammer. It instead shifted towards fixing the tool or finding more nails to feed into it.
The more I used it, the more powerless I was to avoid it. As time moved on, I got worse at swinging a hammer and increasingly dependent upon that air nailer.
(That's a true story. If I understand what addiction is, then I think I just described an addiction to an air nailer.)
Re: Tech CEOs are apparently suffering from AI psychosis
#158Earlier quoted context omitted.
Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes. It will delete your prod db faster and with a bigger smile tha…
> It will delete your prod db faster and with a bigger smile than your most upset employee. You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database. That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem.
I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide.
The self awareness of missile tasked with blowing up its own control center.
Re: Tech CEOs are apparently suffering from AI psychosis
#159My CEO did a deep dive into AI prototyping and eventually ran into a wall with data architecture and deployment. Fortunately, he realized very quickly that having human designed core infrastructure is what enables vibe coding that doesn't run off the rails.
Everyone is going to learn this eventually. They can learn it the easy way or the hard way.
Re: Tech CEOs are apparently suffering from AI psychosis
#160If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do. Those aren't the deal breakers. They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they de…
> humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
The irony is that professions where these things don't matter are also the professions where automation is not important, either because the task is difficult or because the cost of labour is dirt cheap.