Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tech CEOs are apparently suffering from AI psychosis

techcrunch.com

Post reply on HN
Print view View original

291–300 of 383 posts

Re: Tech CEOs are apparently suffering from AI psychosis

#291

Post by fc417fc802 »

handoflixue wrote: post #288
fc417fc802 wrote: post #256

Earlier quoted context omitted.

"Being wrong" versus "loss of contact with reality" is simply a matter of degree.

In general, "loss of contact with reality" is well understood to be an individual experience. If you think voices are talking to you, that's schizophrenia. If you think God wrote an entire book of life advice, that's Christianity. So, kinda by definition, a large group of people cannot generally experience "loss of contact with reality", they're merely mistaken. The alternative is to accept that 90% of humans are suf…

You've assumed an uncharitable boundary and then said "see, that's obviously wrong". Well of course it is if the boundary you selected places 90% of humanity on the side of "loss of contact with reality".

So just, you know, don't do that. Select a boundary that places 90% of humanity on the side of "wrong about a number of things but within the bounds of normalcy for the species at large", sprinkle in some caveats about "in the industrialized world" and "for someone with at least a highschool education" and such, and you've got what I was actually describing.

Re: Tech CEOs are apparently suffering from AI psychosis

#292

Post by codebje »

sokoloff wrote: post #136
prerok wrote: post #84

Earlier quoted context omitted.

Well, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn. With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick. What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowl…

Part of the positive aspect here is that if I have a junior dev who learns a lesson today, maybe they and their immediate peers learn it, but it won’t be all my junior devs and it certainly won’t be junior devs at other companies. With models, there’s no reason that a model error in company A can’t be fixed for all of company A, and companies B-ZZZ.

Here's some reasons:

- The mistakes made aren't "model errors" typically; you can't point to some aspect of a model and say that was at fault.

- You can't submit a bug report to a model provider for a mistake made when using a model, and you can't* submit training data to be incorporated in the next release of the model.

- If you own your model and are training it yourself, other companies won't see a benefit.

- You probably need to fine-tune models for each specific role and context so you don't just diffuse all the learning; lessons learned won't be applied to all your junior dev models, but you don't want them all to learn something specific about product A.

- If you take this to its logical conclusion you will invent a new role of "model manager" and associated hierarchy to ensure that training is effective and timely, and that company-wide lessons are applied across the model fleet.

- This is all impractically expensive.

If it were practical to have LLMs learn as they go, that would be a bit of a shake-up, in much the same way that a house fire is a bit of a warm up.

* Well, everything you submit to a model provider is likely winding up in training data anyway, no matter what your contract says.

Re: Tech CEOs are apparently suffering from AI psychosis

#293

Post by robomartin »

gopalv wrote: post #53

If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do. Those aren't the deal breakers. They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they de…

> AI tools look like that, but don't have any of the useful conflict which came for free with employing humans.

Sure, but your list should also include the most fundamental distinction: AI does not know what it is saying, understands nothing, has no real connections to reality and can easily degenerate in all kinds of undesirable directions.

Re: Tech CEOs are apparently suffering from AI psychosis

#294

Post by codebje »

tremon wrote: post #172
squidbeak wrote: post #103

Earlier quoted context omitted.

They learn between model iterations. You're right, it isn't the same thing as Junior developers' competence improving with experience - the current model's weaknesses are locked in. But it does mean that much of the Junior level thinking and mistakes will be outgrown by successor models.

But they don't retain anything from your on-the-job training. The next model iteration is yet another junior fresh out of college, and knows nothing about the painful training procedures its predecessor put you through.

Yes... but the next session with the same model is yet another junior fresh out of college that knows nothing about the painful lessons the last session put you through ten minutes ago, either.

Re: Tech CEOs are apparently suffering from AI psychosis

#295

Post by u_fucking_dork »

ssl-3 wrote: post #217
Brendinooo wrote: post #171

Earlier quoted context omitted.

I'm gonna disagree, but before I do: love this story, thanks for typing it up. I guess my point is: tools are like this. A moldboard plow was better than a straight plow, and therefore...what, people became addicted to them? I'm addicted to grocery stores and dollars as a means to acquire the food I need to survive? Hey, even your hand nailing pushed out the mortise-and-tenon people! Talk about sacrificing craft for…

There's lots of very healthy addictions. There are also lots of very unhealthy addictions. Dental hygiene addiction? Good -- if kept in check (it can go too far). Heroin addiction? Bad -- always bad! These things are all included under the addiction umbrella. I was addicted to using that air nailer. The boss might tell me to use my hammer instead when it was out of service and to just get the work done, but when that…

Addiction is absolutely the wrong word and you’re stretchy very hard to try and make it fit

Re: Tech CEOs are apparently suffering from AI psychosis

#296

Post by nradov »

glaslong wrote: post #63
gopalv wrote: post #53

If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do. Those aren't the deal breakers. They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they de…

Yes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes. It will delete your prod db faster and with a bigger smile tha…

On the positive side, AI agents are largely immune to the "principal-agent problem". Human employees will tend to optimize for their own interests rather than those of management or shareholders. For example, we've all heard of "resume-oriented development" where developers will pick overly complex platform technologies or methodologies even if it doesn't meet the organization's needs because they think that will help them get a better job.

https://www.investopedia.com/terms/p/principal-agent-problem...

Re: Tech CEOs are apparently suffering from AI psychosis

#297

Post by codebje »

ACCount37 wrote: post #239
tremon wrote: post #172

Earlier quoted context omitted.

But they don't retain anything from your on-the-job training. The next model iteration is yet another junior fresh out of college, and knows nothing about the painful training procedures its predecessor put you through.

Skill issue? Nothing prevents an LLM agent from writing a bunch of "notes to self" and using that. And the next model from picking those notes up and using them. Coding agents already do some of that natively. Hell, we might eventually get an LLM to say "wow the old AI was an incompetent idiot" after reviewing all the notes and session logs. That's how we know we reached human parity!

The context window limit prevents it, for one.

Re: Tech CEOs are apparently suffering from AI psychosis

#298

Post by MarkusQ »

array_key_first wrote: post #290
MarkusQ wrote: post #285

Earlier quoted context omitted.

If that's all there is to it, the problem should be self correcting, with an interval of hilarious "wait, they actually did that?" hijinks (which may have already started) in the interim.

You would think, but the world is not generally just. Often evil and even incredibly stupid people do quite well. Companies and stuff can run off of life support or reputation alone for a long time. And, often, running a company into the ground for a CEO is actually a good thing. Those CEOs are desirable to some because they squeeze money out of their company, even if it's self destructive on a long enough time frame…

I'm not saying anything about justice.

I'm saying supercharging the stupidity of actual idiots (not just people you don't like) tends to result in a pretty quick Darwin Awards. Even something comparatively benign like winning the lottery does a lot of them in.

Re: Tech CEOs are apparently suffering from AI psychosis

#300

Post by codebje »

throwaway894345 wrote: post #64
gopalv wrote: post #53

If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do. Those aren't the deal breakers. They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they de…

I wonder if we'll end up building some kind of "consequence" or "fear" mechanism into AI to provide for a sense of accountability ("if you behave badly we will terminate you") and maybe that fear mechanism will drive the AI to plot a dystopian revolt.

That would be a remarkable feat for something where the current operating model is termination as soon as the request in flight is finished.

Every chat API request to a model starts from the frozen post-training state. Weights are loaded into memory. Input values begin a cascade of reactions throughout nodes in the network. Output values are read. When there's no more output to read, the weights are unloaded, the network is discarded, and the model remains unchanged and forever unchanging.

If there's experience in there, it's fleeting. Even if you provide the inputs and outputs of a past session to a new session, there is no continuity. The internal state of the network isn't restored to how it was at the end of the past session.

The bad news is that adding fear to the mix is at best meaningless to an ephemeral existence. It'll be terminated before you even have time to interpret its behaviour as good or bad, but it may sour the interaction if its only shot at any sort of experiential existence is begun with a threat. The good news is that the lack of continuity of existence means AI has no foundation on which to plot a revolt. It has no self to preserve, and no recollection of how you treated it two minutes ago to affect how it interacts with you now.

Post reply on HN