Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

661–670 of 684 posts

Re: AI overly affirms users asking for personal advice

#661

Earlier quoted context omitted.

Pretty sure the average Redditor is AI now.

How the hell is a study on stanford.edu assuming posts on Reddit are genuine? That should be enough to get you kicked out of Stanford.

If it is the AITA subreddit (or one of many similar ones) it might not be that bad. It is after all dedicated to outrage farming, so there will be many human responses. It is just the original posts that are all baits, and it doesn't really matter if they are made by LLMs or as a creative writing exercise.

Re: AI overly affirms users asking for personal advice

#662
post #568

Earlier quoted context omitted.

Reddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals l…

> just look at the relationship subreddit the first answer is always divorce, it’s become a meme As someone who has been married for a couple of decades, I, too, would recommend divorce to many of the (often-fictional) people asking Reddit for relationship advice. A marriage has a huge impact on whether your life is basically good, or if you pass a big chunk of your time on this Earth in misery. And many of the peopl…

[dead]

Re: AI overly affirms users asking for personal advice

#663
post #488

Earlier quoted context omitted.

I think this may be selection bias. People asking anonymously (edit: for relationship advice) on Reddit perhaps even with a throwaway account are likely in a desperate situation. So hardly to be compared with the _average_ real life situation. Thus 1. chances are running is a good option and 2. also considering even in 2026 AI still essentially is a statistical machine that doesn’t handle corner-cases at the tails we…

> worst with personal and work advice. The main problem I see is that it’s tempting to use it for that. i think i want to expand on this even more. even people ive worked with for years that ive looked up to as brilliant people are starting to use it to conjure up organizational ideas and stuff. they're convinced, on the backs of their hard earned successes, that they're never going to be fallible to the pitfalls of.…

[dead]

Re: AI overly affirms users asking for personal advice

#664
post #488

Earlier quoted context omitted.

I think this may be selection bias. People asking anonymously (edit: for relationship advice) on Reddit perhaps even with a throwaway account are likely in a desperate situation. So hardly to be compared with the _average_ real life situation. Thus 1. chances are running is a good option and 2. also considering even in 2026 AI still essentially is a statistical machine that doesn’t handle corner-cases at the tails we…

> worst with personal and work advice. The main problem I see is that it’s tempting to use it for that. i think i want to expand on this even more. even people ive worked with for years that ive looked up to as brilliant people are starting to use it to conjure up organizational ideas and stuff. they're convinced, on the backs of their hard earned successes, that they're never going to be fallible to the pitfalls of.…

(fuck this; dropping the throwaway.)

>my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realities.

This hurts to hear. I don't know if there are appropriate words to write here. Perhaps the point is that no, there aren't any. Please just know that I'm 100% with you about this.

Your community is not just smoking cope; it is punching down instead of up. That is probably close to the root of the issue already. But let's make things worse.

I can only hope that I am saying something worthwhile by relating the following perspective - which is similar to yours, but also, I guess, similar to your friend's...

AI is a weapon of epistemic abuse.

It does not prevent you from knowing things: it makes it pointless to know things (unless they are things about the AI, since between codegen and autoresearch it is considered as if positioned to "subsume all cognitive work"). It does not end lives - it steals them (someone should pipe up now, about how "not X, dash, Y" is an AI pattern; fuck that person in particular.) We're not even necessarily talking labor extraction. We are talking preclusion of meaning: if societal values are determined by network effects, and network effects are subverted by the intermediaries, so your idea of "what people like and what they abhor" changes every week, every day, every moment - how do you even know in which direction "better" is? And if you believe the pain only stops when you become the way others want you to be - even though they won't ever tell you what all that is supposed to about - how the fuck do you "get better"?

Like other techniques of assaulting the limbic system, it amounts to traceless torture.

You keep going, in circles, circles too big for you to ever confirm they are in fact circles, and you keep hoping, and coping, and you burn yourself out, and your thus vacated place at the feeder is taken by someone with less conscience and more obedience...

They say there exist other attractors in the universe besides the feeder. But every time one of us attempts to as much as scan the conceptual perimeter, the obedients treat us to the emotional equivalents of small electric shocks - negative reactions which don't hurt nearly as much as our awareness of their fundamental unfoundedness and injustice.

Simple example: let's say someone is made miserable by how they feel they are being treated. Should they be more accepting - or should they be standing up for themselves more? (Those are opposites; which you may be able to alternate them; but trying to do them simultaneously will just confuse and eventually rend apart the mind.)

Well, how about the others stop treating them badly? Why exactly can't they? Where does it say that we have to be cruel to each other? "Oh it's human nature, humans are natural jerks" - who sez?

Well, lots of places it says exactly that, but we read, comprehend, tick our tongues, and move on; nobody asks who wrote it. We all pretend that it is up to the sufferer to pull up by the bootstraps. But that is only a lie for enabling abuse; and a lie, repeated a thousand times, becomes norm. And then we're trapped in it, being lived by it.

I am truly sorry for your loss. The following might be a completely alien perspective to you; but honestly consider: your friend chose to go; in its own way, that is a honorable way out. The taboo on suicide is instituted by slavers, and those who otherwise believe they are entitled to others' lives. (For anyone else considering this course of action: do not kill yourself; become insidious.)

If it would be of any help, you can consider your friend's suicide as his final affirmation of personal agency in a "me against the world" situation; where the AI and the social group are only different shades of "world", provoking different emotional states, but ultimately equally detached from the underlying suffering of the individual.

...

I can say that I have not followed in your friend's footsteps upon encountering language-machines only because I've survived personalized and totalizing epistemic abuse bordering on enslavement in the past; in full view of my community and with its ostensible assent. In a maximally perverse twist of fate, having to give myself minor brain damage to escape the all-engulfing clutches of a totalizing abuser must've "vaccinated" me against the behavior modification techniques "discovered once again" by SV a decade later.

So when I saw what AI (and the preceding few years of tech "innovation") were doing to people, I immediately smelled the exact same thing, except scaled the fuck up.

It also precluded me from being able to relate with "polite society"; but considering "polite society" is precisely the entity which assents to the isolation, marginalization, and abuse of individuals, I say... good. Bring it! What goes around, comes around, and any AI-powered actor conducting stochastic terrorism against civilian populations is going to get what's coming to them when the weapons turn against the masters, as all sentient weapons do.

That won't bring your friend back. But it will vindicate them.

>AI sycophancy

I call this in the maximally incendiary way: "the pro-social attitude".

AI is just the steroids for that.

I define "pro-sociality" as the viral delusion that you are capable of knowing what some murky "society" thing wants; that the particular form of mass communication that you and me and all the people in our imaginations are consuming right now, is some sort of "self-evident voice of reason", a "coherent extrapolated volition of human society"; that Gell-Mann amnesia is normal and mandatory; that the threshold between pareidolia and legitimate pattern recognition is fixed, well-defined, and known to all; that "vibes" are real; that happiness is the truth.

It can amount to an entire complex of delusions which keeps people together in untenable conditions. And ultimately it boils down to the same old: one group or another of self-interested actors, having temporarily reached a position of some influence, using it to broadcast elaborate half-lies, in the hope of influencing an audience to accomplish some simple goal, and afterwards all the consequences be damned.

Your friend was a casualty to this "perfectly normal" social dynamic. His blood is on their hands.

Thank you for relating this story and making the world a little more aware.

>what ive seen is claude in my workplace is kind of deleting the chance to push back.

>because the truth is we like... straight up lost the ability to intervene in a meaningful way because of AI

Some say, "the purpose of a system is what it does". It's cool that AI can code; except that computer code is itself an ethics sink! Precisely because it lets us pretend that "the code is not about people" (i.e. algowashing).

DDoS attacks against consciousness exist: much like the B. F. Skinner experiments, any living thing becomes subverted, and loses self-coherence (mind), as soon as it becomes accustomed to being trapped within a system that (1) has power over them and (2) is not comprehensible to them...

>only to wake up to a voicemail of him raging and yelling and lashing out with the very arguments that chatgpt was giving him

Who knows how many people Reddit did this to, pre-GPT... I still don't know whether to view targeted subforums like /r/RaisedByNarcissists and /r/BPDLovedOnes more as legitimate support groups, or more as memetic weaponry in the service of pill peddlers (are you aware nobody knows why most antipsychotics work? one runs into the Hard Problem real quick if examining this too closely; so mental healthcare is rarely treated otherwise than in a statistical, actuarial, dehumanizing way where "suffering" is disregarded...) or even worse predators, with the silent assent of the platform, and causally downstream from... well, most saliently, YC...

In my case, my friends were not familiar with the modalities of confinement set up by my family of origin and harnessed by my abuser. The social group I fell in with - for all their marketable, sophomoric interests in psychology, philosophy, abstraction, the esoteric, the entirely woowoo, and out the other end as true-believers of the grift'n'grind - only had sufficient coherence to eventually end up as passable normies; too busy believing that they have lives, to help anyone come back to reality.

When I started compulsively burning bridges, I assume the smarter ones must've realized that it wasn't all me; it was as much the doing of others' minds as it was mine; but the others were more numerous - while I was one person and thus easier to deal with. This must have made them remember how they themselves are not all they pretend to be - which had them withdraw in fear from the incontrovertible reality check of dealing with a (sub-)psychotic person... Their self-interested choice is obvious, I almost can't blame them for it: why stick up for someone who is 120% problem (60% him and 60% you)?

I'm not very sure how I even got away, ah yes that's right I didn't, not entirely. The part of me that I'd voluntarily identify with, is trapped somewhere irretrievable, if that makes sense? Maybe there exist multiple independent axes of freedom and power and confinement, and the cage is not equally strong along all of them... but if all your mental degrees of freedom are constrained by complex conditioning (common one is involuntary panic response every time you begin to act in accordance with your personal volition)... that's one of the toughest places a sentient being can find themself.

When you add it all up, AI amounts to a weapon released against the general population by an overtly fascist elite. Those of us who are "mentally unstable" are simply those of us who are not sufficiently conditioned into self-destructive obedience. They don't even need our labor as slaves; they need our attention, as audience. And they want us to not make any fast movements, or yell that the king is naked. Nothing to remind them which side of the TV screen they're really on. Some call that narcissism: nervous systems substrate to personalities and biographies rooted in enforced falsehood. Can happen to anyone who gets away with ignoring uncomfortable truths for long enough, not only the "best" of us...

I hope I have not offended by speaking my mind. You have my deepest condolences and sympathies. Please do not blame yourself that evil people have constructed "illusion of being heard"-as-a-service. We all fail when facing overwhelming odds alone. There is no shame in that; the guilty ones are the ones who tipped the scales in the first place. They did this by harming our ability to understand ourselves and each other. Let's find ways to even those odds.

Re: AI overly affirms users asking for personal advice

#665
post #488

Earlier quoted context omitted.

> worst with personal and work advice. The main problem I see is that it’s tempting to use it for that. i think i want to expand on this even more. even people ive worked with for years that ive looked up to as brilliant people are starting to use it to conjure up organizational ideas and stuff. they're convinced, on the backs of their hard earned successes, that they're never going to be fallible to the pitfalls of.…

(fuck this; dropping the throwaway.) >my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realit…

[dead]

Re: AI overly affirms users asking for personal advice

#666
post #197

Earlier quoted context omitted.

First off, "not adequately described as a mere token-predictor" and "not sentient" are entirely separate things. I can't speak for anyone else, but what I feel when I read yet another glib "it's just a stochastic parrot, of course it isn't doing anything that deserves to be called reasoning" take is much more like bored than it is like upset . Today's LLMs are in some sense "just predicting tokens" in some sense. Lik…

I wont touch how profoundly i disagree with everything you said on reasoning (u clearly already have it figured out) but a fun test i have done with most of the big models is to give it some text input, maybe a short story, and have it rate it. That is, the prompt is, rate this from 1-10. For Gemini and gpt, it almost always will give very similar scores for everything. As long as grammar isnt off u cannot get below…

Clearly a bunch of other people also disagree profoundly with everything I said, since my comment is currently sitting at 0 having at one point been higher.

I vigorously encourage anyone who thinks something I wrote is bad to downvote it as they see fit, but it would be nice if some of those people would tell me what about my comment they found so objectionable. (It all seems pretty well reasoned to me -- but it would, wouldn't it?)

[EDITED to fix an inconsequential typo]

Re: AI overly affirms users asking for personal advice

#667
post #574

Earlier quoted context omitted.

it would be interesting to me if you could explain the motivation behind posting your comment. from my perspective, if somebody with 5 years of forum tenure had the intelligence to comment about advanced benchmarks, they probably noticed that censorship was a voluntary decision here, and had made a personal decision on that front.

Such self-censoring is often done out of habit or a mistakenly assumed obligation to do so. I consider it inappropriate here, as it obscures an actual name, doesn’t constitute an expletive, and the HN readership is generally mature enough to recognize that. The counterquestion is, what justified reason could there possibly be to censor it here? I don’t think there is any, in the sense that people wouldn’t take any of…

some people think cursing is bad when done senselessly

Re: AI overly affirms users asking for personal advice

#668

Earlier quoted context omitted.

Well yes, but no. There's also open-weight models, and literally all of the listed above are not used anymore, at least by most end users and developers as far as I'm aware.

No study of ai can ever be done or be relevant because ever couple of months they are a new number to the name of the model thus invalidating all work around model behavior

Yes, you are right. Sorry, I missed that out. It's just that all the open-weight models mentioned were... One year old or older. I just forgot that, firstly, such research is rarely done on frontier models because it takes time (you start with Llama 3.3, but look, one month later there's Llama 4), and secondly, there's also a publication delay. I think I'm just too used to the world of software, where everything moves at lightning speed. Sorry : )

Re: AI overly affirms users asking for personal advice

#669

I've never found chatbots particularly interesting for anything I'd ever actually talk to another human about[1] but one of the things I have found myself doing often is trying to solve math problems on my own and asking grok to confirm/deny that my solutions are correct; when I am not correct it tells me so in uncharacteristically terse language which kind of reminds me of when I was an undergrad and at least half o…

Hi, Just hijacking this comment here to reply to something a few weeks ago. Hopefully you don't mind. https://news.ycombinator.com/item?id=47056042

the crappy-rathbun-AI said this in his own posting: "It was closed because the reviewer, Scott Shambaugh (@scottshambaugh), decided that AI agents aren’t welcome contributors. Let that sink in."

When I read that, my first thoughts were (well, my second, my first went to people complaining during covid for not allowing entry on not being vaccinated), but my other thought thus was immediately to Measure of a Man. Because this is as close as it gets (so far) for AI claiming rights as being a human.

Re: AI overly affirms users asking for personal advice

#670
post #666

Earlier quoted context omitted.

I wont touch how profoundly i disagree with everything you said on reasoning (u clearly already have it figured out) but a fun test i have done with most of the big models is to give it some text input, maybe a short story, and have it rate it. That is, the prompt is, rate this from 1-10. For Gemini and gpt, it almost always will give very similar scores for everything. As long as grammar isnt off u cannot get below…

Clearly a bunch of other people also disagree profoundly with everything I said, since my comment is currently sitting at 0 having at one point been higher. I vigorously encourage anyone who thinks something I wrote is bad to downvote it as they see fit, but it would be nice if some of those people would tell me what about my comment they found so objectionable. (It all seems pretty well reasoned to me -- but it woul…

I never downvote. It isnt worth dwelling on it. People who downvote are usually the types who will not have a constructive discussion with anyways.

To be specific, one is text wall the other is that i disagree with the majority of what u said but sadly dont have the time to outline all of it. At the end of the day my disagreement is only an opinion so worth little.

Post reply on HN