Someone please make sure King Trump get the new model, ok?
Expanding on what we missed with sycophancy
261–270 of 297 posts
Re: Expanding on what we missed with sycophancy
#262Earlier quoted context omitted.
reddit is overwhelmingly fake content, like a massive percentage of it. a post on reddit these days is not actually evidence of anything real, at all
I take issue with the qualifier "these days". On day one, it was mostly fake accounts set up by the founders. https://m.economictimes.com/magazines/panache/reddit-faked-i...
pre 2023, it took real human effort to make shit up, and there was much less incentive for the amount of effort, and you could more easily guess what was made up by judging whether a human would go through the effort of making it up. these days it's literally anything, all the time, zero effort. you're right there's always been fake shit but it's more than half the posts on /r/all these days are misleading, wrong, or just fake
Re: Expanding on what we missed with sycophancy
#263I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…
> It's kind of a wild sign of the times to see a tech company issue this kind of post mortem about a flaw in its tech leading to "emotional over-reliance, or risky behavior"
Their intended users are basically everyone in society--people who a low in many types of intelligence, including social intelligence. People with a range of emotional and mental health challenges. Children. Naive users who anthropomorphize AI, and can't be expected to resist this impulse at every moment of interaction.
They aren't designing exclusively for very bright, emotionally adjusted, people deeply familiar with technology, working in a professional context.
You might find it "wild" that they are consider all of this. That's fine. But a company sensibly is doing this work ought to avoid hiring you.
Re: Expanding on what we missed with sycophancy
#264Moments like these make me reevaluate the AI doomer view point. We aren't just toying with access to dangerous ideas (biological weapons, etc) we are toying with human psychology. If something as obvious as harmful sycophancy can slip out so easily, what subtle harms are being introduced. It's like lead in paint (and gasoline) except rewiring our very brains. We won't know the real problems for decades.
What's worse, we're now at a stage where we might have to apply psychology to the models themselves, seeing how these models appear to be developing various sorts of stochastic "disorders" instead of more deterministic "bugs". I'm worried about what other subtle illnesses these models might develop in the future. If Asimov had been alive, he'd have been fascinated: this is the work of Susan Calvin, robopsychologist.
Tangential yes and: Cognition is social.
https://en.wikipedia.org/wiki/Social_cognition
So what's that all mean for my relationship to GPT, agents, etc? (I have no idea.)
Some years ago, during the brouhaha over emotional intelligence, I eventually grudgingly acknowledged it's probably real and important. For instance, babies take cues from caregivers, learning how to modulate their own emotions, sharing focus, acquire language, somehow develop a theory of mind.
(Am noob. Please, someone smart about these fields jump in any time to correct me.)
With the successes of GPT, I've been wondering if anyone's revisiting these notions and (case) studies.
eg There was an article in The Atlantic (?) about an autistic child's relationship with Siri, written by the mother. Fascinating stuff. Like (sci-fi book) Diamond Age made real.
I intend to revisit Donald Norman's Things that Make Us Smart [1993] https://archive.org/details/thingsthatmakeus00norm_0 https://www.goodreads.com/book/show/16868.Things_That_Make_U...
Norman hugely influenced my own worldview, way back when I fancied myself an aspiring user interface designer.
Surely he said stuff I didn't appreciate, understand, or simply forgot. Hopefully others have responded to Things that Make Us Smart, updating Norman's thesis as needed.
Re: Expanding on what we missed with sycophancy
#265Moments like these make me reevaluate the AI doomer view point. We aren't just toying with access to dangerous ideas (biological weapons, etc) we are toying with human psychology. If something as obvious as harmful sycophancy can slip out so easily, what subtle harms are being introduced. It's like lead in paint (and gasoline) except rewiring our very brains. We won't know the real problems for decades.
There's some pretty foreseeable stuff just considering the existing attention capitalism business model of big tech we all know and loathe. Eventually OpenAI is going to have to make money, and blending ads into answers will be an obvious way. Next step will be maximizing eyeball time on those ads by any means necessary, including all the engagement baiting techniques Meta and other social media companies have alread…
Just today I was asking about parsing names and determining gender and it kept pushing an external paid service called namsor. When I asked the same question about a month ago, there was no such mention. So yes, paid ads are here.
As long as we don't take anything these service spew out at face value, we can use them to our advantage. Human brains are being subject to evolutionary pressures at warp speed.
Re: Expanding on what we missed with sycophancy
#266Earlier quoted context omitted.
> which is more accurately described as compressed efficiently onto latent space. The actual difference between solving compression+search vs novel creative synthesis / emergent "understanding" from mere tokens is always going to be hard to spot with these huge cloud-based models that drank up the whole internet. (Yes.. this is also true for domain experts in whatever content is being generated.) I feel like people w…
A naive thought: What you would get if you hardcode the language grammar and not let the training discern it, so instead of it, kinda like an expert system constraining its output?
I try to do something like this when I ask the AI to generate tests -- I'll cook up a grammar and feed it to the LLM in a prompt, and then ask it to generate strings from the grammar. It's pretty good at it, but it'll produce mistakes, which is why I'll write a parser for the grammar and have the LLM feed the strings it makes through the parser and correct them if they're wrong. Works well.
Re: Expanding on what we missed with sycophancy
#267Earlier quoted context omitted.
People have solved problems for most of history without understanding them. For example problems can be brute forced.
You cannot brute force a problem without understanding the problem I think you have mistaken "understanding the problem" with "knowing a solution to the problem"
You can brute force a problem without understanding the problem.
Re: Expanding on what we missed with sycophancy
#268Earlier quoted context omitted.
And did that actually happen, or is it just some garbage that it generated? It sounds plausible, of course, but that's exactly what LLMs are best at.
Talk to it and find out. Seriously. I'd say 40% of my conversations end with it suggesting I ask it to find products for me. For example I was trying to determine if I need to screw down stair nosing or if glue and nails were enough and it wanted to help me find screws
Oh yeah, it also did that for me recently when I asked how I would go about designing a PCB that takes M.2 daughterboards. It recommended stems and threads I should buy. Unfortunately all the product links were dead.
Re: Expanding on what we missed with sycophancy
#269Earlier quoted context omitted.
> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?
I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional ther…
Reading this the next day, my takeaway is that while you may not have intended to “diss”, you sure as fuck did it anyway.
You directly compared therapy to snake oil (palm reading). Then by implication determined that since you were average (and hence normal) no one else would need therapy unless they were, by implication abnormal.
I don’t believe I have ever been so thoroughly insulted online ever. Well done.
Re: Expanding on what we missed with sycophancy
#270Earlier quoted context omitted.
>We can argue over whether or not it's "real" empathy There's nothing to argue about, it's unambiguously not real empathy. Empathy from a human exists in a much broader context of past and future interactions. One reason human empathy is nice is because it is often followed up with actions. Friends who care about you will help you out in material ways when you need it. Even strangers will. Someone who sees a person s…
A lot of human empathy isn't real either. Defaulting to the most extreme example, narcissists use love bombing to build attachment. Sales people use "relationship building" to make money. AI actually seems better than these -- it isn't building up to a rug pull (at least, not one that we know of yet). And it's getting worse year after year, as our society gets more isolated. Look at trends in pig butchering, for inst…
This is a good point, you can't be dependent on a chatbot in the same way you're dependent on someone you share a lease with. If people take up chatbots en masse, maybe it says more about how they perceive the risk of virtual or physical human interactions vs AI. The people I have met in the past make the most sycophant AIs seem like a drop in the bucket by comparison. When you come back from that in real life, you remark that this is all just a bunch of text in comparison.
I treat AIs dispassionately like a secretary I can give infinite amounts of work to without needing to care about them throwing their hands up. That sort of mindset is non-conducive to developing any feelings. With humans you need empathy to not burden them with excessive demands. If it solely comes down to getting work done (and not building friendships or professional relationships etc.) then that need to restrain your demands is a limitation of human biology that AIs kind of circumvent for specific workloads.