Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Expanding on what we missed with sycophancy

openai.com

Post reply on HN
Print view View original

171–180 of 297 posts

Re: Expanding on what we missed with sycophancy

#171

Post by kornork »

breakingcups wrote: post #153
kornork wrote: post #85

[flagged]

Stop it, please. Em-dashes are perfectly fine. On a throwaway Reddit-post, no. I understand the signal. But on a corporate publication or some other piece of professional writing, absolutely. Humans do use em-dashes on those.

I love the em dash; I use 'em all the time. But finding them all over this post indicates, maybe, possible, (probably?), that the author at least used AI generated text as a first draft.

In the worst case, this is like "We released this sycophantic model because we're brain dead. To drive home the point, we had ChatGPT write this article too (because we're brain dead)."

I tend to rely on AI to write stuff for me that I don't care too much about. Writing something important requires me to struggle with the words to make sure I'm really saying what I want to say. So in the best case, if they relied on ChatGPT so much that it left a fingerprint, to me they're saying this incident really wasn't important.

Re: Expanding on what we missed with sycophancy

#173

Post by labrador »

grey-area wrote: post #146
labrador wrote: post #68

Earlier quoted context omitted.

I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it

You are, and you should stop doing that.

Point taken. I admit my comment was silly the way I worded it.

Here's the line I’m trying to walk:

When I ask ChatGPT about its own internal operations, is it giving me the public info about it's operation, and also possibly revealing propreitary info, or making things up obfuscate and preserve the illusion of authority? Or all three?

Re: Expanding on what we missed with sycophancy

#174

Post by labrador »

throwaway314155 wrote: post #164
labrador wrote: post #68

Earlier quoted context omitted.

I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it

Put simply, GPT has no information about its internals. There is no method for introspection like you might infer from human reasoning abilities. Expecting anything but an hallucination in this instance is wishful thinking. And in any case, the risk of hallucination more generally means you should really vet information further than an LLM before spreading that information about.

True, the LLM has no information but OpenAI has provided it with enough information to explain it's memory system in regards to Project folders. I tested this out. If you want a chat without chat memory start a blank project and chat in there. I also discovered experientially that chat history memory is not editable. These aren't hallucinations.

Re: Expanding on what we missed with sycophancy

#175

Post by labrador »

svat wrote: post #141
labrador wrote: post #68

Earlier quoted context omitted.

I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it

I didn't downvote but it would be because of the "I'd don't know if any of this is made up" — if you said "GPT said this, and I've verified it to be correct", that's valuable information, even it came from a language model. But otherwise (if you didn't verify), there's not much value in the post, it's basically "here is some random plausible text" and plausibly incorrect is worse than nothing.

see my other comments about the trustworthiness about asking a chat system how it's internals work. They have reason to be cagey.

Re: Expanding on what we missed with sycophancy

#177

Post by brookst »

Tostino wrote: post #113
labrador wrote: post #10

Earlier quoted context omitted.

A sense that I was talking to a sentient being. That doesn’t matter much for programming task, but if you’re trying to create a companion, presence is the holy grail. With the sycophantic version, the illusion was so strong I’d forget I was talking to a machine. My ideas flowed more freely. While brainstorming, it offered encouragement and tips that felt like real collaboration. I knew it was an illusion—but it was a…

I need pushback, especially when I ask for it. E.g. if I say "I have X problem, could it be Y that's causing it, or is it something else?" I don't want it to instantly tell me how smart I am and that it's obviously Y...when the problem is actually Z and it is reasonably obvious that it's Z if you looked at the context provided.

Exactly. ChatGPT is actually pretty good at this. I recently asked a tech question about a fairly niche software product; ChatGPT told me my approach would not work because the API did not work the way I thought.

I thought it was wrong and asked “are you sure I can’t send a float value”, and it did web searches and came back with “yes, I am absolutely sure, and here are the docs that prove it”. Super helpful, where sycophancy would have been really bad.

Re: Expanding on what we missed with sycophancy

#178

Post by slashtmpslashme »

Henchman21 wrote: post #67
NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

It's the easiest way to cope with not having a purpose in life and depending on external validation / temporary pleasures.

Like jordan peterson (though I don't like the guy) has said - happyness is fleeting, you need a purpose in life.

Most of current gen has no purpose and grown up on media which glorify aesthetics and pleasure and to think that's what the whole life is about. When they don't get that level of pleasure in life, they become depressed and may turn to therapy. This is very harmful to the society. But people are apparently more triggered by slang words than constant soft porn being pushed through Instagram and the likes.

Re: Expanding on what we missed with sycophancy

#179

Post by Sharlin »

kadushka wrote: post #73
Henchman21 wrote: post #67

Earlier quoted context omitted.

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional ther…

I'm pretty sure that just about every single person could use a therapist. That is, an empathetic, non-judgemental Reasonable Authority Figure who you can talk to about anything without worrying about inconveniencing or overloading them, and who knows how to gently guide you towards healthy, productive thought patterns and away from unhealthy ones. People who truly don't need someone like that in their life are likely a small minority; much more common is, probably, to simply think that you don't.

Re: Expanding on what we missed with sycophancy

#180

Post by photonthug »

neuroelectron wrote: post #168
photonthug wrote: post #150

Earlier quoted context omitted.

> But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case Seems like the same issue as the evil vector [1] and it could have been predicted that this would happen. > It's kind of a wild sign of the times to see a tech company issue this kind of post mortem about a flaw in its te…

Well, that's always what LLM-based AI has been. It can be incredibly convincing but the bottom line is it's just flavoring past text patterns, billions of them it's been "trained" on, which is more accurately described as compressed efficiently onto latent space. Like if someone lived for 10,000 years engaging in small talk at the bar, has heard it all, and just kind of mindlessly and intuitively replied with somethi…

> which is more accurately described as compressed efficiently onto latent space.

The actual difference between solving compression+search vs novel creative synthesis / emergent "understanding" from mere tokens is always going to be hard to spot with these huge cloud-based models that drank up the whole internet. (Yes.. this is also true for domain experts in whatever content is being generated.)

I feel like people who are very optimistic about LLM capabilities for the later just need to produce simple products to prove their case; for example, drink up all the man pages, a few thousand advanced shell scripts that are easily obtainable, and some subset of stack-overflow. And BAM, you should have a offline bash oracle that makes this tiny subset of general programming endeavor a completely solved problem.

Currently, smaller offline models still routinely confuse the semantics of "|" vs "||". (An embarrassing statistical aberration that is more like the kind of issue you'd expect with old school markov chains than a human-style category error or something.) Naturally if you take the same problem to a huge cloud model you won't have the same issue, but the argument that it "understands" anything is pointless, because the data-set is so big that of course search/compression starts to look like genuine understanding/synthesis and really the two can no longer be separated. Currently it looks more likely this fundamental problem will be "solved" with increased tool use and guess-and-check approaches. The problem then is that the basic issue just comes back anyway, because it cripples generation of an appropriate test-harness!

More devs do seem to be coming around to this measured, non-hype kind of stance gradually though. I've seen more people mentioning stuff like, "wait, why can't it write simple programs in a well specified esolang?" and similar

Post reply on HN