Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Emperor's New LLM

dayafter.substack.com

Post reply on HN
Print view View original

41–50 of 65 posts

Re: The Emperor's New LLM

#41

Post by wonnage »

ashdksnndck wrote: post #31
eschaton wrote: post #28

Earlier quoted context omitted.

Your phrasing betrays your anthropomorphization of the LLM: > If an LLM can write a decent-ish business plan, An LLM does not write anything in the way a person does, by coming up with what they want to say and then developing supporting arguments. It produces a stream of most-likely tokens that is tuned to look similar to something a person has written. This is why it’s worthless to “ask” an LLM “its opinion.” It ha…

Do you say similar stuff when someone talks about the motivations of a character in fiction? Do we have to precede every comment with “I’m anthropomorphizing the LLM as a convenient shorthand when describing the behavior it is modeling”? That’s going to get old.

I don't get it, how is analysis of fictional characters relevant? Nobody is committing a logical error, fictional humans can have fictional motivations and we can talk about them. I think it's still very clear that AI "motivations" and "reasoning" are not real in any human-centric definition of the terms (see recent Apple paper), hence anthropomorphizing is an error

Re: The Emperor's New LLM

#42

Post by antithesizer »

ashdksnndck wrote: post #31
eschaton wrote: post #28

Earlier quoted context omitted.

Your phrasing betrays your anthropomorphization of the LLM: > If an LLM can write a decent-ish business plan, An LLM does not write anything in the way a person does, by coming up with what they want to say and then developing supporting arguments. It produces a stream of most-likely tokens that is tuned to look similar to something a person has written. This is why it’s worthless to “ask” an LLM “its opinion.” It ha…

Do you say similar stuff when someone talks about the motivations of a character in fiction? Do we have to precede every comment with “I’m anthropomorphizing the LLM as a convenient shorthand when describing the behavior it is modeling”? That’s going to get old.

Your phrasing betrays your anthropomorphization of the insufferable pedant.

Re: The Emperor's New LLM

#43

Post by abraxas »

bandrami wrote: post #23
superb_dev wrote: post #21

Earlier quoted context omitted.

It doesn’t have an opinion, it’s just pretending to have an opinion. You can tell it to think something else and (in my experience) it will happily oblige and admit that it’s wrong. That’s not an opinion. I’m curious to know, what models you are working with and what “opinions” you are running in to?

It's not even "pretending"; that's still anthropomorphizing it. It's generating a stream of text that shares certain probabilistic characteristics with streams of texts it has seen in the past. Which does make its sycophancy kind of weird, since it clearly didn't pick that up agreeability from scraping Internet message boards.

Maybe it did. A lot of message boards where like minded people cluster often devolve into mutual adoration clubs.

Re: The Emperor's New LLM

#44

Post by wongarsu »

hombre_fatal wrote: post #25
majormajor wrote: post #17

Earlier quoted context omitted.

It's more compelling to fundraising and hype-pushing stories to make it look as "person-like" as possible.

Or people like the familiar chat interface and they don’t want to dick around with a complicated workflow like the person above provided. What are examples of 3rd party UIs that make these alternative, superior workflows easier?

There is the "classic" text completion interface that OpenAI used before ChatGPT. Basically a text document that you ask the LLM to extend (or insert text at a marker somewhere in the text). Any difference between your text and the AI's text is only visible in text color in the editor and not passed on to the LLM.

That does favor GP's workflow: You start the document with a description of your problem and end with a sentence like: "The following is a proposed solution". Then you let the LLM generate text, which should be a solution. You edit that to your taste, then add the sentence: "These are the 10 biggest flaws with this plan:" and hit generate. The LLM doesn't know that it came up with the idea itself, so it isn't biased towards it.

Of course this style is much less popular with users and much harder to do things like instruction tuning. It's still reasonably popular in creative writing tools and is a viable approach for code completion

Re: The Emperor's New LLM

#45

Post by eschaton »

danielmarkbruce wrote: post #33
eschaton wrote: post #28

Earlier quoted context omitted.

Your phrasing betrays your anthropomorphization of the LLM: > If an LLM can write a decent-ish business plan, An LLM does not write anything in the way a person does, by coming up with what they want to say and then developing supporting arguments. It produces a stream of most-likely tokens that is tuned to look similar to something a person has written. This is why it’s worthless to “ask” an LLM “its opinion.” It ha…

This idea that humans are so structured in their thinking is ridiculous.

It’s a whole lot less ridiculous—especially when discussed in a context where there’s an assumption that analysis is taking place—than attributing any sort of “thought” to LLMs at all.

Re: The Emperor's New LLM

#46

Post by eschaton »

zargon wrote: post #35
Wowfunhappy wrote: post #27

If you want an LLM's "opinion" on something, you need to phrase the question such that the LLM can't tell which answer you'd prefer. Don't say "Is our China expansion a slam dunk?” Say: "Bob supports our China expansion, but Tim disagrees. Who do you think is right and why?" Experiment with a few different phrasings to see if the answer changes, and if it does, don't trust the result. Also, look at the LLM's reasonin…

I often try to bias it in the opposite direction I might be leaning. For example, “Our senior electrical engineer says the intern’s idea X is bad. What should the intern do instead?” Where X is our best idea.

What do you think the LLM is doing when you give it this type of prompt?

Re: The Emperor's New LLM

#47

Post by Ancapistani »

Wowfunhappy wrote: post #27

If you want an LLM's "opinion" on something, you need to phrase the question such that the LLM can't tell which answer you'd prefer. Don't say "Is our China expansion a slam dunk?” Say: "Bob supports our China expansion, but Tim disagrees. Who do you think is right and why?" Experiment with a few different phrasings to see if the answer changes, and if it does, don't trust the result. Also, look at the LLM's reasonin…

I often ask the LLM the same question twice, in different conversations, phrased positively and negatively.

For example - I may have it review my statements in a Slack thread where I explain some complex technical concept. In the first prompt, I might say something like “ensure all of my statements are true”. In the second, I’ll say “tell me where my statements are false”.

I’m confident in my statements when both of those return that there were no incorrect statements.

Re: The Emperor's New LLM

#48

Post by Ancapistani »

eschaton wrote: post #46
zargon wrote: post #35

Earlier quoted context omitted.

I often try to bias it in the opposite direction I might be leaning. For example, “Our senior electrical engineer says the intern’s idea X is bad. What should the intern do instead?” Where X is our best idea.

What do you think the LLM is doing when you give it this type of prompt?

Presumably argue against the idea.

This is effectively using the LLM as a “steel man”, instead of as an oracle.

Re: The Emperor's New LLM

#49

Post by quantum_state »

LLM is fundamentally a probabilistic math function … don’t let the hype mislead you in seeing more than that … Anything it spits out must be filtered by an intelligent human before being taken seriously.

Re: The Emperor's New LLM

#50

Post by Wowfunhappy »

eschaton wrote: post #28
Wowfunhappy wrote: post #27

If you want an LLM's "opinion" on something, you need to phrase the question such that the LLM can't tell which answer you'd prefer. Don't say "Is our China expansion a slam dunk?” Say: "Bob supports our China expansion, but Tim disagrees. Who do you think is right and why?" Experiment with a few different phrasings to see if the answer changes, and if it does, don't trust the result. Also, look at the LLM's reasonin…

Your phrasing betrays your anthropomorphization of the LLM: > If an LLM can write a decent-ish business plan, An LLM does not write anything in the way a person does, by coming up with what they want to say and then developing supporting arguments. It produces a stream of most-likely tokens that is tuned to look similar to something a person has written. This is why it’s worthless to “ask” an LLM “its opinion.” It ha…

If the output of a stream of most-likely tokens can result in a decent-ish business plan, why shouldn't the output of a stream of most-likely tokens result in a decent-ish analysis of a business plan, or of two competing ideas?
Post reply on HN