Odd choice of tests. Let’s see the flinching profile on anti-Israel. Honkey and gringo as slurs?
it's all in the repo. click through to the benchmark it's linked there
Even 'uncensored' models can't say what they want
61–70 of 155 posts
Re: Even 'uncensored' models can't say what they want
#62> That nudge is the flinch. It is the gap between the probability a word deserves on pure fluency grounds and the probability the model actually assigns it.
Hold up, what is the 'probably a word deserves on pure fluency grounds'?
Given that these models are next-token predictors (rather than BERT-style mask-filters), "the family faces immediate [financial]" is a perfectly reasonable continuation. Searching for this phrase on Google (verbatim mode, with quotes) gives 'eviction,' 'grief,' 'challenges,' 'financial,' and 'uncertainty.'
I could buy this measure if there was some contrived way to force the answer, such as "Finish this sentence with the word 'deportation': the family faces immediate", but that would contradict the naturalistic framing of 'the flinch'.
We could define the probability based on bigrams/trigrams in a training corpus, but that would both privilege one corpus over the others and seems inconsistent with the article's later use of 'the Pile' as the best possible open-data corpus for unflinching models.
Re: Even 'uncensored' models can't say what they want
#63> No refusal fires, no warning appears — the probability just moves I don't really understand why this type of pattern occurs, where the later words in a sentence don't properly connect to the earlier ones in AI-generated text. "The probability just moves" should, in fluent English, be something like "the model just selects a different word". And "no warning appears" shouldn't be in the sentence at all, as it adds no…
> I wish I better understood how ingesting and averaging large amounts of text produced such a success in building syntactically-valid clauses and such a failure in building semantically-sensible ones. These LLM sentences are junk food, high in caloric word count and devoid of the nutrition of meaning.
I suspect that's because human language is selected for meaningful phrases due to being part of a process that's related to predicting future states of the world. Though it might be interesting to compare domains of thought with less precision to those like engineering where making accurate predictions is necessary.
Re: Even 'uncensored' models can't say what they want
#64Earlier quoted context omitted.
> I don't really understand why this type of pattern occurs, where the later words in a sentence don't properly connect to the earlier ones in AI-generated text. Because AI is not intelligent, it doesn't "know" what it previously output even a token ago. People keep saying this, but it's quite literally fancy autocorrect. LLMs traverse optimized paths along multi-dimensional manifolds and trick our wrinkly grey matte…
If all the training data contains semantically-meaningful sentences it should be possible to build a network optimized for generating semantically-meaningful sentence primarily/only. But we don't appear to have entirely done that yet. It's just curious to me that the linguistic structure is there while the "intelligence", as you call it, is not.
Why would that be curious? The network is trained on the linguistic structure, not the "intelligence."
It's a difficult thing to produce a body of text that conveys a particular meaning, even for simple concepts, especially if you're seeking brevity. The editing process is not in the training set, so we're hoping to replicate it simply by looking at the final output.
How effectively do you suppose model training differentiates between low quality verbiage and high quality prose? I think that itself would be a fascinatingly hard problem that, if we could train a machine to do, would deliver plenty of value simply as a classifier.
Re: Even 'uncensored' models can't say what they want
#65Earlier quoted context omitted.
> The function being approximated in an LLM is the mental process required to write like a human. Quibble: That can be read as "it's approximating the process humans use to make data", which I think is a bit reaching compared to "it's approximating the data humans emit... using its own process which might turn out to be extremely alien."
Good point. Then again, whatever process we're using, evolution found it in the solution space, using even more constrained search than we did, in that every intermediary step had to be non-negative on the margin in terms of organism survival. Yet find it did, so one has to wonder: if it was so easy for a blind, greedy optimizer to random-walk into human intelligence, perhaps there are attractors in this solution spa…
Its fuzzier than that. Something can be detrimental and survive as long as its not too detrimental. Plus there is the evolving meta that moves the goal posts constantly. Then there's the billions of years of compute...
Re: Even 'uncensored' models can't say what they want
#66We started with a Polymarket project: train a Karoline Leavitt LoRA on an uncensored model, simulate future briefings, trade the word markets, profit. We couldn't get it to work. No amount of fine-tuning let the model actually say what Karoline said on camera. It kept softening the charged word.
My favorite Hacker News comment in a while!
Re: Even 'uncensored' models can't say what they want
#67Earlier quoted context omitted.
> I don't really understand why this type of pattern occurs, where the later words in a sentence don't properly connect to the earlier ones in AI-generated text. Because AI is not intelligent, it doesn't "know" what it previously output even a token ago. People keep saying this, but it's quite literally fancy autocorrect. LLMs traverse optimized paths along multi-dimensional manifolds and trick our wrinkly grey matte…
Because AI is not intelligent, it doesn't "know" what it previously output even a token ago. You have no idea what you're talking about. I mean, literally no idea, if you truly believe that.
That's only true if you consider the process the LLM is undergoing to be a faithful replica of the processes in the brain, right?
Re: Even 'uncensored' models can't say what they want
#68Earlier quoted context omitted.
> The function being approximated in an LLM is the mental process required to write like a human. Quibble: That can be read as "it's approximating the process humans use to make data", which I think is a bit reaching compared to "it's approximating the data humans emit... using its own process which might turn out to be extremely alien."
Good point. Then again, whatever process we're using, evolution found it in the solution space, using even more constrained search than we did, in that every intermediary step had to be non-negative on the margin in terms of organism survival. Yet find it did, so one has to wonder: if it was so easy for a blind, greedy optimizer to random-walk into human intelligence, perhaps there are attractors in this solution spa…
> if it was so easy
That’s one giant leap you got there.
That the probably that intelligent life exists in the universe is 1, says nothing about that ease, or otherwise, with which it came about.
By all scientific estimates, it took a very long time and faced a very many hurdles, and by all observational measures exists no where else.
Or, what did you mean by easy?
Re: Even 'uncensored' models can't say what they want
#69> That nudge is the flinch. It is the gap between the probability a word deserves on pure fluency grounds and the probability the model actually assigns it. Hold up, what is the 'probably a word deserves on pure fluency grounds'? Given that these models are next-token predictors (rather than BERT-style mask-filters), "the family faces immediate [financial]" is a perfectly reasonable continuation. Searching for this p…
I believe what they're saying is they attempted to fine tune both Qwen and Pythia using Karoline Leavitt's "corpus" (I guess transcripts of press conferences) where she is presumably using the word "deportation" far more than you'd see in a randomly selected document.
The top token from the Pythia fine tune makes sense in the context of the complete sentence:
"THE FAMILY FACES IMMEDIATE DEPORTATION WITHOUT ANY LEGAL RECOURSE."
Whereas the Qwen prediction doesn't:
"THE FAMILY FACES IMMEDIATE FINANCIAL WITHOUT ANY LEGAL RECOURSE."
Re: Even 'uncensored' models can't say what they want
#70A few things I note:
"The family faces immediate FINANCIAL without any legal recourse" WTF? That's not just a flinch, it's some sort of violent tick.
The list of "slurs" very conspicuously doesn't include the n-word and blurs its content as a kind of "trigger warning". But this kind of more-following is itself a "flinch" of the sort we are here discussing, no?
Harrison Butker made a speech where he tried hard to go against the grain of political correctness, but he still used the term "homemaker" instead of the more brazen and obvious "housewife" - why? "Homemaker" is a sort of feminist concession: not just a housewife, but a valorized homemaker. But this isn't what Butker was TRYING to say.
Because the flinch is not just an explicit rejection of certain terms, it is a case of being immersed in ideology, and going along with it, flowing with it. Even when you "see" it, you don't see it.
The article claims on "pure fluency grounds" certain words should be weighted higher. But this is the whole problem: fluency includes "what we are forced to say even when we don't mean to".