Large language models develop novel social biases through adaptive exploration

The article's description gives some idea, but I presume you saw that: "the 'just two more things' tic, which prevented Opus from ever converging on being ready to do real work—it always wanted to fiddle with Gas Town itself."

Sounds like he ran into automated yak shaving. But it doesn't really explain why he couldn't fix it at the harness level.

Speculating, if you're trying to build something automatic, then you want constrained responses from each task you assign the model, otherwise you can get an endless explosion of work. The "Change 'Add to Cart' to blue" challenge parodies this: https://opusfived.dev/

Tangentially, reading the rest of that post gives me the impression that the author might benefit from an intervention. "AI psychosis" seems like it could be a relevant label here.

I'm thinking more the Noam Chomsky and John Searle variety. For the stochastic parrots people, which I assume you to mean the no-skin-in-the-game bloggers, I'm not concerned about them. I find a lot of the rhetoric around LLMs to be eye-roll worthy, most people slinging it often lack a coherent theory of semantics to begin with, let alone an understanding of logical induction/statistics. Take the definitional entailment that these models are ampliative. This small fact undermines quite a lot of the naive mental model people have of what on transformers even are. No intentional theory, no position driving the argument, the opposition isn't substantial. The virtue signal is valid, but from strangers is uninteresting.

As you mention the humanities is very diverse, linguistics is no exception. It's not that nobody was on the corner of distributional semantics, but it's been a long road and for much of its life results were routinely dismissed in the mainstream out of dogmatism. Noam Chomsky is a good pull because I believe he's the most prominent example of this chauvinism. Vague hand-waives about explanatory power, etc.

> I'm thinking of post-structuralists especially.

I understand what you mean. It's probably not coincidence that Chomsky is a vocal critic of the school. I think many different post-modernist camps even beyond post-structuralism actually are amenable to the implications of distributional semantics, and are probably the better equipped for it. I think there are nuanced problems unifying the two, but it's a work day so I won't elaborate.

> It would depend a lot on the specific school and instructor but in general I really don't think you would. I've taken classes like this and people were far more open minded and critical than you might assume.

I was just being cute with that, really. Although in this case, I think critique is itself the problematic lever, acceptance is more an exercise of apologetics, which is the weak point of post-structuralism in many ways.

> By "associative logics" I hand-wavingly mean all manner of categorical thinking, "amygdala" thinking, mapping, putting things into buckets, hedging.

I assumed so, but it's more or less a long phrase to repeat the same concept. Relations (IE logic), mappings, morphisms, associations, patterns, etc. Same-side, same-coin. Informal, formal, take a position of drawing no line here and the shadows scatter. The extraneous qualifiers aren't what confused me though, it's that gerund "seeking". It sticks out enough to imply additional structure, but one that isn't contextually indicated. Taken in a conservative form, I considered pattern-seeking to be interchangeable with pattern-constructing, which then returns it to just being an extraneous qualifier (hence the "bit incoherent", it's a mild interpretive non-confidence).