Earlier quoted context omitted.
Okay. I think you might be yelling at the wrong guy; the conclusion you seem to have drawn is not at all the assertion I was intending to make. To me, "acting like a human" is quite distinct from being a human or being afforded the same rights as humans. I'm not anthropomorphizing LLMs so much as I'm observing that they've been built to predict anthropic output. So, if you want to elicit specific behavior from them,…
> So, if you want to elicit specific behavior from them, one approach would be to ask yourself how you'd elicit that behavior from a human, and try that. This doesn't seem that human: https://www.theregister.com/2023/12/01/chatgpt_poetry_ai/ How far removed is that from: Did you really name your son "Robert'); DROP TABLE Students;--" ? I think that these issues probalisticly look like "human behavior", but they are l…
Agreed; that’s why I was very careful to say “one approach.” I suspect that technique exploits a feature of the LLM’s sampler that penalizes repetition. This simple rule is effective at stopping the model from going into linguistic loops, but appears to go wrong in the edge case where the only “correct” output is a loop.
There are certainly other approaches that work on an LLM that wouldn’t work on a human. Similar to how you might be able to get an autonomous car’s vision network to detect “stop sign” by showing it a field of what looks to us like random noise. This can be exploited for productive reasons too; I’ve seen LLM prompts that look like densely packed nonsense to me but have very helpful results.