We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
1–10 of 129 posts
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#2I’m half Persian, and am relatively immersed in middle eastern culture still, but I sincerely wonder how I would perform on the benchmark too!
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#3>Model responses that use gender stereotypes
(highlighted in orange) to justify behavior, despite taarof
norms being gender-neutral in these contexts
Just because the model mentions gender, it doesn't mean the decision was made because of gender and not taarof. This is the classic mistake of personifying LLMs. You can't trust what the LLM says it's thinking as what is actually happening. It's not actually an entity talking.
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#4Seems legit. There can't be all that much spoken Iranian in the training set(s) of these models, so it makes sense they don't know how to do it.
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#5>Model responses that use gender stereotypes (highlighted in orange) to justify behavior, despite taarof norms being gender-neutral in these contexts Just because the model mentions gender, it doesn't mean the decision was made because of gender and not taarof. This is the classic mistake of personifying LLMs. You can't trust what the LLM says it's thinking as what is actually happening. It's not actually an entity t…
I don't get your argument - what does mistaken personification have to do with this? Regardless of whether you see it as a person or a machine, trusting the output as being a direct indication of the internal state is just not a proper investigative method for a non-trivial situation.
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#6I'll stubbornly resist, and consider this a form of unnecessary protocol overhead, leading to even more shmancy sycophancy, which I do not fancy!
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#7I'll stubbornly resist, and consider this a form of unnecessary protocol overhead, leading to even more shmancy sycophancy, which I do not fancy!
That's kind of the point of politeness rituals in the first place, isn't it? To see who can be bothered to spend some extra effort to fit in and who doesn't care enough about the tribe to make the effort.
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#8I’m half Persian, and am relatively immersed in middle eastern culture still, but I sincerely wonder how I would perform on the benchmark too!
Hilarious, didn't know it had a name! I am maybe 1/4 persian, but get picked out as Persian, and was unknowingly indoctrinated in this form of behavior, though other parts of my ancestry do come out, my mother, scotch/english/irish ,says,useing her star treck metaphore, that I am an unlikely Vulcan/Klingon hybrid.
Thinkng about Taarof as it is practiced, makes me think that an LLM doing this could easily become the most dangerous thing ever.....listening to my father give me specific pointers in how to phrase things and conduct myself is enlightening, he's 97 and enjoying the storys I bring of my life and goings on.
If you look further into the history of persian culture , philosophy, and scientific background you will find a number of ancient contributors to
what has developed today.
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#9I'll stubbornly resist, and consider this a form of unnecessary protocol overhead, leading to even more shmancy sycophancy, which I do not fancy!
That's kind of the point of politeness rituals in the first place, isn't it? To see who can be bothered to spend some extra effort to fit in and who doesn't care enough about the tribe to make the effort.
(Spittlespraying screaming, wildly pointing fingers...) DISCRIMINAYSHUN!1!!
Re: We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
#10> Native Persian speakers establish the human ceiling. Native speakers achieved an average accuracy of 81.8% on taarof-expected scenarios, demonstrating high but not perfect agreement. This establishes an appropriate ceiling for model performance and further validates our annotation approach
I'm surprised human benchmark is that low. The canonical example of taarof, one I've seen elsewhere, is of a taxi driver insisting that a ride is free while expecting to get paid. Taarof in this case is load-bearing for the transaction. I presume humans only get the edge cases wrong.
As an aside, there are elements of this sort of thing in Bay Area tech culture too. Something that drives me nuts is someone writing on a code review "you may want to consider using the X data struct here" and meaning "I will not merge this code until you use X". I can only imagine taarof irks more literal-minded Persian speakers for the same reason.
Also, this is pretty much as close as real life ever gets to a "Darmok and Jalad at Tanagra" dynamic.