Earlier quoted context omitted.
I think they kind of have. Musk replied "that's fine" to Ye posting the Musk yacht photo and "that is not" to the Star of David Nazi symbol which suggests that was the final line for the ban. The "going deathcon one on the Jews", which led to the pre-Musk ban, is arguably a more explictly a violent threat than the symbol.
The swastika itself represents historical violence on the Jews. I would argue it alone is an explicit threat of violence because I can't see what other meaning today it could have?
wikipedia.org/wiki/Swastika
So when kayne used it, of course he referred to the Nazi symbol and just wanted attention and strongly hint at his antisemitism.
But other people can mean something very different:
"In several major Indo-European religions, the swastika symbolises lightning bolts, representing the thunder god and the king of the gods, such as Indra in Vedic Hinduism, Zeus in the ancient Greek religion, Jupiter in the ancient Roman religion, and Thor in the ancient Germanic religion."