I am not getting the exact usecase. If the speed is the issue, then I think tf.function solves that. For most of the operation we need to use python via pythonkit, so the things which can't be sped up using tf.function, can't be sped up using swift. Also if we need to use python everywhere, type safety is also very minor.
But regardless unique features, it's a have cake and eat it too type of interface. You don't need to learn a second language within the language like tensorflow's tf.* making it even more natural and flexiblethan pytorch, including all debug mechanisms of the host language itself, but you still get compile time graph creation like tensorflow, including all kinds of optimizations. It makes other approaches seem primitive by comparison, but creating it is much more complex, and the main audience is already more than used to using language within language solutions (like numpy) which can provide something almost as good even if less elegantly, so it's not easy to convince people as well (when it involves changing programming languages).