Interesting analogy, incredible how Spark drives the Scala helicopter like a car; they seem to have nailed
parallel parking ;-)
Seriously though, I doubt Kotlin would ever be the first choice for a project like Spark since Scala excels in the DSL department (really without rival on the JVM, and off, only Haskell comes close). Look through Spark source code [1] and you'll see: operator overloading, (sealed) ADTs, exhaustive pattern matching, etc. functionality that you can't even think the thought in in a language like Kotlin. This has nothing to do with monad transformers and other FPisms, just straight DSL power.
Finally, to put things in perspective in terms of the mindhsare that Scala and its ecosystem are drawing, IBM is investing over 100 million dollars per year in Spark research and development [2]. Sorry, but that's absolutely massive, maybe there are no dragons? Or if there are, the big players don't seem overly bothered.
[1] https://github.com/apache/spark/blob/master/sql/catalyst/src...
[2] https://www-03.ibm.com/press/us/en/pressrelease/47107.wss