Earlier quoted context omitted.
Kotlin has lots of syntax sugar but the semantics are crazy. It's essentially an imperative language where the only way to reason about effects (suspend functions) is by reasoning about control flow. That already makes things like error handling incomprehensible, and it'll be a really bad fit for Spark's way of doing things (where values have to be serializable and shipped around the cluster) - imagine trying to use…
I'm curious what your use case is for using a suspend function in spark. I have written several spark jobs with Kotlin and have never had a reason to create one. The only difference from the Scala/Java jobs is that I had to be much more explicit with certain things (encoders in higher order funcs, etc.).
Re: From First Principles: Why Scala?
#341All of the reasons you'd want to use a suspend function at all - e.g. mutually recursive functions, async I/O (e.g. database queries), error handling - suspend functions seem to be kotlin's answer to anything even slightly non-vanilla. Writing in Scala I can, with care, use async and iteratees in spark and be able to reason about what's going to get executed where and when, because these things are represented as values.