I would say the aspect that defines a functional programming language is the support of higher order functions. I.e. functions that can take functions as arguments and more importantly can return functions as return value.
Rather than saying that higher-order functions is the defining feature of functional programming, I would say that it's an essential feature to support its object of defining programs in terms of (mathematical) function application rather than procedural state changes.
For example, an instance map is just a cons and a recursive call. No state change needed. You can say something about exposing unwanted details in the cons, but that is a hairier point of contention.