Why Isn't Functional Programming the Norm? [video]

Actually, a lot of programming language improvements have come from trying to make lisp performant.

VS Code, VIM works for me. Conda or PIP also. Not sure what is missing for you.

>> but not everything in the world is glue code

I never claimed that.

>> and as soon as you need to do something O(n) on your dataset, you’re either paying an enormous performance penalty or you’re not writing that bit in Python

Depends what you need to do.

My entire comment was about that details matter and you can't just blindly pick a language because of out of the box performance.

You cannot have referential transparency and encapsulation at the same time.

In order to prevent mutations (which is a requirement of FP), a module cannot hold any state internally; this necessarily means that the state must be passed to each module action from the outside. If state has to be passed to each module action from the outside, then this necessarily means that the outside logic needs to be aware of which state is associated with which action of which child module. If higher level modules need to be aware of all the relationships between the logic and state of all lower level (child) modules, that is called 'leaky abstraction' and is a clear violation of encapsulation.

Encapsulation (AKA 'blackboxing') is a very important concept in software development. Large complex programs need to have replaceable parts and this requires encapsulation. The goal is to minimize the complexity of the contact areas between different components; the simpler the contact areas, the more interchangeable the components will be. It's like Lego blocks; all the different shapes connect to each other using the same simple interface; this gives you maximum composability.

Real world software applications need to manage and process complex state and the best way to achieve this is by dividing the state into simple fragments and allowing each fragment to be collocated with the logic that is responsible for mutating it.

If you design your programs such that your modules have clear separation of concerns, then figuring out which module is responsible for which state should be a trivial matter.

Did you watch the video? The most popular language is JavaScript, which is only not functional but a quirk of history.

The video makes an argument for marketing being the reason.

I think functional programming gives you powerful tools to reason about the construction of programs. Even down to the machine level it's amazing how amortized functional data structures change the way you think about algorithmic complexity. I think laziness was the game changer here. And if you go all in with functional programming it's surprising how much baseline performance you can get with such little effort and how easy it is to scale to multiple cores and multiple hosts.

There are some things like vectorization that most functional languages I know of are hard pressed to take advantage of so we still reach out to C for those things.

However I think we're starting to learn enough about functional programming languages and how to make efficient compilers for them these days. Some interesting research that may be landing soon that has me excited would enable a completely pure program to do register and memory mutations under the hood, so to speak, in order to boost baseline performance. I don't think we're far off from seeing a dependently typed, pure, lazy functional language that can have bounded performance guarantees... and possibly be able to compile programs that don't even need run time support from a GC.

I grew up on an Amiga, and later IBM PCs, and that instinct to think about programs in terms of a program counter, registers, and memory is baked into me. It was hard to learn a completely different paradigm 18 or so years into my professional career. And to me, I think, that's the great accident that prevented FP from being the norm: several generations were simply not exposed to it early on on our personal computers. We had no idea it was out there until some of us went to university or the Internet came along. And even then... to really understand the breakthroughs FP has made requires quite a bit of learning and learning is hard. People don't like learning. I didn't. It's painful. But it's useful and worth it and I'm convinced that FP will come to be the norm if some project can manage to overcome the network effects and incumbents.