> What we're speculating about here (at least I think we are?) is if this is a sustainable model for general development and if we can do better.
Here is the kind of difficulty I'm talking about. Instead of thinking about this on a continuum from Aeronautics & medical (where people could die) to yet another throwaway web TODO app (e.g. "general dev"), there is this tendency to say that they are "completely different" in some way. Let's be clear, the theories do help both. Side-effect free programming is useful. But you cannot take theory and just map it directly to the real world with no caveats. Just because Haskell tracks side-effects, doesn't make it superior to C in every context.
The Mars Rover shows this clearly -- C was chosen because someone was used to it, yes. But the part you didn't catch was the implicit decision that it makes no sense to use a compiler and ecosystem you don't understand the caveats to when you need to understand all the caveats to build a successful system.
Similarly, the TODO app developer isn't using Haskell because Haskell doesn't have anywhere near the libraries Python or even Go does, and building and deploying Haskell programs is somewhat of a chore compared to those. It's no contest, Go is superior to Haskell. :-) It's also one of the reasons almost anything is superior to C/C++ in this very same domain. :-) It's not about being "entrenched", because Go is way, way younger than haskell -- it's about the priorities the language designers and developers for that language have -- ie. the culture.
> If we can get our specifications right, the rest becomes trivial.
This is what I mean about theory and practice. Note the big "if" there. I totally understand the sentiment, and I wish it were true. I even have my own meta-programming based language in the wings I'd like to release some day.
But the reason I've stalled a bit is I've never seen this work in practice because our minds (and specs) tend to paper over the devilish details. I'm not saying we shouldn't use meta-programming, I'm saying that it should not become dogma. Invariably, you get caught up in details. This kind of domain specific "meta-programming" is a great bootstrap technique, but doesn't appear to be "the way" programs should be written.
The OMeta folks created a TCP stack that compiles (almost) directly from the specification. But that was a academic exercise. How many special cases do you think they cover? Does anyone really believe that the stack in question is anything but a way to bootstrap a more robust/performant implementation later on?
Similar for the PyPy folks. Yes, you can JIT compile a python interpreter, but how many years have they worked on special cases for that, and how much farther would they have gone if they hadn't used the meta-circular approach and just addressed the real problem to begin with[1]?
> Compared to compiler-assisted reasoning about side-effects, the difference between SML and O'Caml is completely trivial.
I think as a general statement, this is true, but is a poor way of thinking about things. You're not comparing OCaml to SML, you're comparing it to C++ or Java. If all the language brings is side-effect reasoning, it's not enough, because I can add decorators to C++ code to do what you're asking.
Even with "side-effect handling" these languages don't assist you with all of the side-effects someone actually cares about. Is there a decoration for runtime speed, for memory consumption, not just for IO, but for the amount of IO? Is it even predictable? These are the real things people care about, not just whether a function peers into some global state somewhere (although that is an important thing to track, it isn't the biggest issue, IMO).
> Your [2] is just absurd :). Clearly, you don't have to understand the body/implementation of a function, just its type :)
Is this sarcastic? :-) I mean, anyone with a reasonable amount of experience knows that this is not true most of the time.
It's not completely false -- one doesn't always have to look at the implementation of the operating system facilities or even the standard library. But in code that is less than tangential to what you're working on, you certainly do end up having to understand how it's implemented. As a developer, you spend more time reading code others wrote than writing it.
> More seriously, I'd be interested if there's a particular experience that soured you on FP (or perhaps Haskell, in particular)...?
I'm not soured on FP as a concept (which could mean many things, but I just mean state-awareness), just haskell. The fn call operator combined with currying in particular seems like an advancement to rubyists[2] and those think succinctness is a virtue above all others[4], but it is an engineering disaster -- completely unreadable at the call site unless you know the arity of every function by heart.[3]
But maybe I've just read bad code... I'm not dismissing that possibility. :-)
[1] The problem with python was never optimizing plain python (look at JS as an example) or the "GIL", the problem was (stupid) performance requirements about the GIL from guido, and later on, C extension API compatibility.
[2] Do not get me started on the "domain specific language" shit-fest that ruby (and progenitors like groovy/gradle) have unleashed on the world. At least stack overflow gets money and page views from it, I guess.
[3] As much as people like to put down smalltalk (cum obj-c) keyword argument syntax, it is a revelation when you're maintaining code (and I'm sorry Swift seems likely to drop it as a default).
[4] Is Arc (PG's lisp) used anywhere significant, but for this website?