Why functional programming?

Careful, you're arguing here that FP is better because the person using it is a better person. There is nothing wrong with that statement if it is actually an unbiased fact, but simply choosing to use or not use FP would automatically bias you towards declaring that true or false. Avoiding that line of reasoning is a good idea because it's not useful.

Stick to the merits of FP itself.

I would wager that FP is used more in universities than in any other setting, and by underclassmen, not just graduate students. So I don't exactly think this is true.

The OP was asking for cases where you'd prefer to be using FP over the type of code he is used to. Manipulating blobs of sensor data are simply unlikely to be a productive use of FP - although it could certainly be done. However, calculating statistics from that data is going to be a place where FP would shine and would be a good choice for it.

Not everything is a nail simply because you can spend hours hitting it with enough transformations to make it look like one.

EDIT: Thinking about it, fragment and vertex shaders actually seem like a very good place to use FP.

The poster was saying that FP has a certain cachet (no pun intended) that is probably due to the quality of the average programmer.

It's a bit like if someone was asked to explain higher salaries in Silicon Valley. Are we allowed to refer to the quality of programmers? Of course there is a risk of personal bias, but how could one have this discussion without referring to this factor?

Lately I've been studying the source code of Aleph, a web server written by Zach Tellman. Zach is a very smart guy, and reading his source code is an education (I've actually found this true of the majority of Clojure projects: the quality of the code is very good, and Clojure is the only language I've worked with so far where I have been able to learn so much reading the source code of the 3rd party libraries that I rely on).

Let's start with an interesting comment about Java, on StackOverflow:

Why doesn't more Java code use PipedInputStream / PipedOutputStream?

http://stackoverflow.com/questions/484119/why-doesnt-more-ja...

In response to that question, matt_b suggests this is part of the reason why people don't use pipedinputstream or pipedoutputstream:

"Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread."

So now, look at how PipedInputStream and PipedOutputStream are used in Aleph:

https://github.com/ztellman/aleph/blob/perf/src/aleph/format...

In particular, look at this function:

(defn channel->input-stream "Consumes messages from a channel that emits bytes, and feeds them into an InputStream."

  ([ch]

     (channel->input-stream ch "utf-8"))

  ([ch charset]

     (let [out (PipedOutputStream.)
           in (PipedInputStream. out 16384)
           bytes (map* #(bytes->byte-array % charset) ch)]

       (future
         (try
           (loop []
             (when-let [msg @(read-channel* bytes :on-drained nil)]
               (.write out ^bytes msg)
               (recur)))
           (finally
             (.close out))))
              
       in)))

On a different subject, I have learned a new style of programming by using closures in a functional language. Maybe I am a bit stupid, but it took me a long time before I began to realize how useful closures are. My only other exposure to closures was with Ruby and Javascript, but I never did concurrent programming with Ruby or Javascript, and it seems to me the greatest use of closures is when you are dealing with a lot of threads. When I first started writing Clojure code, I did not use closures at all. I wrote some clumsy code. For instance, I wrote a function that got some data from the database, and I wrote another function that stored that data in a hashmap in memory (using the memory as a cache) and I wrote another function that would check the hashmap for the data, before making the call to the database, and then I wrote another function that ran every 15 minutes and erased items in the hashmap, thus invalidating the cache. But then I realized I could combine several of these operations by creating a closure at the moment that I store data in the hashmap: the closure could call Thread.sleep(900000) and then delete the data from the hashmap. I found it convenient to create the closure while I was storing the data in the hashmap as, at that moment, I have all the information I need, including the key with which that data is being stored. Then I push the closure onto a queue and it gets processed in 15 minutes. This simplifies things quite a bit.

Again, Zach Tellman's code has been educating me about how powerful closures can be in the context of concurrent programming. Look here:

https://github.com/ztellman/aleph/blob/perf/src/aleph/http.c...

And look especially at this function:

(defn wrap-ring-handler "Takes a normal Ring handler, and turns it into a handler that can be consumed by Aleph's start-http-server. If the Ring handler returns an async-promise, this will be handled properly and sent along as a long-poll response whenever it is realized. This should be the outermost middleware around your function. To use an Aleph handler at a particular endpoint within the scope of this middleware, use wrap-aleph-handler."

  [f]

  (fn [ch request]

    (run-pipeline request
      {:error-handler (fn [ex] (error ch ex))}

      ;; call into handler
      (fn [{:keys [body content-type character-encoding] :as request}]
        (if (channel? body) 

          (if (options/channel-ring-requests? request)

            ;; leave channels as is
            (f (assoc request ::channel ch))

            ;; move onto another thread, since there will be blocking reads
            (task "input-stream-reader"
              (f (assoc request
                   ::channel ch
                   :body (formats/channel->input-stream body character-encoding)))))

          (f (assoc request
               ::channel ch
               :body (formats/bytes->input-stream body character-encoding)))))

      ;; send response
      (fn [response]
        (when-not (::ignore response)
          (enqueue ch response))))))

You are asking why functional programming is a big deal. I'm sure different people will have different answers. I think the difficulty in answering is that there is no one thing that by itself makes an answer, rather, it is a whole series of things that combine in powerful ways. For me, the immutable data structures of Clojure are an education. I have learned a lot. I think using closures in the context of concurrent programming could be an absolute nightmare, if it was not for the safety offered by immutable data structures. And yet, I don't think immutable data structures are a big deal by themselves, nor do I think that closures are a big deal by themselves, nor do I think concurrency is a big deal by itself. Only when I look at the combination of these 3 do I find myself enlightened, like I've just discovered a whole new level of programming, a level of power I have never felt in any other language.

A final note: Adam Bard has a blog post "Effortless async by design" in which he says "I dare you to find a thread-safer language. Well, maybe Erlang." And then he also uses some code from Zach Tellman to demonstrate what he means. Bard's post is worth a read:

http://adambard.com/blog/why-clojure-part-2-async-magic/

I could list a lot of things that seem to me very powerful about the coding style I've learned this last year: composability, currying, closures, immutable data structures, thread-safe, etc. But I think descriptions like that tend to be a bunch of meaningless buzz words until you've dived in deep enough to see how they all interact with each other. Because its the interaction of these features that is really powerful, not any of them alone.