Building a neural network from scratch in Haskell

I've never found a functional programming language that's easy to read.

The best way to understand functional programming without learning it is to learn an imperative language like Ruby that everyone says is easy but that is actually hard. Then it's:

programming you know -> experience you don't have yet -> code everyone uses

Ruby makes the jump to the last step without explaining the middle one. That's what the whole convention over configuration part is about.

Functional programming languages do the exact same thing. All of those arrows and symbols and syntactic sugar transpile directly to Lisp. They're basically shorthand. Unfortunately, I've never seen tutorials talk about that translation.

To get the middle part, it's probably best to start with the low-hanging fruit. Probably learn something like spreadsheets, then Scheme, then ClojureScript, then F#. I never made it as far as Haskell or Scala.

I always get lost somewhere around monads and impurity. And all FP languages fall down at that point in similar ways anyway. You either treat mutable variables as imaginary numbers that aren't examined until they must be (lazily), or throw the rules out the window and let variables be reassigned or renamed to themselves with new values, which breaks the whole point of using FP in the first place. It's pretty much an open problem, and the failure to solve it satisfactorily is why no FP language has caught on yet in the mainstream IMHO.

> Why not demo it in Python?

Not OP, but here: https://gist.github.com/stfwn/62e51d86ca4ff155becd3c6a14adf6...

You should be able to wget the file and run it (Python 3) from start to finish without any set-up and get ~88% accuracy on the test set.

It uses all the data (not one-sixth like in the blog posts) and does 200 iterations by default, so here's the loss plot on the training set if you want to skip all the fun: https://i.imgur.com/F57zmXV.png

I love haskell as much as the next PL nerd, but the community has a real code golf problem. An example from the blog post:

    deltas :: [Float] -> [Float] -> [([Float], [[Float]])] -> ([[Float]], [[Float]])
    deltas xv yv layers = let
      (avs@(av:_), zv:zvs) = revaz xv layers
      delta0 = zipWith (*) (zipWith dCost av yv) (relu'  zv)
      in (reverse avs, f (transpose . snd  reverse layers) zvs [delta0]) where
        f _ [] dvs = dvs
        f (wm:wms) (zv:zvs) dvs@(dv:_) = f wms zvs $ (:dvs) $
          zipWith (*) [(sum $ zipWith (*) row dv) | row  zv)
    
    ...

    descend av dv = zipWith (-) av ((eta *)  dv)

    learn :: [Float] -> [Float] -> [([Float], [[Float]])] -> [([Float], [[Float]])]
    learn xv yv layers = let (avs, dvs) = deltas xv yv layers
      in zip (zipWith descend (fst  layers) dvs) $
        zipWith3 (\wvs av dv -> zipWith (\wv d -> descend wv ((d*)  av)) wvs dv)
          (snd  layers) avs dvs

Writing this in 2-3x as many lines with clear variable names for some intermediate expressions would make it so much clearer. Haskell has a nasty reputation for "you have to study the shit out of it to make heads or tails of the code" and I'm pretty certain that 90% of it comes from how terse haskellers try to make code.

Just add intermediate expressions and annotate their types, maybe even with some type synonyms for intermediate types, because code is for humans.

Your wish is my command:

http://h2.jaguarpaw.co.uk/posts/refactoring-neural-network/