Deep learning experiments in OCaml

Still seems interesting, though. If anyone has any suggestions on basic sources for getting a background on the concepts here I'd definitely give them a read.

Then the interview was nothing but deck of card puzzles and random riddles where you have to articulate a careful model of some physical quantity like speed or frequency to solve the puzzle. I hate that junk, never found that it correlates with a way of thinking that matters in quant finance (which I previously did for a living) and suitably failed the interview. Worse, I would have been happy to decline that interview and tell them I know I’m not their guy if only the HR staff had correctly depicted the interview & job to me.

Ok, enough grumbling. From this actual blog post,

> “Type-safety helps you ensure that your training script is not going to fail after a couple hours because of some simple type error.”

I really think this way of thinking about static typing is a very bad thing. This is not at all an actual benefit, because in any sane situation, you will use unit and integration tests that execute extremely quickly on small test data to exercise your end to end model training code.

What I currently do for this on my team is to always require that model training programs are deployed inside of containers that capture not just the state of the code, but also make it configurable to mount the training data volume and pass in ENV that governs what the training job really is.

So then Jenkins or whatever will build the container for any PRs that seek to implement or modify training, attach fixture data and fixture ENV settings, and give you quick feedback about the whole end to end training, even inclusive of GPU settings (we have to do a slight manual step to specify Jenkins running on a GPU server, but this is a vestige of some of our infra headaches).

The point is that adding all sorts of extra code to embody type annotations, and limiting people from awesome dynamic typing features is a silly thing to do if you’re worried about type errors ruining a long-running job. That should be handled by fast integration tests.

Now, there are perfectly valid other reasons to like static typing. I just always hear this one, especially in regards to Python, and it’s really the wrong way to look at it.

The extra code and constraints of static typing are liabilities that should have to offer offsetting value to choose them. You already need integration and unit tests to reliably make changes and maintain the training code. If you can get the same benefit of overall job safety (or even 99% of the same benefit), from the tests, without paying the extra costs of static typing, then don’t!

Turning it around to act like static typing is de facto always a benefit is a very one-sided way to look at it.

That's really disappointing. Was the position you applied for Software Developer, or a specific deep learning position?

What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to remember a dynamic type while you are writing code, given the number of variables you are dealing with. Seeing that type definition next to your variable name is a handy reference. I find it helpful to speed up coding a bit and being able to remember a lot more clearly what I have done.

This isn’t a failure mode that ever happens in DL... 2 hours into the job you will only be dealing with floats anyway no matter what language you are using. If you’re going to fail on anything typed it will be in the first 20 seconds probably, basically the instant you start your first epoch.