Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning experiments in OCaml

blog.janestreet.com

Post reply on HN
Print view View original

11–20 of 99 posts

Re: Deep learning experiments in OCaml

#11

Post by mlthoughts2018 »

deepGem wrote: post #8
mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this. What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to r…

Static typing is also a nice way to communicate design intentions. But for this to work, the annotations have to be very expressive.

I don’t know the first thing about OCaml, but I have worked professionally with Haskell and static typing is a joy when it adds clarity and makes the contracts of functions instantly readable.

Contrast this with Scala, which I have also worked with professionally and the difference is stark. Scala type annotations are much harder to read, and the mechanism of implicits can make for extremely mysterious code that looks like it shouldn’t compile and only once you track down some distant implicit that’s somehow in scope, can you make sense of the way types are flowing through some function contract.

Re: Deep learning experiments in OCaml

#12

Post by burkaman »

mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

I applied for a more entry level software position at Jane Street, and while I would have failed the interview regardless, I had the same experience where the HR person had no idea what the interview was like.

They assured me I'd be required to write OCaml, so I spent the weekend brushing up, and that I should bring my own laptop prepared with whatever development I wanted to use. In fact it was a couple "whatever language you want" questions using their floating interview laptop, which threw me off a lot. But like I said I would have failed anyway, it's the hardest interview I've ever had.

Re: Deep learning experiments in OCaml

#14

Post by mlthoughts2018 »

phonebucket wrote: post #13

This is great. Functional languages have such an elegant representation of so many mathematical concepts. It's a bit of a shame that they don't have more widespread use in scientific computing.

I would also suggest looking into Keras and PyTorch too. I think they honestly achieve a greater degree of elegance and a greater degree of mapping the programming constructs into the mental model space of the domain expert, than any FP interface to neural nets that I’ve seen yet.

Re: Deep learning experiments in OCaml

#15

Post by deepGem »

mlthoughts2018 wrote: post #11
deepGem wrote: post #8

Earlier quoted context omitted.

The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this. What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to r…

Static typing is also a nice way to communicate design intentions. But for this to work, the annotations have to be very expressive. I don’t know the first thing about OCaml, but I have worked professionally with Haskell and static typing is a joy when it adds clarity and makes the contracts of functions instantly readable. Contrast this with Scala, which I have also worked with professionally and the difference is s…

Yep agreed. Though you can communicate design intentions in comments, no ? There is another argument that programmers might not follow the comments, so strict enforcement by types helps - I don't believe in such a philosophy though. Most programmers will do the right thing and mistakes are not intentional.

Re: Deep learning experiments in OCaml

#17

Post by phonebucket »

mlthoughts2018 wrote: post #14
phonebucket wrote: post #13

This is great. Functional languages have such an elegant representation of so many mathematical concepts. It's a bit of a shame that they don't have more widespread use in scientific computing.

I would also suggest looking into Keras and PyTorch too. I think they honestly achieve a greater degree of elegance and a greater degree of mapping the programming constructs into the mental model space of the domain expert, than any FP interface to neural nets that I’ve seen yet.

I use PyTorch a lot; it's definitely my preferred framework at the moment. I just wish there was something as thoughtfully done and well-supported in a more functionally oriented language.

Flux.jl on Julia is the frontrunner in this regard, IMO. The added benefit is that being written in Julia the whole way down makes it easy for practitioners to delve into the source code and extend it in a performant way without going into the C level nitty gritty.

Re: Deep learning experiments in OCaml

#18

Post by hackermailman »

pc86 wrote: post #2

So I was lost at the VGG19 example code, but probably because I have (a) no OCaml experience; and, (b) no ML/NN experience. Still seems interesting, though. If anyone has any suggestions on basic sources for getting a background on the concepts here I'd definitely give them a read.

Look through youtube for university lectures, like these ones https://www.youtube.com/playlist?list=PL_Ig1a5kxu57NQ50jSuf0...

Most intro classes just require familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability all of which you can just lookup directly on https://www.expii.com for a short tutorial. Toolkits are usually in Python or Lua, plus the numerous textbooks 'Deep learning with python' that are around and specific DL books such as http://www.deeplearningbook.org/.

Afterwards look around for Adversarial Learning, like detecting perturbations that force mis-classification and other attacks described in papers by Carlini and Wagner. Currently there isn't a perfect defense developed for all of these attacks, except robust optimization that provably defend some of them. Attacks are an interesting area in DL you can get into since we don't have access to large resources and can only do DL on a small scale (in my case anyway).

Post reply on HN