Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep learning experiments in OCaml

blog.janestreet.com

Post reply on HN
Print view View original

21–30 of 99 posts

Re: Deep learning experiments in OCaml

#21

Post by flavio81 »

phonebucket wrote: post #13

This is great. Functional languages have such an elegant representation of so many mathematical concepts. It's a bit of a shame that they don't have more widespread use in scientific computing.

> It's a bit of a shame that they don't have more widespread use in scientific computing

In truth, they have. Lisp was the first functional language (or the first language that allowed that paradigm), and has been used a lot in scientific computing, for example doing symbolic calculus and manipulation.

Re: Deep learning experiments in OCaml

#22

Post by flavio81 »

mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

>Now, there are perfectly valid other reasons to like static typing. I just always hear this one, especially in regards to Python, and it’s really the wrong way to look at it.

>The extra code and constraints of static typing are liabilities that should have to offer offsetting value to choose them.

Agree, agree SO much.

After years of only using statically-typed languages, and then switching to Python and Lisp, i never understood why "catching typos and type errors" was touted as the benefit. I also agree they are like a sort-of liability that has to be taken into account in order to turn it around into a benefit.

For me, it was mostly performance benefits.

Note that not talking about strong typing (vs weak typing). Strong typing is always a good thing.

Re: Deep learning experiments in OCaml

#23

Post by nnq »

deepGem wrote: post #8
mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this. What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to r…

> You'd be a moron to pass an argument of a different type 'a couple of hours' into the training

Huh?! At least in non-ML code this happens all the time, data fetched by whatever thinggie that uses zillion chained libraries of code nobody has time to audit, comes in hours or days late in a long running service blowing it up... eg. "oops, point.x is now no longer and integer but more like a map[ErrorObject->vector[int]]" bc something blew up in a very unexpected way in some other nodejs code light years away from the business logic you hold in your head... (yeah, the service gets restarted, but at some point some data that should have been saved in the DB hasn't been am may need to be recovered manually from some obscure log if even recoverable)

Re: Deep learning experiments in OCaml

#24

Post by Nelkins »

mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

Agreed that type-safety preventing training script failure is not the strongest argument for OCaml. In my experience, the far more compelling reason to use expressive type systems is that they allow you to be more specific about your domain models. This helps to not just prevent type errors (common but not that big of a deal) but also logic errors (I think more common, harder to suss out, and harder to catch with the equivalent amount of test coverage). This idea is often rephrased in FP communities as "Make illegal states unrepresentable."

Re: Deep learning experiments in OCaml

#25

Post by deepGem »

nnq wrote: post #23
deepGem wrote: post #8

Earlier quoted context omitted.

The type safety argument is total BS. First of all the training script will fail for the very first time if there is a type error. You'd be a moron to pass an argument of a different type 'a couple of hours' into the training. No sane programmer writes such code. What kind of nonsensical argument is this. What I have found static typing to be really useful for is in remembering what I have coded. It's quite hard to r…

> You'd be a moron to pass an argument of a different type 'a couple of hours' into the training Huh?! At least in non-ML code this happens all the time, data fetched by whatever thinggie that uses zillion chained libraries of code nobody has time to audit, comes in hours or days late in a long running service blowing it up... eg. "oops, point.x is now no longer and integer but more like a map[ErrorObject->vector[int…

Yeah I should have been more explicit that this comment is in reference to ML code. Training is nothing but a loop so its unlikely you pass type A in iteration 1 and type B in iteration 200. If that happens most likely your training data is messed up and type safety cannot help you as you would have compiled and tested for type A.

Re: Deep learning experiments in OCaml

#26

Post by mlthoughts2018 »

deepGem wrote: post #25
nnq wrote: post #23

Earlier quoted context omitted.

> You'd be a moron to pass an argument of a different type 'a couple of hours' into the training Huh?! At least in non-ML code this happens all the time, data fetched by whatever thinggie that uses zillion chained libraries of code nobody has time to audit, comes in hours or days late in a long running service blowing it up... eg. "oops, point.x is now no longer and integer but more like a map[ErrorObject->vector[int…

Yeah I should have been more explicit that this comment is in reference to ML code. Training is nothing but a loop so its unlikely you pass type A in iteration 1 and type B in iteration 200. If that happens most likely your training data is messed up and type safety cannot help you as you would have compiled and tested for type A.

You could also have a complex training job that trains a shallow model for a while, then uses it to train a deeper model, or extract an embedding from some layer and train a classical prediction model that uses the embedding as the feature vector.

But the point is that the right way to ensure safety is with realistic fixture-based integration testing. That’s not what static typing is for in that type of use case and is not a de facto benefit of static typing.

Re: Deep learning experiments in OCaml

#27

Post by nikofeyn »

flavio81 wrote: post #21
phonebucket wrote: post #13

This is great. Functional languages have such an elegant representation of so many mathematical concepts. It's a bit of a shame that they don't have more widespread use in scientific computing.

> It's a bit of a shame that they don't have more widespread use in scientific computing In truth, they have. Lisp was the first functional language (or the first language that allowed that paradigm), and has been used a lot in scientific computing, for example doing symbolic calculus and manipulation.

that doesn't matter when very few scientists have even heard of an ml (standard ml, ocaml, f#) or a lisp/scheme (common lisp, racket), much less have an inkling to use them. their use does of course exist but by no measure is it widespread.

Re: Deep learning experiments in OCaml

#28

Post by typon »

mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

I've always thought that static typing is the least interesting thing about functional languages.

Re: Deep learning experiments in OCaml

#29

Post by nikofeyn »

burkaman wrote: post #12
mlthoughts2018 wrote: post #4

I had a very unpleasant interview regarding deep learning with Jane Street. I spoke to a member of their HR team to try to get significant assurances that the interview would actually be focused on deep learning and not puzzles or brain teasers, and that the job would really focus on deep learning for their actual business, and not just be a proxy for being generally smart and then work on whatever existing inhouse m…

I applied for a more entry level software position at Jane Street, and while I would have failed the interview regardless, I had the same experience where the HR person had no idea what the interview was like. They assured me I'd be required to write OCaml, so I spent the weekend brushing up, and that I should bring my own laptop prepared with whatever development I wanted to use. In fact it was a couple "whatever la…

i wouldn't sweat it. people design those interviews to make them feel better about themselves rather than as an effective way to gauge someone's aptitude to work there.

you'll notice that none of these places that ask these types of questions allow the candidate to ask them technical questions. it's always a one way street. there's times where i have "failed" interviews of this type when i could guarantee i have "simple" questions that they couldn't answer about software and programming.

Re: Deep learning experiments in OCaml

#30

Post by amelius »

phonebucket wrote: post #17
mlthoughts2018 wrote: post #14

Earlier quoted context omitted.

I would also suggest looking into Keras and PyTorch too. I think they honestly achieve a greater degree of elegance and a greater degree of mapping the programming constructs into the mental model space of the domain expert, than any FP interface to neural nets that I’ve seen yet.

I use PyTorch a lot; it's definitely my preferred framework at the moment. I just wish there was something as thoughtfully done and well-supported in a more functionally oriented language. Flux.jl on Julia is the frontrunner in this regard, IMO. The added benefit is that being written in Julia the whole way down makes it easy for practitioners to delve into the source code and extend it in a performant way without go…

My biggest wish is that new research implementations in the field of ML come out in 1 framework, in 1 language.

But for engineering purposes, it's nice that there is an ocaml framework now.

Post reply on HN