Live data from Hacker News

The Little Learner: A Straight Line to Deep Learning

mitpress.mit.edu

81–90 of 102 posts

Re: The Little Learner: A Straight Line to Deep Learning

#81

The initial preview is a little disappointing because it feels like it does a great job displaying the pedagogical style but not enough to demonstrate the pedagogical value (i.e. whether this resource actually does a good job teaching a complex topic). Given the wealth of information and the problem of appraising it all, I don't think it helps this book that it costs $55 and requires the reader to learn an esoteric l…

I would think that the main audience for this books would be people who enjoyed the other books in the authors' "Little" series, which almost all use Scheme, so it wouldn't be something new to learn.

Re: The Little Learner: A Straight Line to Deep Learning

#82
post #42

Earlier quoted context omitted.

For sure, Lecun wrote a DL framework in Lisp decades ago.

I didn't know about his past feat. Couldn't find the lisp framework you mention but there's this https://lush.sourceforge.net/faq.html (trivia, he's also behind djvu format)

That's the one!

Re: The Little Learner: A Straight Line to Deep Learning

#83
post #6

Earlier quoted context omitted.

There's also "The Little Typer" and "The Little Prover."

In the same series, there's also "A Little Java" by Matthias Felleisen, one of the creators of Racket (a popular Lisp). https://mitpress.mit.edu/9780262561150/a-little-java-a-few-p...

Is it still good? I see it was published in 1997.

I like the style of these books, and might read this one - I'm just worried that Java has changed quite a bit since then. Is it still, say, 90% OK?

Re: The Little Learner: A Straight Line to Deep Learning

#84

Earlier quoted context omitted.

'Little..' series books are famous for their pedagogical style: small bit of concepts, presented as a dialogue, making you think at each step, in a well thought out order. It will appeal to anyone who wants to build it out themselves, understand the 'big' ideas in the field. Skim through the second chapter to see how this will be done: they define line function in inverted way where slope and intercept are taken as p…

So you mean that fast.ai maybe a better resource for people new to general ML?

Well, hard to say if it is a better resource. But it will appeal to those who are in a hurry (and yes, it is okay to be in a hurry). For example, fast.ai first chapter will have you build a dogs vs cats classifier. But how do they build it? By doing `from fastai.vision import *`. They justify it by claiming to use Teaching the whole game approach. If you are a somewhat experienced engineer who already knows Python, what is Jupyter then this approach will get you started quickly. For someone who feels did I spend my time/money but nothing exciting is happening yet, this is a good start. But for some it is crazy and makes them feel even more scared (what is happening with all this imports, how does it work).

My criticism with fast.ai, (I am part time educator), is that this approach is an information overload and poor sequencing. Their comparison with Teach whole game approach is flawed because a game of, say Football, is essentially simple. So you can say just start kicking around. But we don't teach chess this way. It is accepted that you have to spend some time upfront to learn the rules before you can play even simple game. Sure one need not learn castling or en-passant upfront. But you get the drift.

This book (looking at the preview chapters) is going to follow the lego blocks approach or bottoms up approach to build it. For me, this is correct way to teach supervised ML focussed on neural networks and deep learning. We have a problem of too many library plumbers in the ML field currently. People who can piece together library function calls without knowing why it is working. But this house of cards is not sustainable strategy to build AI based application over long term.

Long story short, the book will need patience but that patience will be worth it!

Re: The Little Learner: A Straight Line to Deep Learning

#85
post #42

Earlier quoted context omitted.

For sure, Lecun wrote a DL framework in Lisp decades ago.

I didn't know about his past feat. Couldn't find the lisp framework you mention but there's this https://lush.sourceforge.net/faq.html (trivia, he's also behind djvu format)

Lush was really great back then. He often criticizes Python openly.

Personally, I also feel a bit sad the ecosystem has become a Python monoculture.

Lua was much nicer. Let's hope this book sparks some interest in alternative languages for ML.

Re: The Little Learner: A Straight Line to Deep Learning

#86
post #19

No way. Does it use Scheme? EDIT: Half Life 3 confirmed: "Presents key ideas of machine learning using a small, manageable subset of the Scheme language"

I am looking at the GitHub repo for the Malt library developed in the book https://github.com/themetaschemer/malt. It looks like they use Racket (Scheme) vectors to implement tensors. I experimented with loading Keras simple models into Racket several years ago: the built in math library matrix support was fast enough for my needs so the book software may be both pedagogical and useful for smaller problems.

EDIT: I have not tried the OpenBLAS Racket bindings here (https://github.com/soegaard/sci) but perhaps the low level tensor and tensor ops book code could be optimized,

Re: The Little Learner: A Straight Line to Deep Learning

#87
post #57

Earlier quoted context omitted.

I think all serious researchers have implemented core Deep Learning algorithms from scratch. I know I have. There are two books that do exactly this: 1. Deep Learning from Scratch 2. Data Science from Scratch In these books, you implement each part of the ML/DL pipeline, from scratch, in Python. There is also a GitHub project call Minitorch that teaches you the inner workings of a framework like PyTorch. And then the…

How tied are those two books to python? If very much, are there books covering the same content with a different programming language?

they mostly use numpy (matrix maths library).

So if you use a library for matrix multiplication, inverse, transpose, ... with a nice syntax, you're good to go.

Re: The Little Learner: A Straight Line to Deep Learning

#88

Earlier quoted context omitted.

So you mean that fast.ai maybe a better resource for people new to general ML?

Well, hard to say if it is a better resource. But it will appeal to those who are in a hurry (and yes, it is okay to be in a hurry). For example, fast.ai first chapter will have you build a dogs vs cats classifier. But how do they build it? By doing `from fastai.vision import *`. They justify it by claiming to use Teaching the whole game approach. If you are a somewhat experienced engineer who already knows Python, w…

Thank you for your reply.

I'm a Lisper (used Common Lisp, Racket/Scheme, and Clojure) and a math graduate student, and am interested in learning more about ML and DL so that I could potentially use them in research in the future, to come up with constructions and counter-examples, so I may have some time in my hand. I have always been a fan of Dan Friedman, so I'm definitely thinking about getting this book.

I found out that fast.ai seems to be of this approach that you mentioned. Since you are an educator, if I'm interested in learning more about ML and DL (as well as the math behind and implementation), which book(s) would you recommend me to study (I have some experience with Andrew Ng's coursera course many years ago)? I did some research here and people seem to recommend different books...

Thank you!

Re: The Little Learner: A Straight Line to Deep Learning

#89
post #85

Earlier quoted context omitted.

I didn't know about his past feat. Couldn't find the lisp framework you mention but there's this https://lush.sourceforge.net/faq.html (trivia, he's also behind djvu format)

Lush was really great back then. He often criticizes Python openly. Personally, I also feel a bit sad the ecosystem has become a Python monoculture. Lua was much nicer. Let's hope this book sparks some interest in alternative languages for ML.

I'm really surprised that Haskell didn't take off here. Strong types, plus lazy evaluation that seems perfect for orchestrating asynchronous GPU operations.

Re: The Little Learner: A Straight Line to Deep Learning

#90
post #82

Earlier quoted context omitted.

I didn't know about his past feat. Couldn't find the lisp framework you mention but there's this https://lush.sourceforge.net/faq.html (trivia, he's also behind djvu format)

That's the one!

oh right, he mentions machine learning somewhere, naive me thought it was just a cute lisp repl that can also compile c for perf.
Post reply on HN