Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: ML From Scratch – free online textbook

dafriedman97.github.io

Post reply on HN
Print view View original

11–20 of 56 posts

Re: Show HN: ML From Scratch – free online textbook

#11

Post by Cieric »

dafrdman wrote: post #5
Cieric wrote: post #3

First thing I thought of when I read the title was sentdex's nnfs.io (neural networks from scratch.) I don't have time to look at this in full right now, are there any major differences between the these? From my quick glance over this looks like it's heavy on the math notation and when it gets to the code it's already getting to complex (for someone who doesn't know python) to read.

The approach to this book is very similar to nnfs.io (a similar focus on deriving models from the bare bones). The biggest difference is that his focuses on deep learning while mine covers a) a wider range of models and b) more introductory methods (linear regression, logistic regression, naive bayes, etc.) It is definitely heavy on the math notation. Since the purpose of the book is to provide mathematical derivatio…

Thanks for the summary, the object oriented stuff is fine in my mind. The trouble with the code comes from things like the numpy functions. I have experience in matrices for what's used in computer graphics, but some of numpys functions are conflicting with what I understand those constructs to do. I don't know maybe it's the lack of any type information that makes this hard for me, I can't just look at the something I know and figure out what is going on from there, I have to go through each function one by one to make sure I know what it actually returns.

Either way, good luck with the book. I've considered doing knowledge dumps myself for game/game engine oriented information but I don't even know where I would start with that.

Re: Show HN: ML From Scratch – free online textbook

#12

Post by fartcannon »

Excellent! Very kind of you to do this. Are you accepting comments? If not, ignore the following. :)

As someone who learned how to program from trial and error via tutorials on the internet (some of the people who are going to read your book are people like me), I just have one comment:

Try not leaning on libraries in your tutorials.

I know it sounds insane to suggest you not use numpy in any kind of ML tutorial, but libraries like numpy used to make my new programmer's eyes glaze over. The fact that those imports are black boxes and can do literally anything used to make my noob mind overload. And I'd develop a blind spot for them. Before I could read code that exploited numpy's power, I had to work with numpy for a while to gain an intuition for it. New people don't have that. This isn't just towards numpy, but for all libraries in tutorials. If it's at all possible (I know it's not always possible in a reasonable way), make a super simple example of what you're going to use the library to do before you replace it with the library.

As mentioned elsewhere in this thread, Sentdex is actually really good at teaching new developers. I think this is because he often starts without numpy. For example in NNFS videos, he starts with just lists. He gets to numpy eventually, but knowing what he replaced with numpy helps make things more clear.

Just a thought.

Re: Show HN: ML From Scratch – free online textbook

#13

Post by stevesimmons »

dafrdman wrote: post #2

I'm linking to a free online book I just finished called Machine Learning from Scratch. The book aims to cover the complete, technical, "under the hood" details that other ML textbooks don't. To do that, it shows all the mathematical steps to derive common algorithms and demonstrates how to fit each one from scratch in Python (using just numpy). You might like this book if you are interested in ML and like to really…

Nice work!

Can we flag edits on github somewhere?

I found a typo at line 60 of https://dafriedman97.github.io/mlbook/content/conventions_no..., where your second "is written as $$" is missing the empty line before the "$$".

Re: Show HN: ML From Scratch – free online textbook

#14

Post by abhayhegde »

fartcannon wrote: post #12

Excellent! Very kind of you to do this. Are you accepting comments? If not, ignore the following. :) As someone who learned how to program from trial and error via tutorials on the internet (some of the people who are going to read your book are people like me), I just have one comment: Try not leaning on libraries in your tutorials. I know it sounds insane to suggest you not use numpy in any kind of ML tutorial, but…

This was exactly what my professor used to say while we were learning programming. Our thought process should be at building the program, to construct a progressive logic train of thoughts that is robust enough to avoid basic errors. Using packages defeats that purpose. Once we have a minimal working solution, we can then worry about efficiency, scalability and other benchmarking parameters to judge how far is the first try from the best one.

While developing something though, it could be appropriate to use existing and better solutions.

Re: Show HN: ML From Scratch – free online textbook

#15

Post by dafrdman »

fartcannon wrote: post #12

Excellent! Very kind of you to do this. Are you accepting comments? If not, ignore the following. :) As someone who learned how to program from trial and error via tutorials on the internet (some of the people who are going to read your book are people like me), I just have one comment: Try not leaning on libraries in your tutorials. I know it sounds insane to suggest you not use numpy in any kind of ML tutorial, but…

Thanks so much for your feedback. Definitely open to comments!

I agree 100% that any use of packages can be intimidating for newbies. I experimented at first with creating the models without using numpy and I thought that it actually made it less clear rather than more clear. It's obviously a tradeoff--you see where everything comes from (rather than np.mysterious_function()) but you take 5 lines of code to do the same thing that a single numpy command could accomplish. I felt in the end that it distracted from the real purpose of the code, which is to demonstrate how the model works.

Do you think a compromise would be to add a section to the appendix introducing numpy? Introducing arrays, random instantiation, stuff like that? Otherwise I might consider adding a no-numpy version in the future.

Thanks so much for your feedback!

Re: Show HN: ML From Scratch – free online textbook

#16

Post by [deleted] »

stevesimmons wrote: post #13
dafrdman wrote: post #2

I'm linking to a free online book I just finished called Machine Learning from Scratch. The book aims to cover the complete, technical, "under the hood" details that other ML textbooks don't. To do that, it shows all the mathematical steps to derive common algorithms and demonstrates how to fit each one from scratch in Python (using just numpy). You might like this book if you are interested in ML and like to really…

Nice work! Can we flag edits on github somewhere? I found a typo at line 60 of https://dafriedman97.github.io/mlbook/content/conventions_no... , where your second "is written as $$" is missing the empty line before the "$$".

[deleted]

Re: Show HN: ML From Scratch – free online textbook

#17

Post by dafrdman »

stevesimmons wrote: post #13
dafrdman wrote: post #2

I'm linking to a free online book I just finished called Machine Learning from Scratch. The book aims to cover the complete, technical, "under the hood" details that other ML textbooks don't. To do that, it shows all the mathematical steps to derive common algorithms and demonstrates how to fit each one from scratch in Python (using just numpy). You might like this book if you are interested in ML and like to really…

Nice work! Can we flag edits on github somewhere? I found a typo at line 60 of https://dafriedman97.github.io/mlbook/content/conventions_no... , where your second "is written as $$" is missing the empty line before the "$$".

Thanks for the catch! Ugh those pesky $$s. Changed it now.

Looks like you found the repository. Do you think it would be enough to raise an issue there? (at https://github.com/dafriedman97/mlbook)? I'll look into linking directly from the book to the repo.

Thanks!

Re: Show HN: ML From Scratch – free online textbook

#18

Post by stevesimmons »

fartcannon wrote: post #12

Excellent! Very kind of you to do this. Are you accepting comments? If not, ignore the following. :) As someone who learned how to program from trial and error via tutorials on the internet (some of the people who are going to read your book are people like me), I just have one comment: Try not leaning on libraries in your tutorials. I know it sounds insane to suggest you not use numpy in any kind of ML tutorial, but…

I have a different view :)

I feel this book is the ideal companion for intermediate to advanced Python developers/data scientists. People who know pandas pretty well, and so are comfortable with numpy array operations too. They have probably used scikit-learn's fit/predict API in a black-box way, but have never quite had time to look at the code underneath.

I love "ML from Scratch" because it leverages my math and Python/numpy knowledge (even if some of it is rusty and half-forgotten) to show me high quality, well-commented, mathematically rigorously explanations of how I might have implemented the types of algorithms I use from sklearn. While the maths is formal, it is totally straightforward in its presentation.

It will definitely be my bedtime reading for the next few nights.

Re: Show HN: ML From Scratch – free online textbook

#19

Post by dafrdman »

Cieric wrote: post #11
dafrdman wrote: post #5

Earlier quoted context omitted.

The approach to this book is very similar to nnfs.io (a similar focus on deriving models from the bare bones). The biggest difference is that his focuses on deep learning while mine covers a) a wider range of models and b) more introductory methods (linear regression, logistic regression, naive bayes, etc.) It is definitely heavy on the math notation. Since the purpose of the book is to provide mathematical derivatio…

Thanks for the summary, the object oriented stuff is fine in my mind. The trouble with the code comes from things like the numpy functions. I have experience in matrices for what's used in computer graphics, but some of numpys functions are conflicting with what I understand those constructs to do. I don't know maybe it's the lack of any type information that makes this hard for me, I can't just look at the something…

Thanks for the feedback. Sounds like you're not alone in your thoughts on numpy. I'll brainstorm better solutions--maybe explaining each numpy function in a side note or adding a numpy overview to the appendix. It just makes things so much easier so I am hesitant to go numpy-less.

Getting started was definitely the hardest part. I put the book together with JupyterBook (https://jupyterbook.org/intro.html), which I would highly recommend! Really neat tool and a helpful community.

Re: Show HN: ML From Scratch – free online textbook

#20

Post by infinitybeyond »

fartcannon wrote: post #12

Excellent! Very kind of you to do this. Are you accepting comments? If not, ignore the following. :) As someone who learned how to program from trial and error via tutorials on the internet (some of the people who are going to read your book are people like me), I just have one comment: Try not leaning on libraries in your tutorials. I know it sounds insane to suggest you not use numpy in any kind of ML tutorial, but…

I wholeheartedly agree with this. As a software developer without a strong math education, any time I open a "from scratch" ml learning resource, I immediately close it because I don't have an understanding of the math happening in the black box of numpy.

Thank you for the Sentdex recommendation.

Post reply on HN