21–30 of 35 posts
Also interesting in this context is the PyTorch Developer Podcast [1] by the same author. Very comforting to learn about PyTorch internals while doing the dishes. [1] https://pytorch-dev-podcast.simplecast.com/
It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.
TORCH_CHECK(self.dim() == 1)
For learning internals of ML frameworks I recommend reading the source code of MLX: https://github.com/ml-explore/mlx . It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.
I'm asking because it seems to have nice autodiff functionality. It even supports differentiating array mutation (https://ml-explore.github.io/mlx/build/html/usage/indexing.h...), which is something JAX and Zygote.jl can't do. Instead, both have ugly tricks like `array.at[index].set` and the `Buffer` struct.
So it would be cool to have this functionality on a "regular" CPU.
For learning internals of ML frameworks I recommend reading the source code of MLX: https://github.com/ml-explore/mlx . It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.
Why is MLX Apple silicon only? Is there something fundamental that prevents it from working on x86? Are some core features only possible on Apple silicon? Or do the devs specifically refuse to port to x86? (Which is understandable, I guess) I'm asking because it seems to have nice autodiff functionality. It even supports differentiating array mutation ( https://ml-explore.github.io/mlx/build/html/usage/indexing.h...…
Also interesting in this context is the PyTorch Developer Podcast [1] by the same author. Very comforting to learn about PyTorch internals while doing the dishes. [1] https://pytorch-dev-podcast.simplecast.com/
i think the problem w the podcast format (ironic for me to say) is that it assumes a lot higher familiarity with the apis than is afforded by any visual medium including blogs
For learning internals of ML frameworks I recommend reading the source code of MLX: https://github.com/ml-explore/mlx . It is a modern and clean codebase without legacies, and I could understand most things without seeking external articles.
Why is MLX Apple silicon only? Is there something fundamental that prevents it from working on x86? Are some core features only possible on Apple silicon? Or do the devs specifically refuse to port to x86? (Which is understandable, I guess) I'm asking because it seems to have nice autodiff functionality. It even supports differentiating array mutation ( https://ml-explore.github.io/mlx/build/html/usage/indexing.h...…