Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MLX: An array framework for Apple Silicon

github.com

Post reply on HN
Print view View original

41–47 of 47 posts

Re: MLX: An array framework for Apple Silicon

#41

Post by sgu999 »

domschl wrote: post #29
sgu999 wrote: post #15

Earlier quoted context omitted.

Note that there is a Metal backend for PyTorch [0]. Sadly it doesn't work well with codebases that didn't account for it from the start... [0] https://developer.apple.com/metal/pytorch/

This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed. There are few limitations left when compared with other backends. Instead of using 'cuda' device, one simply uses 'MPS' as device. What remains is: the optimizations Pytorch provides (especially compile() with 2.1) focus on cuda…

> This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed.

Not sure what you're referring to, the link I provided shows how to use the "mps" backend / device from the official PyTorch release.

> lots of energy goes into developing architectural work-arounds in order to limit the copying between graphics HW and CPU memory

Does this remark apply to PyTorch running on NVidia's platforms with unified memory like the Jetsons?

Re: MLX: An array framework for Apple Silicon

#42

Post by boywitharupee »

jphoward wrote: post #11
kiratp wrote: post #7

I wish there was more information as to how this differs from or improves on Jax. Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see. What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the n…

I was excited about JAX, but I think the developers missed a trick when they decided it should be entirely immutable. It sounds silly, but I think if I have an array `x` and want to set index 0 to 10, it's a big mistake if I can't do: x[0] = 10 And instead I have to do: y = x.at[0].set(10) Of course this has advantages, and I know it sounds lame, but as someone whose brain works in numpy, this is really offputting.

JAX is a wrapper on top of XLA. Instead of writing pure python, you're writing JAX abstractions.

for ex, a simple loop in JAX:

  def solve(i, v): return i+v
  x = jax.lax.fori_loop(0, 5, solve, 10)

Re: MLX: An array framework for Apple Silicon

#43

Post by kiratp »

GeekyBear wrote: post #39
kiratp wrote: post #7

I wish there was more information as to how this differs from or improves on Jax. Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see. What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the n…

> I wish there was more information as to how this differs from or improves on Jax. The quick start guide has an overview. > The Python API closely follows NumPy with a few exceptions. MLX also has a fully featured C++ API which closely follows the Python API. The main differences between MLX and NumPy are: Composable function transformations: MLX has composable function transformations for automatic differentiation,…

Given that Apple has built PyTorch for Metal, I would assume they handle unified memory properly in there too… Same for Tensorflow and Jax on Metal.

Re: MLX: An array framework for Apple Silicon

#44

Post by hnfong »

sgu999 wrote: post #41
domschl wrote: post #29

Earlier quoted context omitted.

This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed. There are few limitations left when compared with other backends. Instead of using 'cuda' device, one simply uses 'MPS' as device. What remains is: the optimizations Pytorch provides (especially compile() with 2.1) focus on cuda…

> This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed. Not sure what you're referring to, the link I provided shows how to use the "mps" backend / device from the official PyTorch release. > lots of energy goes into developing architectural work-arounds in order to limit the copyin…

Your link suggests downloading nightly previews and v1.12 torch which are both slightly out of date info.

Re: MLX: An array framework for Apple Silicon

#45

Post by woodson »

rsp1984 wrote: post #26
kiratp wrote: post #7

I wish there was more information as to how this differs from or improves on Jax. Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see. What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the n…

The fact that I can run this on mobile (iOS) devices using the C++ interface makes all the difference for me. I find that extremely refreshing among all those other Python/Server/PC -only frameworks. Running non-trivial ML workloads on the edge has been on my wishlist for years and it sounds like Apple has just the thing.

I’m somewhat concerned that it doesn’t support ANE from the get-go (they list CPU and GPU, for now). Not even for inference only.

Other frameworks have (non-python) solutions for mobile in place, e.g. tflite, libtorch, and onnxruntime.

Re: MLX: An array framework for Apple Silicon

#46

Post by woodson »

lifthrasiir wrote: post #13
m00x wrote: post #10

Earlier quoted context omitted.

I'm not sure why there aren't more companies supporting ONNX. It's so nice to use if it's supported by the platform/model.

Not every model can be easily converted to ONNX though, especially with PyTorch.

That’s true, one has to design the model for the deployment target. Especially avoiding in-place tensor operations and python control flow helps for tracing.

Re: MLX: An array framework for Apple Silicon

#47

Post by yobanate »

domschl wrote: post #17
sgu999 wrote: post #14

Earlier quoted context omitted.

I don't understand why Apple isn't trying to integrate better with the standard tools for that field. I guess it makes sense to lock in app devs, but ML eng.? That said I've had good success with onnxruntime recently [0]. [0] https://onnxruntime.ai/docs/execution-providers/CoreML-Execu...

The project probably at least partially serves as documentation for other platforms to integrate Silicon acceleration. It basically demonstrates how to use macOS Accelerate and Metal MPS (metal performance shaders) using C++ for Machine Learning and training optimization. Thus other platforms can simply take this backend-code and integrate it. (Pytorch basically did that already with Apple's help).

Nailed it. I think more than partially. What happens in this repo will spread to the other major frameworks and over time, clever ideas that spawn on other projects will be reimplemented with Apple's adjustments back into the repo. It's a brilliant and efficient way to interact with the community, that can likely be measured in more sales of their hardware over time.
Post reply on HN