Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MLX: An array framework for Apple Silicon

github.com

Post reply on HN
Print view View original

11–20 of 47 posts

Re: MLX: An array framework for Apple Silicon

#11

Post by jphoward »

kiratp wrote: post #7

I wish there was more information as to how this differs from or improves on Jax. Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see. What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the n…

I was excited about JAX, but I think the developers missed a trick when they decided it should be entirely immutable. It sounds silly, but I think if I have an array `x` and want to set index 0 to 10, it's a big mistake if I can't do:

  x[0] = 10
And instead I have to do:

  y = x.at[0].set(10)
Of course this has advantages, and I know it sounds lame, but as someone whose brain works in numpy, this is really offputting.

Re: MLX: An array framework for Apple Silicon

#12

Post by nikhil896 »

amilios wrote: post #6

It seems like it's matching PyTorch's API very closely, which is great. Part of me wishes they took it a step further and just made it completely API-compatible, such that code written for PyTorch could run out-of-the-box with MLX, that would be killer.

"import mlx as torch"

Re: MLX: An array framework for Apple Silicon

#13

Post by lifthrasiir »

m00x wrote: post #10
babl-yc wrote: post #9

I've found the process of porting custom ML models to iOS extremely difficult. AFAIK the only way to leverage Apple Neural Engine (and get the best performance) is to use CoreML. The only documented way to use CoreML is via coremltools, which takes a trace of a PyTorch model and attempts to translate it into a protobuf graph understood by CoreML. This process often fails and requires model changes, or worse "succeeds…

I'm not sure why there aren't more companies supporting ONNX. It's so nice to use if it's supported by the platform/model.

Not every model can be easily converted to ONNX though, especially with PyTorch.

Re: MLX: An array framework for Apple Silicon

#14

Post by sgu999 »

babl-yc wrote: post #9

I've found the process of porting custom ML models to iOS extremely difficult. AFAIK the only way to leverage Apple Neural Engine (and get the best performance) is to use CoreML. The only documented way to use CoreML is via coremltools, which takes a trace of a PyTorch model and attempts to translate it into a protobuf graph understood by CoreML. This process often fails and requires model changes, or worse "succeeds…

I don't understand why Apple isn't trying to integrate better with the standard tools for that field. I guess it makes sense to lock in app devs, but ML eng.?

That said I've had good success with onnxruntime recently [0].

[0] https://onnxruntime.ai/docs/execution-providers/CoreML-Execu...

Re: MLX: An array framework for Apple Silicon

#15

Post by sgu999 »

amilios wrote: post #6

It seems like it's matching PyTorch's API very closely, which is great. Part of me wishes they took it a step further and just made it completely API-compatible, such that code written for PyTorch could run out-of-the-box with MLX, that would be killer.

Note that there is a Metal backend for PyTorch [0]. Sadly it doesn't work well with codebases that didn't account for it from the start...

[0] https://developer.apple.com/metal/pytorch/

Re: MLX: An array framework for Apple Silicon

#17

Post by domschl »

sgu999 wrote: post #14
babl-yc wrote: post #9

I've found the process of porting custom ML models to iOS extremely difficult. AFAIK the only way to leverage Apple Neural Engine (and get the best performance) is to use CoreML. The only documented way to use CoreML is via coremltools, which takes a trace of a PyTorch model and attempts to translate it into a protobuf graph understood by CoreML. This process often fails and requires model changes, or worse "succeeds…

I don't understand why Apple isn't trying to integrate better with the standard tools for that field. I guess it makes sense to lock in app devs, but ML eng.? That said I've had good success with onnxruntime recently [0]. [0] https://onnxruntime.ai/docs/execution-providers/CoreML-Execu...

The project probably at least partially serves as documentation for other platforms to integrate Silicon acceleration. It basically demonstrates how to use macOS Accelerate and Metal MPS (metal performance shaders) using C++ for Machine Learning and training optimization.

Thus other platforms can simply take this backend-code and integrate it. (Pytorch basically did that already with Apple's help).

Re: MLX: An array framework for Apple Silicon

#18

Post by leewlving »

jphoward wrote: post #11
kiratp wrote: post #7

I wish there was more information as to how this differs from or improves on Jax. Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see. What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the n…

I was excited about JAX, but I think the developers missed a trick when they decided it should be entirely immutable. It sounds silly, but I think if I have an array `x` and want to set index 0 to 10, it's a big mistake if I can't do: x[0] = 10 And instead I have to do: y = x.at[0].set(10) Of course this has advantages, and I know it sounds lame, but as someone whose brain works in numpy, this is really offputting.

I think it is a feature from functional programming. In some level, I more agree with jax style.

Re: MLX: An array framework for Apple Silicon

#19

Post by lifthrasiir »

leewlving wrote: post #18
jphoward wrote: post #11

Earlier quoted context omitted.

I was excited about JAX, but I think the developers missed a trick when they decided it should be entirely immutable. It sounds silly, but I think if I have an array `x` and want to set index 0 to 10, it's a big mistake if I can't do: x[0] = 10 And instead I have to do: y = x.at[0].set(10) Of course this has advantages, and I know it sounds lame, but as someone whose brain works in numpy, this is really offputting.

I think it is a feature from functional programming. In some level, I more agree with jax style.

But you can convert automatically a mutable code into a functional code if that makes things easier. That's what Haskell's `do` notation does, and PyTorch even has `torch.func.functionalize` for that. Immutable should be default, but not compulsory.

Re: MLX: An array framework for Apple Silicon

#20

Post by wokwokwok »

salimmadjd wrote: post #16

Awni Hannun of Apple AI research team has a deep dive thread on X [0] with some additional info and answers some of the questions [0] https://x.com/awnihannun/status/1732184443451019431?s=46&t=O...

Deep dive? Answering questions?

Literally “thumbs up” to “does it use GPU?”? Uh…

The links on that thread are the same links from the top of the GitHub repo.

I mean, here’s a nitter link https://nitter.net/awnihannun/status/1732184443451019431#m for anyone else who’s interested, but the info on x seems to be a nothing that isn’t already on the GitHub page.

The GitHub repo seems to be more useful and informative.

Post reply on HN