MLX: An array framework for Apple Silicon
1–10 of 47 posts
Re: MLX: An array framework for Apple Silicon
#2This is really cool. I wonder how long it will be till we have GPT-4 quality models that run locally (if we ever will). Would open up a lot of possibilities.
Re: MLX: An array framework for Apple Silicon
#3Going in I was really worried there would be a bunch of wonkiness with unique apis different from torch or tf. But it seems pretty close to torch, which is a pleasant surprise. No need to reinvent the wheel.
Re: MLX: An array framework for Apple Silicon
#4>MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.
Does this mean its from Apple directly (as part of the machine-learning engineering team)?
Re: MLX: An array framework for Apple Silicon
#5>MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research. Does this mean its from Apple directly (as part of the machine-learning engineering team)?
Yes, I believe so.
Re: MLX: An array framework for Apple Silicon
#6It seems like it's matching PyTorch's API very closely, which is great. Part of me wishes they took it a step further and just made it completely API-compatible, such that code written for PyTorch could run out-of-the-box with MLX, that would be killer.
Re: MLX: An array framework for Apple Silicon
#7I wish there was more information as to how this differs from or improves on Jax.
Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see.
What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the needle at all against Nvidia?
I guess the fact that Google is pushing OpenXLA is making the other giants not want to truly lean in?
I don’t want 10 competing “choices”. I want one clear, open, competitor to Cuda that works on all the competing hardware.
Re: MLX: An array framework for Apple Silicon
#8Given the API is similar to existing libraries, I’m curious as to whether the performance is better with this one. And if so, what’s stopping existing libraries from being as fast. IIRC, PyTorch at least has a Metal backend.
The README mentions unified memory, but what stops other frameworks from modeling copies as no-ops? I wonder if MLX makes larger architectural decisions based on GPU CPU communication being cheap.
Re: MLX: An array framework for Apple Silicon
#9I've found the process of porting custom ML models to iOS extremely difficult.
AFAIK the only way to leverage Apple Neural Engine (and get the best performance) is to use CoreML. The only documented way to use CoreML is via coremltools, which takes a trace of a PyTorch model and attempts to translate it into a protobuf graph understood by CoreML.
This process often fails and requires model changes, or worse "succeeds" but gives you the wrong output when you run the model. Additionally, you have to play detective to figure out why some operations run on the CPU, or GPU instead of ANE.
It's exciting to see more tools like this for working with tensor-like objects, but I really wish Apple would make porting custom models in a high performance manner easier.
Re: MLX: An array framework for Apple Silicon
#10I've found the process of porting custom ML models to iOS extremely difficult. AFAIK the only way to leverage Apple Neural Engine (and get the best performance) is to use CoreML. The only documented way to use CoreML is via coremltools, which takes a trace of a PyTorch model and attempts to translate it into a protobuf graph understood by CoreML. This process often fails and requires model changes, or worse "succeeds…
I'm not sure why there aren't more companies supporting ONNX. It's so nice to use if it's supported by the platform/model.