Viewing profile — awnihannun

Right, my comment was mostly about decoding speed. For prefill you can get a speed up but there you are less latency bound. In our benchmarks with MLX / mlx-lm it's as much as 3.5x…

For a bit more context, those posts are using pipeline parallelism. For N machines put the first L/N layers on machine 1, next L/N layers on machine 2, etc. With pipeline paralleli…

Everything you want to know about running LLMs with MLX on Apple silicon: - Introduction - MLX LM Introduction - Text generation - Quantization - Fine-tuning - LLMs in MLXSwift

I agree with your point. It can be hard for a US native English speaker to recognize a Scottish accent. But, other Scottish people certainly don't have trouble with understanding a…

There are a few categories that I think TensorFlow is notably strong in. Namely: 1. Deployment. 2. Coverage of the library / built-in functionality. 3. Device management. For more …