51–60 of 94 posts
I found setting up Apple’s M1 fork of TensorFlow to be fairly easy, BTW.
I am writing a new book on using Swift for AI applications, motivated by the “niceness” of the Swift language and Apple’s CoreML libraries.
No, but it's pretty good at retraining the final layer of low memory networks like MobileNet - weirdly a workload that the V100 is very poorly suited for...
Earlier quoted context omitted.
> The special thing about the V100 is that it's driver EULA allows data center usage. Wait what? Is it the only thing? That sounds hard to believe: if true, using the open driver (Nouveau) instead of Nvidia's proprietary one would be a massive money saver for datacenters operators (and even if Nouveau doesn't support the features you'd want already, supporting their development would be much cheaper for a company lik…
NVIDIA has EULA to prevent data centre use of their hardware. Also, NVIDIA does not allow bulk buying of RTX series.
Took me quite a while to get my hands on a 3080.
I categorize this as an exploration of how to benchmark desktop/workstation NPUs [1] similar to the exploration Daniel Lemire started with SIMD. Mobile SoC NPUs are used to deploy inference models on smartphones and IoT devices while discreet NPUs like Nvidia A100/V100 target cloud clusters. We don’t have apples-to-apples benchmarks like SPECint/SPECfp for the SoC accelerators in the M1 (GPU, NPU, etc.) so these earl…
> We don’t have apples-to-apples benchmarks We do: https://mlperf.org/ Just run their benchmarks. Submitting your results there is a bit more complicated, because all results there are "verified" by independent entities. If you feel like your AI use case is not well represented by any of the MLPerf benchmarks, open a discussion thread about it, propose a new benchmark, etc. The set of benchmarks there increases all t…
"trainable_params 12,810" laughs (for comparison, GPT3: 175,000,000,000 parameters) Can Apple's M1 help you train tiny toy examples with no real-world relevance? You bet it can! Plus it looks like they are comparing Apples to Oranges ;) This seems to be 16 bit precision on the M1 and 32 bit on the V100. So the M1-trained model will most likely yield worse or unusable results, due to lack of precision. And lastly, the…
thanks for the thorough comment. the article is, unfortunately, just clickbait.
A huge number of models are "small". I'm currently training game units for autonomous behaviors. The M1 is massively oversized for my need.
Saying "Oh look, GPT-3" just stupidifies the conversation, and is classic dismissive nonsense.
When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…
Depends on your field. In Reinforcement Learning you often really do train just one, at least on the same data set (since the data set often is dynamically generated based on the behavior of the previous iteration of the model).
Earlier quoted context omitted.
> We don’t have apples-to-apples benchmarks We do: https://mlperf.org/ Just run their benchmarks. Submitting your results there is a bit more complicated, because all results there are "verified" by independent entities. If you feel like your AI use case is not well represented by any of the MLPerf benchmarks, open a discussion thread about it, propose a new benchmark, etc. The set of benchmarks there increases all t…
Those benchmarks are absurdly tuned to the hardware. Just look at the result Google gets with BERT on V100s vs the result NVIDIA gets with V100s. It's an interesting measurement of what experts can achieve when they modify their code to run on the hardware they understand well, but it isn't useful beyond that.
These benchmarks measure the combination of hardware+software to solve a problem.
Google and NVIDIA are using the same hardware, but their software implementation is different.
---
The reason mlperf.org exists is to have a meaningful set of relevant practical ML problems that can be used to compare and improve hardware and software for ML.
For any piece of hardware, you can create an ML benchmark that's irrelevant in practice, but perform much better on that hardware than the competition. That's what we used to have before mlperf.org was a thing.
We shouldn't go back there.
Has anyone spotted any work allowing a mainstream tensor library (e.g. jax, tf, pytorch) to run on the neural engine?