Live data from Hacker News

Apple M1 support for TensorFlow 2.5 pluggable device API

developer.apple.com

101–110 of 124 posts

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#101

Earlier quoted context omitted.

If Apple could just scale up their GPU and trounce a 430B market cap competitor's premiere product at 1/2 the power, 60% of the die size, that actually looks pretty bad for nvidia, doesn't it? Scaling is more difficult than that, and who knows if they could so easily, but who thought Apple would render both Intel and nvidia irrelevant? Regardless, Apple's threat to vendors like that is their complete vertical integra…

That's besides the point though. You can't pick and choose which workloads you're going to run on Apple Silicon, because the ultimate goal is that it will be able to compete with with the rest of the industry in raw performance metrics, which is simply not the case right now. My M1 Mac's GPU still loses in several benchmarks against my 7-year-old 1060. If Apple wants to lure people like me into their pro segment, the…

"My M1 Mac's GPU still loses in several benchmarks against my 7-year-old 1060."

The GPU in the AS M1 is the fastest integrated graphics available in the mainstream computing market [1]. That is the competition, not a standalone, 120W GPU. Apple is purportedly now working on separating their GPU designs into a much larger heat and power profile (which contrary to some of the comments on here clearly isn't going to be for laptops, beyond an external TB4 enclosure) and it might just change things a bit.

Scaling a GPU is easier than scaling a CPU, by design. Apple's GPU has nothing to do with ARM.

And to your original point, yes, Apple does largely choose which workloads run on Apple Silicon and how. By controlling the APIs along with the silicon, Apple abstracts it to a degree that gives them enormous flexibility. The Accelerate and CoreML APIs are abstract vehicles that might use one or a thousand matrix engines, neural nets, or an array of GPUs. Apple has built a world where they have more hardware flexibility than anyone. And while close to no one is doing model training on Apple hardware right now, Apple has laid the foundation so a competitive piece of hardware could change that overnight.

[1] The SoC graphics of the chips in the PS5 and Xbox Series X have more powerful graphics, but the GPU chiplet alone on those systems uses a magnitude more power and more die than the entire M1 SoC. In another comment you mentioned that Zen 2 integrated graphics come close. They aren't within a ballfield, literally with 1/4 or worse the performance. In discussions like this unfortunately the boring "n years old / n process" trope is used to excess, yet again there are zero competitive integrated graphics on the market. None. Apple isn't a GPU company, yet here we are.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#102

Earlier quoted context omitted.

> That the M1 is computationally powerful is a myth started out by exceedingly misleading marketing and reinforced with hard-to-compare benchmarks It feels like you've constructed quite the straw-man to tear down. Praise for the M1 is in the context of the form factors it exists in and the efficiency it works at. Of course you can find more powerful hardware in larger form factors drawing 10x the power from the mains…

He's not wrong though. I own an M1 Macbook Air, and while there are some workloads that it can outperform my desktop at, it's still not even close to the level of functionality or comparability of my other machines. Hell, most days I just end up tossing my Thinkpad in my work bag, just because the keyboard and OS gets in my way less. > Then don't buy a bloody M1. The M1 has always been Apple's entry-level efficiency-…

> He's not wrong though. I own an M1 Macbook Air, and while there are some workloads that it can outperform my desktop at

My point is that this isn't even an interesting conversation to have. Your desktop can outperform a chip that runs an iPad? Cool story bro.

> they were forced to revoke their claim of having the "fastest CPU cores" after it was vehemently disproven.

Their announcement explicitly said "when it comes to low power silicon." So many of you insisted on missing this part it's practically a meme now.

Watch the accouncement video here: https://www.youtube.com/watch?v=5AwdkGKmZ0I and scrub to 8:45 and actually listen to what is said.

> laptops with dedicated graphics

Why should they be forced to compare their integrated graphics to laptops with dedicated graphics? This is yet another uninteresting comparison. Of course a machine with some dedicated mobile 3080 is going to win the day in a head to head. But it's an absurd comparison because said machine will draw vastly more power and be heavier to boot. It's a different category of machine at that point.

> I seriously worry for Apple if this is all they were able to get out of the 5nm node on ARM

I think you're the only one who's worried.

Apple very clearly laid out their design goals with the M1: supremacy at performance per watt. And the fanboys protested with "I can build a faster desktop" or "my 4.5 lb laptop with dedicated graphics can get more FPS." It's just baffling. You've missed the whole point.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#103
post #92

Earlier quoted context omitted.

If you thought a MacBook Air was going to replace a purpose-built machine learning workstation of course buying it would be a mistake, because it won’t do that. It only supports up to 16GB of RAM! But what other computer in that form factor comes close? The argument is that the higher efficiency will translate into more powerful chips in HEDT products too. I wouldn’t take that on faith but I think they have a decent…

> But what other computer in that form factor comes close? Basically all of them, as long as you aren't training your models on your CPU like it's 2011.

Which other ultrabooks, specifically? It’s definitely not all of them, if you’ve been reading the benchmarks.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#104

I'm still trying to find a way to monitor the Neural Engine on my Macbook air M1, but the APIs are non-existent, there's barely anything in the docs and no answer from Apple. My models train fast, 3x faster than most i7 computers with GPU, which is excellent for a fanless ultraportable computer but I wish Apple would treat the NE as a 1st class citizen on these machines, with Mac SDK APIs and usage visualization in t…

> 3x faster than most i7 computers with GPU Can you back that statement up with anything, or at least clarify it? You seem to suggesting a non-mac i7 with a separate GPU. Also, just an FYI, "i7" says pretty much nothing. The i7s have existed since 2009. I don't know. The statement is just so vague and ridiculous. The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your…

> The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your main concern. For highly > parallelizable work tasks, the top end GPU alone has 10x the computation power than the M1, and a top end CPU has around 4x the > computation power than the M1. Not to mention a rather limiting 16GB of memory.

This statement is so banal that I am not sure how to comment on it. I never understood the logic of people who point out that an entry-level, low power chip targeted at ultraportable laptops and kitchen computers cannot compete with high-end desktops. It's just as ridiculous as to complain that AMD EPYC is too big to fit into a laptop.

M1 is obviously a terrible choice if you are looking for a deskbound HPC workstation. It's a terrific choice if you are looking for an ultraportable laptop with excellent battery life that you would still like to prototype your ML code on before running the full workload on a mainframe.

> That the M1 is computationally powerful is a myth started out by exceedingly misleading marketing and reinforced with hard-to- > compare benchmarks.

More like a myth perpetuated by people who like to take facts out of the context. In terms of the underlaying IP, M1 is terrific technology. It can deliver the same performance as state of the art designs at a fraction of the power consumption. In terms of absolute performance, it's obviously an entry-level chip, and it performs exceedingly well compared to other offerings in this segment. And even outside its segment it is no slouch either. It runs my database scripts and builds code faster than my Intel i9 laptop, despite using 70% less power and half as many performance-oriented CPU cores.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#105
post #46

Earlier quoted context omitted.

`pip install --upgrade pip` fixed this for me. (not in tensorflow directly, but while installing something else on my M1 last week which required numpy)

OK, so we’re in mid-2021, why is installing Python THAT HARD? I think the only reason Node is so popular is because it JUST WORKS. Windows, Mac, doesn’t matter. One-click installer and you got NPM as well and access to thousands of packages.

i think the comparison with node is good, because my policy for both of them is the same - if i want to use either, i do it inside docker.

i can't be arsed to deal with all the various version dependencies and incompatabilities and system install vs local install nonsense that comes with installing it on my actual computer.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#106
post #81
post #45

Earlier quoted context omitted.

The rumors say they’re going to have a high end of 128 gpu cores by using 4 32 gpu core chiplets.

wouldn't that require pretty hefty active cooling, which doesn't fit so well with Apple devices? It would be great if they did it the eGPU route with TB4, in a slick package

This is more for the Mac Pro line. Not so much the laptops.

We’ve only see the Apple equivalent of the i3 with integrated graphics. It’s going to get interesting over the next several months as Apple unveils their middle and upper performance solutions.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#107

Earlier quoted context omitted.

It's not really fair to compare a discrete GPU to a mobile GPU, I only provided this as a comparison for someone who maybe has one of these at home. And btw, you are talking about TF32 performance not FP32. TF32 actually uses 16 bits. A100's FP32 performance is actually lower than 3090, it's 19.5 TFLOPS: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... FP16 performance is also relevant as a lot of peop…

TF32 is 19 bits! Default for pytorch and TF is TF32!

Oops. You are right.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#108
post #90

https://github.com/tensorflow/tensorflow/releases/tag/v2.5.0 (Linked from Apple's article) Wow, that list of CVEs is 110 lines.

the majority of these are "an attacker can craft a model that causes problems." Are people actually using tensorflow to run untrusted models?

Yes, eg ml developers and researchers test published or informally shared models.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#109

Earlier quoted context omitted.

He's not wrong though. I own an M1 Macbook Air, and while there are some workloads that it can outperform my desktop at, it's still not even close to the level of functionality or comparability of my other machines. Hell, most days I just end up tossing my Thinkpad in my work bag, just because the keyboard and OS gets in my way less. > Then don't buy a bloody M1. The M1 has always been Apple's entry-level efficiency-…

> He's not wrong though. I own an M1 Macbook Air, and while there are some workloads that it can outperform my desktop at My point is that this isn't even an interesting conversation to have. Your desktop can outperform a chip that runs an iPad? Cool story bro. > they were forced to revoke their claim of having the "fastest CPU cores" after it was vehemently disproven. Their announcement explicitly said "when it come…

> My point is that this isn't even an interesting conversation to have. Your desktop can outperform a chip that runs an iPad? Cool story bro.

The desktop I'm comparing it to is 7 years old and cost $600 new. You can't even buy an M1 iPad for that price today.

> Their announcement explicitly said "when it comes to low power silicon."

Which is an arbitrary goalpost that means nothing. The M1 uses 7w at full tilt, does that mean we can compare it to an AMD 5800u running at the same wattage? It's a nothingburger, and that's why Apple doesn't use that zinger anywhere else in their marketing material.

> Why should they be forced to compare their integrated graphics to laptops with dedicated graphics?

Because that's what you can buy for $1000. That's the performance standard. If Apple wanted the M1 to be compared to machines with integrated graphics, they should have released a computer at that price point.

> Apple very clearly laid out their design goals with the M1: supremacy at performance per watt.

Sure, they have it. But I frankly don't care, and I have a hard time believing that other people do too. Performance-per-watt is Apple's neat way of giving performance a denominator, because they simply can't compete with the rest of the industry wholesale. It's something they've done over and over, insisting on pointless metrics like thinness and beauty to measure a product of objective capability. The datacenter market is looking at the M1 and laughing. Unless your business was already Mac-based, it's not like enterprise customers are going to be interested in beta-testing Apple's new hardware either. Honestly, I'm more impressed with Apple's social engineering than hardware engineering here.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#110

Earlier quoted context omitted.

That's besides the point though. You can't pick and choose which workloads you're going to run on Apple Silicon, because the ultimate goal is that it will be able to compete with with the rest of the industry in raw performance metrics, which is simply not the case right now. My M1 Mac's GPU still loses in several benchmarks against my 7-year-old 1060. If Apple wants to lure people like me into their pro segment, the…

"My M1 Mac's GPU still loses in several benchmarks against my 7-year-old 1060." The GPU in the AS M1 is the fastest integrated graphics available in the mainstream computing market [1]. That is the competition, not a standalone, 120W GPU. Apple is purportedly now working on separating their GPU designs into a much larger heat and power profile (which contrary to some of the comments on here clearly isn't going to be…

> They aren't within a ballfield, literally with 1/4 or worse the performance.

Bzzzzt

We aren't comparing performance-per-watt, we're comparing raw compute for the price.

Post reply on HN