Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple M1 support for TensorFlow 2.5 pluggable device API

developer.apple.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#71

Post by okamiueru »

ojosilva wrote: post #52

I'm still trying to find a way to monitor the Neural Engine on my Macbook air M1, but the APIs are non-existent, there's barely anything in the docs and no answer from Apple. My models train fast, 3x faster than most i7 computers with GPU, which is excellent for a fanless ultraportable computer but I wish Apple would treat the NE as a 1st class citizen on these machines, with Mac SDK APIs and usage visualization in t…

> 3x faster than most i7 computers with GPU

Can you back that statement up with anything, or at least clarify it? You seem to suggesting a non-mac i7 with a separate GPU. Also, just an FYI, "i7" says pretty much nothing. The i7s have existed since 2009.

I don't know. The statement is just so vague and ridiculous. The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your main concern. For highly parallelizable work tasks, the top end GPU alone has 10x the computation power than the M1, and a top end CPU has around 4x the computation power than the M1. Not to mention a rather limiting 16GB of memory. That the M1 is computationally powerful is a myth started out by exceedingly misleading marketing and reinforced with hard-to-compare benchmarks.

To possibly save someone the trouble, the responses to a comment such as this, from experience will be:

1. "power consumption is much much better on M1 than anything else." True, but, then again, your use case must then prioritize power consumption, and not computation power. So which is it? The use case for a compute cluster on a train is rather contrived.

2. "When apple scales up the M1 to more cores, they will magically be able to retain all the benefits possible with a low core count, and scale it up without any problems or compromises, just you wait.". Ok, I'll wait.

3. "It's not fair to compare a laptop with a desktop". Of course it is. The constraints for comparison are already stated: computation power being the main priority. If someone buys hardware to do heavy computations, you can pick and chose depending on your needs. If you need it to be a laptop, or you need it to draw little power, then I'm sure you can factor this in accordingly.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#72

Post by rvz »

qmmmur wrote: post #58
rvz wrote: post #19

Earlier quoted context omitted.

> I run everything python under Rosetta. So basically the software that you require to run and do your work is still not available for Apple Silicon? In this case, Apple Silicon support for Python libraries. The whole point of Apple Silicon is to supersede its Intel Mac counterparts and to run software natively with a noticeable performance increase, especially with software with high performance requirements. In the…

There's no need to be so angry about a hypothetical situation in which you by a laptop that you never wanted. I just prefer simplicity right now. I'm a PhD student and I don't need the code to be super fastz just need it to work right now. As it stands j just got it all to run under native so I guess my slight modicum of patience you don't have lags dividends :)

> There's no need to be so angry about a hypothetical situation in which you by a laptop that you never wanted.

Nothing hypothetical about being an early adopter and then complaining and wasting months since the November 2020 release day chaos that several software is still unavailable or unstable, the very basic software doesn't even run on the system and the excessive disk writes on the M1 quickly wears the SSD on the machine. All completely real and happened to many people since launch day.

So, what's the point of buying a laptop that doesn't even run your software in the first place? Might as well stay on your existing laptop. You would rather wait N amount of months for the software to mature and use the laptop reliably for your work than to skip all of that, use your existing laptop and get a better one? (M2 Mac).

Why would I want to wait for months for the developers to port their software and its library ecosystem to Apple Silicon or waste time with broken workarounds when I can use my existing laptop that already 'just works' with everything.

Mind you, I actually bought the M1 MBA recently to try it out and returned it at full price due to the software ecosystem not being ready. Not only I saved my money, I >5x'd that money it in a recent investment anyway and now I'm glad I did that.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#73

Post by yza »

truth_ wrote: post #70
codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

PyTorch currently provides RoCM support. Has anyone tried that?

I tried it on a Radeon VII. It is little hard to get running as it does not work with latest kernel, but otherwise it kind of works with some quirks. One of the quirks is the first epoch of your training is very slow to start compared to Nvidia cards. Also, the speed of training is slow.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#74

Post by jonathanstrange »

okamiueru wrote: post #71
ojosilva wrote: post #52

I'm still trying to find a way to monitor the Neural Engine on my Macbook air M1, but the APIs are non-existent, there's barely anything in the docs and no answer from Apple. My models train fast, 3x faster than most i7 computers with GPU, which is excellent for a fanless ultraportable computer but I wish Apple would treat the NE as a 1st class citizen on these machines, with Mac SDK APIs and usage visualization in t…

> 3x faster than most i7 computers with GPU Can you back that statement up with anything, or at least clarify it? You seem to suggesting a non-mac i7 with a separate GPU. Also, just an FYI, "i7" says pretty much nothing. The i7s have existed since 2009. I don't know. The statement is just so vague and ridiculous. The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your…

I don't know about the M1, to be honest, but I used to be an Apple customer and ardent fanboy (so embarrassing!) during the golden PPC age and remember very well Apple's inflated claims about performance, which all turned out to be false the minute they switched to Intel. So I agree with your comment, it's advisable to always take miraculous performance claims with a grain of salt.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#75

Post by andy_ppp »

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

Interesting that AMD is supported, could this mean Apple Silicon 16inch MBP with an AMD GPU?

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#76

Post by rjmunro »

jonathanstrange wrote: post #74
okamiueru wrote: post #71

Earlier quoted context omitted.

> 3x faster than most i7 computers with GPU Can you back that statement up with anything, or at least clarify it? You seem to suggesting a non-mac i7 with a separate GPU. Also, just an FYI, "i7" says pretty much nothing. The i7s have existed since 2009. I don't know. The statement is just so vague and ridiculous. The M1 is probably the worst hardware you could have picked in 2020-2021 if computational power was your…

I don't know about the M1, to be honest, but I used to be an Apple customer and ardent fanboy (so embarrassing!) during the golden PPC age and remember very well Apple's inflated claims about performance, which all turned out to be false the minute they switched to Intel. So I agree with your comment, it's advisable to always take miraculous performance claims with a grain of salt.

Were the performance claims false, or did Intel just catch up, while PowerPC stagnated?

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#77

Post by mjlee »

ksec wrote: post #41
codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

So Apple would need 16x its GPU Core, or 128 GPU Core to reach Nvidia 3090 Desktop Performance. Or roughly 480mm2 Die Size, 192W TDP excluding memory controller and interconnect. Doesn't look too bad for Nvidia, especially when you consider 3090 is still on Samsung 8nm, which is equivalent to TSMC 10nm, compared to 5nm on Apple M1.

https://www.bloomberg.com/news/articles/2021-05-18/apple-rea...

That is exactly what the rumours suggest Apple is working on.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#78

Post by floatboth »

jra101 wrote: post #40
codelord wrote: post #23

Earlier quoted context omitted.

Seems like AMD has been using Vega 20 to refer to two different things. I was talking about the mobile GPU on MacBook Pros which is based on a 14nm chip. The full name is Radeon Pro Vega 20: https://www.amd.com/en/graphics/radeon-pro-vega-20-pro-vega-... https://www.techpowerup.com/gpu-specs/radeon-pro-vega-20.c32... Vega 20 seems to also refer to a discrete GPU. This has been later rebranded to Radeon VII (maybe bec…

Huh, I had no idea they used Vega 20 both as a codename and a product name. Confusing.

Same with 10 LOL. VEGA10 codename is for the original Vega Frontier Edition/RX Vega 56/RX Vega 64. But now there's also "Vega 10" used as a description for the 10 Vega compute units on Ryzen APUs.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#79

Post by floatboth »

mjburgess wrote: post #66
kimixa wrote: post #64

Earlier quoted context omitted.

M1 isn't a chiplet is it? It's a monolithic SoC, and while it is packaged with the RAM that's just closer integration of chips that would normally be on-motherboard or a DIMM?

The M1 itself is an SoC. It's composed of parts, however, which can stack. Eg., just look at the M1 variations present atm: variable gpu cores. The "chiplet" speculation is that given the M1's variable high-perf, low-perf and cpu cores, this can scale up. Leaks at least are all pointing in a 32-core, Xeon-competitor direction. It is theoretically possible they could do the same with GPU count, and try to compete perh…

Basically all silicon works like this. "Variable cores" actually means some cores are disabled. This is usually done to increase overall yields: chips with damage inside one of the cores can still be binned as the lower core count SKUs.

The "you can scale up" thing is actually just how e.g. Intel makes bigger monolithic chips (Xeon/HEDT) with the same or very similar cores as the desktop ones. Meanwhile AMD, actually using chiplets, can cheaply do something more like "scale out" in the sense that they put more of the exact same die on a package.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#80

Post by okamiueru »

sydthrowaway wrote: post #44

I find the benchmarks confusing. If we normalize, is Apple close to beating Nvidia?

Given what the difference in computation power is between a computer with top-end nvidia card and top end CPU, the numbers suggest that if you have a heavy work task that you on such a system, start up in the evening, say 8 PM on a Monday, and have ready the next morning at 8 AM. The same task would not be ready until the following Monday at 8 PM on the M1.
Post reply on HN