CoreML Benchmark for the iPhone 14 Pro • Hacker News Board

Re: CoreML Benchmark for the iPhone 14 Pro

#11

Post by KMnO4 » Sat, Sep 17, 2022, 1:44 PM UTC

matthieurouif wrote: post #9

maegul wrote: post #4

Not trying to nitpick here, but does anyone find it strange that a webpage/website like this doesn’t have a graph of the data, but just a poorly designed table? Is this a trend I’ve missed … some sort of post-D3 nadir of a datavis hype curve or something where graphs are a cringey thing SEOd click bait articles or news pages do?

We are really more into shopping fast than shipping perfect but we improve over time. The fact that we introduce Mac and iPad in the chart make the graph a bit misleading as it’s not a chronological order

I honestly think you should focus on shipping perfect rather than shipping “fast”.

The world has enough quantity; let’s focus on quality.

Re: CoreML Benchmark for the iPhone 14 Pro

#12

Post by kory » Sat, Sep 17, 2022, 1:45 PM UTC

To truly understand perf, ideally one should compare many types and sizes of models. I suspect some model types perform substantially better on the newer ANE / OS compared with others.

Re: CoreML Benchmark for the iPhone 14 Pro

#13

Post by miohtama » Sat, Sep 17, 2022, 2:23 PM UTC

Any hope to run Stable Diffusion locally on iPhone?

Re: CoreML Benchmark for the iPhone 14 Pro

#14

Post by mark_l_watson » Sat, Sep 17, 2022, 2:38 PM UTC

CoreML is really very good, both on its own, and tools for importing models from other platforms and compressing them. I wrote a book earlier this year on Swift and added a few CoreML examples (https://leanpub.com/SwiftAI). Google provides something similar.

Federated privacy preserving learning, local models, etc. all help keep your private data on your devices. Good stuff.

Re: CoreML Benchmark for the iPhone 14 Pro

#15

Post by chmod775 » Sat, Sep 17, 2022, 6:32 PM UTC

nl wrote: post #7

chmod775 wrote: post #3

A bit of a tangent, but where are we at when it comes to energy efficiency in AI? Suppose I had one or two cameras attached to a computer and ran a software that would detect which object I'm pointing at and name it, how much power would that use? The human brain would probably need around 0.5s - 1s to come up with an answer, consuming around 5 milliwatt hours of energy in that time. How much power would the computer…

Something like https://canaan.io/product/kendryte-k510 will outperform a human on object recognition. Standby usage is 2mA, and 2W usage when being used (< 0.1s to run a single recognition).

There's no NN that can outperform a human on accuracy or on number of categories it knows, the best you could hope for is being significantly worse, but faster. Even our best nets know only a tiny fraction of what a human can classify and have noticeably worse accuracy.

You'll want to be running a huge state-of-the-art network trained on large datasets on it to approach human capabilities and I don't think 2.5TFLOPS will cut it.

I had a look around and this thing is probably more in the right ballpark: https://www.nvidia.com/en-us/autonomous-machines/embedded-sy...

It uses up to 60W for 270TFLOPS at full power, but its processing power should be in the right ballpark to at least do decently with something trained on the best datasets there are.

There's a chance much smaller hardware would do if only our software was advanced enough, but it's probably not. I'm not sure where we are really at, hence my original question. You'd need to somehow work out Watts/HumanPerformance.

Re: CoreML Benchmark for the iPhone 14 Pro

#16

Post by nl » Sun, Sep 18, 2022, 5:50 AM UTC

chmod775 wrote: post #15

nl wrote: post #7

Earlier quoted context omitted.

Something like https://canaan.io/product/kendryte-k510 will outperform a human on object recognition. Standby usage is 2mA, and 2W usage when being used (< 0.1s to run a single recognition).

There's no NN that can outperform a human on accuracy or on number of categories it knows, the best you could hope for is being significantly worse, but faster. Even our best nets know only a tiny fraction of what a human can classify and have noticeably worse accuracy. You'll want to be running a huge state-of-the-art network trained on large datasets on it to approach human capabilities and I don't think 2.5TFLOPS…

If you want unlimited categories (aka Zero shot classification), then CLIP does a pretty good job. I'd be a bit surprised if it can't run on a Jetson, although I guess RAM might be an issue.