CoreML Benchmark for the iPhone 14 Pro

Not trying to nitpick here, but does anyone find it strange that a webpage/website like this doesn’t have a graph of the data, but just a poorly designed table?

Is this a trend I’ve missed … some sort of post-D3 nadir of a datavis hype curve or something where graphs are a cringey thing SEOd click bait articles or news pages do?

A human brain requires constant power to the brain and auxiliary systems. It has an inefficient enegery input system (food) and requires ~2-3kwh (depending on weight, sex etc.) of energy per day and cannot operate 24/7/365. When not being used for work, it still requires power.

A camera with a nvidia jetson might consume 0.5 kwh per day and run nonstop.

Ultimatly it is Apples to Oranges. A human brain can do a lot more than simply classify an object. Security guards watching cameras are evaluating the situation, not annotating images.

> It has an inefficient enegery input system (food)

A human body can extract up to ~95% of the energy in that food (depends on the food), which is pretty damn efficient. You may have seen the number 20% thrown around, but that refers to how much of that can be turned into useful mechanical energy.

> and requires ~2-3kwh (depending on weight, sex etc.) of energy per day and cannot operate 24/7/365.

More like a fifth of that energy (which is what the brain uses). If you're going to look at the entire body, you're going to have to match those features in your hardware. I don't think current-gen hardware that could conceivable repair itself and take care of its own needs while being as energy efficient as a human body.

> A camera with a nvidia jetson might consume 0.5 kwh per day and run nonstop.

The nano?

If I had trained a model on the full Open Images Dataset (so we can get a number of categories that at least approaches what a human could do) are you sure that's going to cut it?

YOLOv3 doesn't even reach 2 fps on the nano (YOLOv3-tiny gets more, but using a crippled version won't win us any prizes), and that one only has 80 categories. The Open Images Dataset has five times that - which is still absolutely nothing compared to what a human can do (and the dataset is also a bit odd: the only specific street sign it knows is "stop sign" and there's weird one-offs like "facial tissue holder" but it can't tell a ferris wheel from a car wheel or steering wheel).

Even if you somehow managed to fit something with such a number of categories and acceptable accuracy on a nano, it would probably blow its energy budget, which is about 2 seconds of operation if it wants to match a human.

> Ultimatly it is Apples to Oranges. A human brain can do a lot more than simply classify an object.

Sure, but it's also not going to perform a lot of tasks at the same time. If you ask a human to keep classifying anything you're pointing at, they'll be mostly busy watching you and what you're pointing at, trying to conjure up the appropriate word to name the thing. If not you're not pointing fast enough.

Though I suppose we also have some sort of passive classification mode that we're using most of the time while we do other things. This mode just deals with concepts - it doesn't bother to inform us the thing flying at us is called "ball".

I wish they added Intel MacBooks with eGPU and a top notch Mac Pro with dual GPUs.