Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Is Working on a Dedicated Chip to Power AI on Devices

bloomberg.com

Post reply on HN
Print view View original

41–50 of 123 posts

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#41

Post by jacquesm »

highd wrote: post #14

This is probably going to be a hyper parallel fixed point / integer engine like TPU gen1. Doing fast matrix multiply over really small fields is very subpar on CPUs and GPUs. That was the initial reasoning behind TPU gen1 - improving runtime performance. One question is if it will architecturally be closer to a GPU or an FPGA. The field moves so fast that it might make sense to "future-proof" a bit with a live-reconf…

I'd bet on this being an ASIC, doing this on an FPGA with any serious size matrix would require a very expensive FPGA, whereas an ASIC would allow more gates in a smaller volume and would consume less power to boot.

From the manufacturers point of view phones not being future proof is a feature, not a bug, that way you'll upgrade to that new shiny item which will keep the profits rolling in.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#42

Post by bluGill »

cft wrote: post #19

I have been worried about this trend: if they don't open it up, things like this introduce a disparity between startups that can only have access to GPUs and big companies that make their own proprietary ASICs for their proprietary software, such that startups cannot complete.

The big companies can make their own. The next smaller companies will gather together and create a company (this is essentially how ARM started) to create one chip that works for all.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#43

Post by rtkwe »

highd wrote: post #14

This is probably going to be a hyper parallel fixed point / integer engine like TPU gen1. Doing fast matrix multiply over really small fields is very subpar on CPUs and GPUs. That was the initial reasoning behind TPU gen1 - improving runtime performance. One question is if it will architecturally be closer to a GPU or an FPGA. The field moves so fast that it might make sense to "future-proof" a bit with a live-reconf…

It'll be an ASIC so more GPU than FPGA. The real reason to upgrade the chip would be to add more transistors rather than any real instruction set upgrade so the FPGA doesn't really get Apple anything other than cost and wasted space.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#45

Post by hackuser »

My assumption has been that Google, Amazon, and Microsoft run the heavy-duty AI in the cloud when possible, benefiting from huge scale and easier updates. Maybe that assumption is wrong?

If it's right, is Apple adopting a more decentralized model, with AI (or more AI) running locally? Could that compete with cloud-based AI's advantages? Obviously it would be better for offline usage, for responsiveness when transmission to the cloud is a significant part the latency, and for confidentiality.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#46

Post by yellow_postit »

JumpCrisscross wrote: post #34

Could this mean on-device ANI? My deal breaker with Amazon, Google, Microsoft and even Siri is their role in normalising the hoovering up of sensitive data.

More work needs to be done on training models with less data, differential privacy, and unsupervised learning, but so long as supervised learning continues to be the main path forward for the current set of "AI" centralizing the data into ginormous data sets will continue to be the norm.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#47

Post by georgespencer »

whatisani wrote: post #44
JumpCrisscross wrote: post #34

Could this mean on-device ANI? My deal breaker with Amazon, Google, Microsoft and even Siri is their role in normalising the hoovering up of sensitive data.

what is ANI?

Artificial Narrow Intelligence or "weak" AI. Like Siri.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#48

Post by seanp2k2 »

highd wrote: post #36
imh wrote: post #32

Earlier quoted context omitted.

How do FPGAs compare on space and power? Would differences be enough to matter for mobile?

I don't think space is an issue, but an ASIC designed exactly for a workload will always beat an FPGA on power. But if you don't know the workload exactly or don't have the money to fab an ASIC then an FPGA will be superior if the workload is a bad fit for CPUs or GPUs. So if you can save (2-10)x power on some unknown ML workload in the future that might be preferable to (10-20)x on some fixed workload with a fixed-p…

I doubt they'll do an FPGA. Devices are too concerned with battery life to be running that, plus their margins would suffer or it'd be even more expensive.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#49

Post by halflings »

yellow_postit wrote: post #46
JumpCrisscross wrote: post #34

Could this mean on-device ANI? My deal breaker with Amazon, Google, Microsoft and even Siri is their role in normalising the hoovering up of sensitive data.

More work needs to be done on training models with less data, differential privacy, and unsupervised learning, but so long as supervised learning continues to be the main path forward for the current set of "AI" centralizing the data into ginormous data sets will continue to be the norm.

I don't see how unsupervised learning makes this any better? That data you're training on in an unsupervised manner is still collected somewhere, and could contain as much private information as a labeled dataset.

Re: Apple Is Working on a Dedicated Chip to Power AI on Devices

#50

Post by highd »

blackguardx wrote: post #38
imh wrote: post #32

Earlier quoted context omitted.

How do FPGAs compare on space and power? Would differences be enough to matter for mobile?

FPGAs are pretty bad space-wise and power-wise compared to straight up ASICs. Apple could make some blocks highly configurable, but even an FPGA designer wouldn't use FPGA fabric to do multiplication if they cared about performance. FPGAs are a mix of general purpose logic blocks (the fabric) and dedicated blocks like multipliers, dividers, PLLs, memory, serializers and deserializers, etc.

That's what I'm thinking - some sort of configurable FPGA like fabric around a bulk of TPUv1 style cores, maybe for routing outputs around so you can do some nice pipe-lining like you might want with CV on video.
Post reply on HN