Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Developer preview of TensorFlow Lite

developers.googleblog.com

Post reply on HN
Print view View original

11–20 of 80 posts

Re: Developer preview of TensorFlow Lite

#11

Post by runesoerensen »

aprao wrote: post #9

Is this the next iteration of TensorFlow for Mobile? Is on-device training something planned for the future?

Yes to your first question, from the article: ”As you may know, TensorFlow already supports mobile and embedded deployment of models through the TensorFlow Mobile API. Going forward, TensorFlow Lite should be seen as the evolution of TensorFlow Mobile, and as it matures it will become the recommended solution for deploying models on mobile and embedded devices.”

Also check out this post for more info and examples: https://research.googleblog.com/2017/11/on-device-conversati...

Re: Developer preview of TensorFlow Lite

#13

Post by lern_too_spel »

rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

What about using XLA to compile libraries for mobile deployment fusing only the operations needed by the model?

Re: Developer preview of TensorFlow Lite

#14

Post by bitmapbrother »

rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

Will this leverage the Pixel Visual Core SoC on a Pixel 2 device?

Re: Developer preview of TensorFlow Lite

#15

Post by theDoug »

cyberpunk0 wrote: post #5

Didnt they announce this at Google I/O? Where it was supposed to be available that day

Definitely announced at I/O, but all the language I'm finding from around that time is of the "want to" and "will" variety, like this Wired piece:

https://www.wired.com/2017/05/google-really-wants-put-ai-poc...

> “Google won't say much more about this new project. But it has revealed that TensorFlow Lite will be part of the primary TensorFlow open source project later this year”

Re: Developer preview of TensorFlow Lite

#18

Post by puzzle »

rajatmonga wrote: post #6
blackguardx wrote: post #3

Earlier quoted context omitted.

What tradeoffs did you make compared to the original?

A few tradeoffs we had to make: - As mentioned below - flatbuffers makes the startup time faster while trading off some flexibility - Smaller code size means trading off dependency on some libraries and broader support vs writing more things from scratch more focused on the user cases people care about

Flatbuffers also uses less memory.

Re: Developer preview of TensorFlow Lite

#19

Post by rajatmonga »

bitmapbrother wrote: post #14
rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

Will this leverage the Pixel Visual Core SoC on a Pixel 2 device?

TF Lite supports Android NN API that allows each phone to accelerate these models leveraging the custom accelerator on the phone.

Re: Developer preview of TensorFlow Lite

#20

Post by rajatmonga »

tadeegan wrote: post #12

How does this compare to using XLA for AOT compilation?

XLA for AOT is useful for cases when you know exactly what architecture you are shipping to, and are ok updating the code whenever the model changes.

TF Lite addresses the segment where you need more flexibility

- you ship single app to many types of devices

- would like to update the model independent of the code itself e.g. no change to Android APK, and update the model over the wire.

Even with this generality, TF Lite is still quite fast and lightweight as that was the focus building it up.

Post reply on HN