Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Developer preview of TensorFlow Lite

developers.googleblog.com

Post reply on HN
Print view View original

31–40 of 80 posts

Re: Developer preview of TensorFlow Lite

#31

Post by linuxkerneldev »

rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

> Looking forward to your feedback as you try it out.

Thanks Rajat. We use typical Cortex-A9/A7 SoCs running plain Linux rather than Android. We would use it for inference.

1. Platform choice

Why make TFL Android/iOS only? TF works on plain Linux. TFL even uses NDK and it would appear the inference part could work on plain Linux.

2. Performance

I did not find any info on performance of TensorFlow Lite. Mainly interested in inference performance. The tag "low-latency inference" catches my eye, just want to know how low is low latency here? milliseconds?

Re: Developer preview of TensorFlow Lite

#32

Post by dr1337 »

m3kw9 wrote: post #21

Why would I use this for iOS when I can use CoreML and convert TensorFlow into a CoreML model where there is already native support?

CoreML doesn't actually support Tensorflow. It's support for Tensorflow is only through Keras which is fine if you just want to build stock standard models but if you're doing crazy research implementations then that's not going to work.

Re: Developer preview of TensorFlow Lite

#33

Post by rasmi »

barbolo wrote: post #16

Would that be a viable option to deploy TensorFlow models on serverless environments (Lambda, Functions)?

You can deploy TensorFlow model binaries as serverless APIs on Google Cloud ML Engine [1]. But I would also be interested in seeing a TensorFlow Lite implementation.

[1] https://cloud.google.com/ml-engine/docs/deploying-models

Disclaimer: I work for Google Cloud.

Re: Developer preview of TensorFlow Lite

#34

Post by petewarden »

lern_too_spel wrote: post #13
rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

What about using XLA to compile libraries for mobile deployment fusing only the operations needed by the model?

One nice thing about Lite is that it's a lot easier to just include the operations you need (compared to TensorFlow 'classic'), there's fusion for common patterns, and the base interpreter is only 70KB. That covers a lot of the advantages of using XLA for mobile apps. In return you have the ability to load models separately from the code, and the ops are hand-optimized for ARM.

I'm still a fan of XLA, and I expect the two will grow closer over time, but I think Lite is better for a lot of scenarios on mobile.

Re: Developer preview of TensorFlow Lite

#35

Post by m3kw9 »

dr1337 wrote: post #32
m3kw9 wrote: post #21

Why would I use this for iOS when I can use CoreML and convert TensorFlow into a CoreML model where there is already native support?

CoreML doesn't actually support Tensorflow. It's support for Tensorflow is only through Keras which is fine if you just want to build stock standard models but if you're doing crazy research implementations then that's not going to work.

Is all in the converter tool, if the converter tool can get the tf file into a .mlmodel properly, then it will be supported. Inside is just a bunch of weights and layers and parameters. We just need a proper script to translate it

Re: Developer preview of TensorFlow Lite

#36

Post by azinman2 »

rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

Will CoreML (or any hardware acceleration) on iOS be supported?

Re: Developer preview of TensorFlow Lite

#40

Post by jonnycowboy »

linuxkerneldev wrote: post #31
rajatmonga wrote: post #2

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models. Looking forward to yo…

> Looking forward to your feedback as you try it out. Thanks Rajat. We use typical Cortex-A9/A7 SoCs running plain Linux rather than Android. We would use it for inference. 1. Platform choice Why make TFL Android/iOS only? TF works on plain Linux. TFL even uses NDK and it would appear the inference part could work on plain Linux. 2. Performance I did not find any info on performance of TensorFlow Lite. Mainly interes…

Also interested in answers to these two questions, as well as OpenCL performance in vanilla linux (iMX6 and above).
Post reply on HN