So we'll start to see more and more battery-consuming "AI" apps in mobile devices?
Yes.
Developer preview of TensorFlow Lite
41–50 of 80 posts
Re: Developer preview of TensorFlow Lite
#42This is pretty Android/iPhone-only, wish it can be more flexible to be used on other edge devices such as home routers or other embedded products where resource is constrained.
Re: Developer preview of TensorFlow Lite
#43I'm wondering if TF has something like pytorch's autograd. Does anyone know?
I only just briefly read the doc for autograd, but automatic differentiation is the strong default in TF if that's what you're asking.
Re: Developer preview of TensorFlow Lite
#44Earlier quoted context omitted.
CoreML doesn't actually support Tensorflow. It's support for Tensorflow is only through Keras which is fine if you just want to build stock standard models but if you're doing crazy research implementations then that's not going to work.
Is all in the converter tool, if the converter tool can get the tf file into a .mlmodel properly, then it will be supported. Inside is just a bunch of weights and layers and parameters. We just need a proper script to translate it
"just a bunch of weights and layers and parameters" -- I think you and the GP are agreeing. That's the definition of standard: If the model can be expressed using the currently-blessed set of layer definitions in CoreML, then yes. But if you're doing nonstandard stuff with weird control flow behavior, or RNNs that don't map into some of the common flavors, then all bets are off.
An example: Some of my colleagues put a QP solver in tandem with a DNN, so that the neural network could 'shell out' to the solver as part of its learning, and learned to solve small sudoku problems from examples alone: https://arxiv.org/abs/1703.00443 The pytorch code for it is one of the examples I like to use as a stress-test for doing funky things in the machine learning context.
TensorFlow is a very generic dataflow library at its heart - which happens to have a lot of DNN-specific functionality as ops. It's possible to express arbitrary computations in it, whereas CoreML and and similar frameworks make more assumptions that the computation will fit a particular mould, and optimize it thereby.
Re: Developer preview of TensorFlow Lite
#45So we'll start to see more and more battery-consuming "AI" apps in mobile devices?
And we'll start to see more battery-efficient hardware to run those apps without consuming all of your battery. :)
(I'm saying that glibly, but I'm dead serious -- look at what we've seen emerge just this year in Apple's Neural Engine, the Pixel Visual Core, rumored chips from Qualcomm, and the Movidius Myriad 2. The datacenter was the first place to get dedicated DNN accelerators in the form of Google's TPU, but the phones -- and even smaller devices, like the "clips" camera -- are the clear next spot. And this is why, for example, TensorFlow Lite can call into the Android DNNAPI to take advantage of local accelerators as they evolve.
Being able to run locally, if battery life is preserved, is a huge win in latency, privacy, potentially bandwidth, etc. It'll be good, though it does need advances in both the HW and the DNN techniques (things like Mobilenet, but we need far more).
Re: Developer preview of TensorFlow Lite
#46Earlier quoted context omitted.
What tradeoffs did you make compared to the original?
A few tradeoffs we had to make: - As mentioned below - flatbuffers makes the startup time faster while trading off some flexibility - Smaller code size means trading off dependency on some libraries and broader support vs writing more things from scratch more focused on the user cases people care about
Do you have performance/memory comparisons from using flatbuffer vs protobuf in TF? A quick writeup with how switching effected performance would be really interesting :)
Re: Developer preview of TensorFlow Lite
#47Earlier quoted context omitted.
Is all in the converter tool, if the converter tool can get the tf file into a .mlmodel properly, then it will be supported. Inside is just a bunch of weights and layers and parameters. We just need a proper script to translate it
"just a bunch of weights and layers and parameters" -- I think you and the GP are agreeing. That's the definition of standard: If the model can be expressed using the currently-blessed set of layer definitions in CoreML, then yes. But if you're doing nonstandard stuff with weird control flow behavior, or RNNs that don't map into some of the common flavors, then all bets are off. An example: Some of my colleagues put…
Looks like you are right, CoreML only support these 3 DNNs: Feedforward, convolutional, recurrent. I suppose capsule nets are not any one of those, if it were implemented in TF
Re: Developer preview of TensorFlow Lite
#48On iOS, does TensorFlow Lite utilize the GPU for inference when needed or is it CPU only?
If so, does it use OpenCL or something?
Re: Developer preview of TensorFlow Lite
#49I would like to how small an Inception-V3 model becomes when converted into .tflite format.
Re: Developer preview of TensorFlow Lite
#50This is pretty Android/iPhone-only, wish it can be more flexible to be used on other edge devices such as home routers or other embedded products where resource is constrained.
The current examples talk about Android/iPhone, however the core runtime is pretty lightweight with the goal of supporting all kinds of embedded products.
Do let us know if you build/run on other platforms.