Announcing Android’s updateable, fully integrated ML inference stack

Posted by Oli Gaymond, Product Supervisor, Android ML

On-System Machine Studying gives decrease latency, extra environment friendly battery utilization, and options that don’t require community connectivity. We’ve got discovered that growth groups deploying on-device ML on Android immediately encounter these widespread challenges:

• Many apps are dimension constrained, so having to bundle and handle extra libraries only for ML could be a important value
• In contrast to server-based ML, the compute setting is very heterogeneous, leading to important variations in efficiency, stability and accuracy
• Maximising attain can result in utilizing older extra broadly obtainable APIs; which limits utilization of the most recent advances in ML.

To assist remedy these issues, we’ve constructed Android ML Platform – an updateable, totally built-in ML inference stack. With Android ML Platform, builders get:

• In-built on-device inference necessities – we are going to present on-device inference binaries with Android and preserve them updated; this reduces apk dimension
• Optimum efficiency on all gadgets – we are going to optimize the combination with Android to robotically make efficiency choices primarily based on the machine, together with enabling {hardware} acceleration when obtainable
• A constant API that spans Android variations – common updates are delivered through Google Play Providers and are made obtainable exterior of the Android OS launch cycle

In-built on-device inference necessities – TensorFlow Lite for Android

TensorFlow Lite will probably be obtainable on all gadgets with Google Play Providers. Builders will now not want to incorporate the runtime of their apps, decreasing app dimension. Furthermore, TensorFlow Lite for Android will use metadata within the mannequin to robotically allow {hardware} acceleration, permitting builders to get the very best efficiency potential on every Android machine.

Optimum efficiency on all gadgets – Computerized Acceleration

Computerized Acceleration is a brand new function in TensorFlowLite for Android. It allows per-model testing to create allowlists for particular gadgets taking efficiency, accuracy and stability under consideration. These allowlists can be utilized at runtime to resolve when to activate {hardware} acceleration. With a purpose to use accelerator allowlisting, builders might want to present extra metadata to confirm correctness. Computerized Acceleration will probably be obtainable later this yr.

A constant API that spans Android variations

In addition to preserving TensorFlow Lite for Android updated through common updates, we’re additionally going to be updating the Neural Networks API exterior of OS releases whereas preserving the API specification the identical throughout Android variations. As well as we’re working with chipset distributors to supply the most recent drivers for his or her {hardware} on to gadgets, exterior of OS updates. It will let builders dramatically scale back testing from 1000’s of gadgets to a handful of configurations. We’re excited to announce that we’ll be launching later this yr with Qualcomm as our first accomplice.

Signal-up for our early entry program

Whereas a number of of those options will roll out later this yr, we’re offering early entry to TensorFlow Lite for Android to builders who’re excited by getting began sooner. You possibly can sign-up for our early entry program right here.