TensorFlow Lite IoT Image Classifier

This sample demonstrates how to run TensorFlow Lite inference on Android Things.
Push a button to capture an image with the camera, and TensorFlow Lite will tell
you what it is!
Follow the Image Classifier Codelab
step-by-step instructions on how to build a similar sample.

Note: The Android Things Console will be turned down for non-commercial
use on January 5, 2022. For more details, see the
FAQ page.

Introduction

When a button is pushed or when the touchscreen is touched, the current image is captured from the
camera. The image is then converted and piped into a TensorFlow Lite classifier model that
identifies what is in the image. Up to three results with the highest confidence returned by the
classifier are shown on the screen, if there is an attached display. Also, the result is spoken out
loud using Text-To-Speech to the default audio output.

This project is based on the
TensorFlow Android Camera Demo TF_Classify app
and was adapted to use TensorFlow Lite, a lightweight version of TensorFlow targeted at mobile
devices. The TensorFlow classifier model is
MobileNet_v1
pre-trained on the ImageNet ILSVRC2012 dataset.

This sample uses the TensorFlow Lite inference library
and does not require any native build tools. You can add the TensorFlow Lite inference library to
your project by adding a dependency in your build.gradle, for example:

dependencies {
    compile 'org.tensorflow:tensorflow-lite:0.1.1'
}

Note: this sample requires a camera. Find an appropriate board in the
documentation.

Screenshots

(Watch the demo on YouTube)

Pre-requisites

• Android Things compatible board and an attached camera
• Android Studio 2.2+
• The following optional components:
  • one button and one resistor for triggering the camera
  • one LED and one resistor for the “ready” indicator
  • speaker or headphones for Text-To-Speech results
  • touchscreen or display for showing results

Schematics

Run on Android Things Starter Kit

If you have an Android Things Starter Kit, you can easily run this sample on your i.MX7D development board from the Android Things Toolkit app.

To run the sample on your i.MX7D development board:

1. Set up your device using Toolkit
2. Navigate to the Apps tab
3. Select Run next to the Image Classifier sample.
4. Press the “A” button on your Rainbow HAT or tap on the display to take a photo.

Build and Install

On Android Studio, click on the “Run” button.
If you prefer to run on the command line, type

./gradlew installDebug
adb shell am start com.example.androidthings.imageclassifier/.ImageClassifierActivity

If you have everything set up correctly:

1. Wait until the LED turns on
2. Point the camera to something like a dog, cat or a furniture
3. Push the button to take a picture
4. The LED should go off while running. In a Raspberry Pi 3, it takes about 500 millisecond to
   capture the picture and run it through TensorFlow, and some extra time to speak the results
   through Text-To-Speech
5. Inference results will show in logcat and, if there is a display connected,
   both the image and the results will be shown
6. If a speaker or headphones are connected, the results will be spoken via
   text to speech

Enable auto-launch behavior

This sample app is currently configured to launch only when deployed from your
development machine. To enable the main activity to launch automatically on boot,
add the following intent-filter to the app’s manifest file:

<activity ...>
   <intent-filter>
       <action android:name="android.intent.action.MAIN"></action>
       <category android:name="android.intent.category.HOME"></category>
       <category android:name="android.intent.category.DEFAULT"></category>
   </intent-filter>
</activity>

License

Copyright 2018 The Android Things Samples Authors.

Licensed under the Apache License, Version 2.0 (the “License”);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.