Huseyn Gasimov
Huseyn Gasimov
Creator of this blog.
Jan 23, 2018 4 min read

Deep Learning in a Mobile Phone

thumbnail for this post

Recently, I developed an android application to demonstrate usage of machine learning (image recognition, in this case) in mobile phones. It is called “Detectron”. I built a deep neural network architecture that runs directly on an android phone and trained it with Tensorflow. The model can recognize 5 flowers: daisy, dandelion, rose, sunflower and tulips. It has a decent accuracy, considering the fact that it was trained in a small set of images and runs in a mobile device.

While the app was developed for a demo purpose, it is a nice example of using deep learning in a mobile phone. All AI work is powered 100% by the user’s device and images are processed without ever leaving their phone. This provides users with a snappier experience (no round trip to the cloud), offline availability, and better privacy. This also allows me to run the app at a cost of $0, even under the load of a thousand users, providing significant savings compared to traditional cloud-based AI approaches.

Deep learning

I trained a Convolutional Neural Network (CNN) to recognize flower images. One of the main decisions in this process was choosing the network architecture. There are several well known CNN architectures. I was thinking about choosing either Inception V3 or MobileNet. The principal difference is that Inception V3 is optimized for accuracy, while the MobileNet is optimized to be small and efficient at the cost of some accuracy. Inception V3 has a first-choice accuracy of 78% on ImageNet, but the model is 85MB and requires many times more processing than the largest MobileNet configuration, which achieves 70.5% accuracy with just a 19MB model size. Since I was preparing a model for a mobile phone, I chose MobileNet.

Second step was training the model. Deep network models require a lot of data to be trained on. In this case, I had to prepare a training set with thousands of flower pictures. Fortunately, there was another choice – Transfer Learning. With help of Transfer Learning one can retrain an already trained image recognition model to recognize different objects. This way one can build a model with decent accuracy using a small training set in small amount of time. For my android app, I retrained a MobileNet model to recognize 5 flowers: daisy, dandelion, rose, sunflower and tulips. The training set consisted of 3670 images, ~700 images per category. This is a small training set for training a deep neural network architecture. Although, using transfer learning allowed me to get a good enough accuracy.

Android app

After the MobileNet model was trained and was ready to use, the next question was how to use it in an android app. Here Tensorflow Mobile came into play. Traditionally, deep learning has been associated with data centers and giant clusters of high-powered GPU machines. However, it can be very expensive and time-consuming to send all of the image data across a network connection. Running on mobile makes it possible to deliver very interactive applications in a way that’s not possible when you have to wait for a network round trip.

To add TensorFlow to an android app, the simplest way is to add the following lines to the Gradle build file:

allprojects {
    repositories {
        jcenter()
    }
}

dependencies {
    compile 'org.tensorflow+'
}

This automatically downloads the latest stable version of TensorFlow as an AAR (Android Archive) and installs it in the project.

But wait a minute. Android apps need to be written in Java, and core TensorFlow is in C++. Actually, TensorFlow has a JNI (Java Native Interface) library to interface between the two. Its interface is aimed only at inference, so it provides the ability to load a graph, set up inputs, and run the model to calculate particular outputs. You can see the full documentation for the minimal set of methods in TensorFlowInferenceInterface.java

After being able to use the trained deep neural network in the android app, the rest of the app was pretty straightforward to build.

Demo

As a demo, you can see the following screencast which was recorded from the screen of an android phone.

The app has 3 views:

  • Detect from picture – Use can take a picture using the camera app. The taken picture is sent back to Detectron and it recognizes the flower in the taken picture.
  • Detect live – Image recognition happens real-time in the app.
  • About – Small description of the app.

You are welcome to install the app from Google Play using this link and play with it.