Android AICamera Demo

Demo/Android/AICamera/.gitignore

@@ -0,0 +1,56 @@
+#Built application files
+*.apk
+*.ap_
+
+# Files for the ART/Dalvik VM
+*.dex
+
+# Java class files
+*.class
+
+# Generated files
+bin/
+gen/
+out/
+
+# Gradle files
+.gradle/
+build/
+
+# Local configuration file (sdk path, etc)
+local.properties
+
+# Proguard folder generated by Eclipse
+proguard/
+
+# Log Files
+*.log
+
+# Android Studio Navigation editor temp files
+.navigation/
+
+# Android Studio captures folder
+captures/
+
+# IntelliJ
+*.iml
+.idea/workspace.xml
+.idea/tasks.xml
+.idea/gradle.xml
+.idea/dictionaries
+.idea/libraries
+
+# Keystore files
+# Uncomment the following line if you do not want to check your keystore files in.
+#*.jks
+
+# External native build folder generated in Android Studio 2.2 and later
+.externalNativeBuild
+
+# Google Services (e.g. APIs or Firebase)
+google-services.json
+
+# Freeline
+freeline.py
+freeline/
+freeline_project_description.json

Demo/Android/AICamera/README.md

@@ -0,0 +1,103 @@
+# PDCamera Android Demo
+
+- [Intro](#intro)
+  - [Models](#models)
+  - [Demo](#demo)
+- [Quick Install](#quick-install)
+- [Development](#development)
+  - [Prepare Models](#prepare-models)
+  - [Integrate Paddle Library to Android](#integrate-paddle-library-to-android)
+
+## Intro
+
+This Android demo shows PaddlePaddle running SSD(Single Shot MultiBox Detector）Object detection on Android devices locally and offline. It loads a pretrained model with PaddlePaddle and uses camera to capture images and call PaddlePaddle's inference C API from library to show detected objects to users.
+
+You can look at SSD model architecture, and how to train SSD Model with PaddlePaddle [here](https://github.com/PaddlePaddle/models/tree/develop/ssd) and a linux demo [here](https://github.com/PaddlePaddle/Mobile/tree/develop/Demo/linux)
+
+### Models
+
+We have provided two pretrained models for demo purposes. `pascal_mobilenet_300_66` can classify and detect 20 common categories while `face_mobilenet_160_91` is for human facial detection. Following table shows the corresponding accuracy for each model:
+
+| Model                    | Dimensions | Accuracy |  Size |
+| ------------------------ |:----------:| --------:|------:|
+| [pascal\_mobilenet\_300\_66.paddle](http://cloud.dlnel.org/filepub/?uuid=39c325d9-b468-4940-ba47-d50c8ec5fd5b) | 300 x 300 | 66% | 23.2MB |
+| [face\_mobilenet\_160\_91.paddle](http://cloud.dlnel.org/filepub/?uuid=038c1dbf-08b3-42a9-b2dc-efccd63859fb) | 160 x 160 | 91% | 18.4MB |
+
+Click on the model name in the table to download the corresponding model through the browser.
+
+Following is a list of 20 objects that pascal model can classify:
+
+- aeroplane
+- bicycle
+- background
+- boat
+- bottle
+- bus
+- car
+- cat
+- chair
+- cow
+- diningtable
+- dog
+- horse
+- motorbike
+- person
+- pottedplant
+- sheep
+- sofa
+- train
+- tvmonitor
+
+### Demo
+
+The app requires Camera and Storage permission. After granting permissions, tap on the screen to toggle settings which shows (pic-1):
+
+- Models: Select Pascal MobileNet 300 or Face MobileNet 160 (Requires restart)
+- Camera: Select Front or Back Camera
+- Accuracy Threshold: Adjust threshold to filter more/less objects based on probability
+
+
+<p align="center">
+<img src="readme_assets/demo_app.jpg" width = "25%" />
+<img src="readme_assets/demo_pascal.jpg" width = "25%" />
+<img src="readme_assets/demo_face.jpg" width = "25%" /><br/>
+（pic-1）App Settings &nbsp;&nbsp;&nbsp;&nbsp;
+（pic-2）Common object classifications &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+（pic-3）Face detection
+</p>
+
+
+For Pascal model, detected object will be highlighted as a bounding box with a classified object label and probability(pic-2).
+For Face model, detected face will be highlighted as a bounding box and probability(pic-3).
+
+
+## Quick Install
+
+To simply run the demo with Android devices, scan the QR code below to download and install the apk. It runs on Android 5.0+ devices, support both arm64 and armv7a architecture.
+
+<p align="center">
+<img src="readme_assets/qr_code_android.jpg" width = "30%" />
+</p>
+
+
+## Development
+
+Use latest Android Studio for development. This demo requires a camera for object detection, therefore you must use an Android device for development and testing. Emulators will not work as they cannot access camera.
+
+For developers, feel free to use this as a reference to start a new project. This demo fully demonstrates how to integrate Paddle C Library to Android, including CMake scripts and JNI wrapper and called from Java. Download Android NDK tools from Android Studio in order to debug c++ code. Click [here](https://developer.android.com/ndk/guides/index.html) for Android NDK guide.
+
+
+### Prepare Models
+
+Our models are too large to upload to Github. Create a model folder and add to assets folder. Download [face_mobilenet_160_91.paddle](http://cloud.dlnel.org/filepub/?uuid=038c1dbf-08b3-42a9-b2dc-efccd63859fb) and [pascal_mobilenet_300_66.paddle](http://cloud.dlnel.org/filepub/?uuid=39c325d9-b468-4940-ba47-d50c8ec5fd5b) and put them to model folder.
+
+Here, we are using **merged model**. If you want to know how to use model config file (such as `config.py`) and model params file from training(such as `params_pass_0.tar.gz`) to get **merged model** file, please see [Merged model config and parameters](../../../deployment/model/merge_config_parameters/README.md)
+
+
+### Integrate Paddle Library to Android
+
+- Follow this guide [Build PaddlePaddle for Android](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/mobile/cross_compiling_for_android_en.md) to generate paddle libs(include, lib, third_party).
+- Create a folder paddle-android and add to PROJECT_ROOT/AICamera/app . Put the 3 paddle libs folder(`include`,`lib`,`third_party`) under paddle-android.
+- Add [FindPaddle.cmake](https://github.com/PaddlePaddle/Mobile/blob/develop/Demo/Android/AICamera/app/FindPaddle.cmake) in PROJECT_ROOT/AICamera/app to locate the dependencies of Paddle libraries. You should be using this file for your android project.
+- Add [CMakeList.txt](https://github.com/PaddlePaddle/Mobile/blob/develop/Demo/Android/AICamera/app/CMakeList.txt) in PROJECT_ROOT/AICamera/app to link your C++ files to Paddle Libraries.
+- Create JNI wrapper such as [image_recognizer_jni.cpp](https://github.com/PaddlePaddle/Mobile/blob/develop/Demo/Android/AICamera/app/src/main/cpp/image_recognizer_jni.cpp) to access C++ API in Paddle Library and a bridge calling from Java client.

Demo/Android/AICamera/app/.gitignore

@@ -0,0 +1,56 @@
+#Built application files
+*.apk
+*.ap_
+
+# Files for the ART/Dalvik VM
+*.dex
+
+# Java class files
+*.class
+
+# Generated files
+bin/
+gen/
+out/
+
+# Gradle files
+.gradle/
+build/
+
+# Local configuration file (sdk path, etc)
+local.properties
+
+# Proguard folder generated by Eclipse
+proguard/
+
+# Log Files
+*.log
+
+# Android Studio Navigation editor temp files
+.navigation/
+
+# Android Studio captures folder
+captures/
+
+# IntelliJ
+*.iml
+.idea/workspace.xml
+.idea/tasks.xml
+.idea/gradle.xml
+.idea/dictionaries
+.idea/libraries
+
+# Keystore files
+# Uncomment the following line if you do not want to check your keystore files in.
+#*.jks
+
+# External native build folder generated in Android Studio 2.2 and later
+.externalNativeBuild
+
+# Google Services (e.g. APIs or Firebase)
+google-services.json
+
+# Freeline
+freeline.py
+freeline/
+freeline_project_description.json

Demo/Android/AICamera/app/CMakeLists.txt

@@ -0,0 +1,43 @@
+# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License
+
+# Sets the minimum version of CMake required to build the native
+# library. You should either keep the default value or only pass a
+# value of 3.4.0 or lower.
+
+cmake_minimum_required(VERSION 3.4.1)
+
+set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/")
+
+# Prebuilt third-party library: paddle
+set(PADDLE_ROOT ${CMAKE_SOURCE_DIR}/paddle-android)
+find_package(Paddle)
+
+set(SRC_FILES src/main/cpp/image_recognizer_jni.cpp
+              src/main/cpp/paddle_image_recognizer.cpp
+              src/main/cpp/binary_reader.cpp
+              src/main/cpp/image_utils.cpp)
+
+set(LINK_FLAGS "-Wl,--version-script ${CMAKE_CURRENT_SOURCE_DIR}/paddle_jni.map")
+add_library(paddle_image_recognizer SHARED ${SRC_FILES})
+set_target_properties(paddle_image_recognizer PROPERTIES LINK_FLAGS "${LINK_FLAGS}")
+
+target_include_directories(paddle_image_recognizer PRIVATE
+                           ${PADDLE_INC_DIR})
+
+target_link_libraries(paddle_image_recognizer
+                      ${PADDLE_LIBRARIES}
+                      ${PADDLE_THIRD_PARTY_LIBRARIES}
+                      android
+                      log)

Demo/Android/AICamera/app/FindPaddle.cmake

@@ -0,0 +1,91 @@
+# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License
+
+if(NOT PADDLE_ROOT)
+   set(PADDLE_ROOT $ENV{PADDLE_ROOT} CACHE PATH "Paddle Path")
+endif()
+if(NOT PADDLE_ROOT)
+  message(FATAL_ERROR "Set PADDLE_ROOT as your root directory installed PaddlePaddle")
+endif()
+
+set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE BOTH)
+set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY BOTH)
+
+find_path(PADDLE_INC_DIR NAMES paddle/capi.h PATHS ${PADDLE_ROOT}/include)
+find_library(PADDLE_SHARED_LIB NAMES paddle_capi_shared PATHS
+    ${PADDLE_ROOT}/lib/${ANDROID_ABI})
+find_library(PADDLE_WHOLE_LIB NAMES paddle_capi_whole PATHS
+    ${PADDLE_ROOT}/lib/${ANDROID_ABI})
+find_library(PADDLE_LAYERS_LIB NAMES paddle_capi_layers PATHS
+    ${PADDLE_ROOT}/lib/${ANDROID_ABI})
+find_library(PADDLE_ENGINE_LIB NAMES paddle_capi_engine PATHS
+    ${PADDLE_ROOT}/lib/${ANDROID_ABI})
+if(PADDLE_INC_DIR AND PADDLE_LAYERS_LIB AND PADDLE_ENGINE_LIB)
+  add_library(paddle_capi_layers STATIC IMPORTED)
+  set_target_properties(paddle_capi_layers PROPERTIES IMPORTED_LOCATION
+                        ${PADDLE_LAYERS_LIB})
+  add_library(paddle_capi_engine STATIC IMPORTED)
+  set_target_properties(paddle_capi_engine PROPERTIES IMPORTED_LOCATION
+                        ${PADDLE_ENGINE_LIB})
+  set(PADDLE_LIBRARIES -Wl,--start-group
+                       -Wl,--whole-archive paddle_capi_layers
+                       -Wl,--no-whole-archive paddle_capi_engine
+                       -Wl,--end-group)
+  message(STATUS "Found PaddlePaddle (include: ${PADDLE_INC_DIR}; "
+                 "library: ${PADDLE_LAYERS_LIB}, ${PADDLE_ENGINE_LIB})")
+elseif(PADDLE_INC_DIR AND PADDLE_WHOLE_LIB)
+  add_library(paddle_capi_whole STATIC IMPORTED)
+  set_target_properties(paddle_capi_whole PROPERTIES IMPORTED_LOCATION
+                        ${PADDLE_WHOLE_LIB})
+  set(PADDLE_LIBRARIES -Wl,--whole-archive paddle_capi_whole -Wl,--no-whole-archive)
+else()
+  message(FATAL_ERROR "Cannot find PaddlePaddle on ${PADDLE_ROOT}\n"
+          "\tPADDLE_INC_DIR: ${PADDLE_INC_DIR}\n"
+          "\tPADDLE_LAYERS_LIB: ${PADDLE_LAYERS_LIB}\n"
+          "\tPADDLE_ENGINE_LIB: ${PADDLE_ENGINE_LIB}\n"
+          "\tPADDLE_WHOLE_LIB: ${PADDLE_WHOLE_LIB}")
+endif()
+
+include_directories(${PADDLE_INC_DIR})
+
+set(THIRD_PARTY_ROOT ${PADDLE_ROOT}/third_party)
+function(third_party_library TARGET_NAME HEADER_NAME LIBRARY_NAME)
+  find_path(${TARGET_NAME}_INC_DIR ${HEADER_NAME} PATHS
+      ${THIRD_PARTY_ROOT}/${TARGET_NAME}/include
+      NO_DEFAULT_PATH)
+  find_library(${TARGET_NAME}_STATIC_LIBRARY NAMES ${LIBRARY_NAME} PATHS
+      ${THIRD_PARTY_ROOT}/${TARGET_NAME}/lib/${ANDROID_ABI}
+      NO_DEFAULT_PATH)
+  if(${TARGET_NAME}_INC_DIR AND ${TARGET_NAME}_STATIC_LIBRARY)
+    add_library(${TARGET_NAME} STATIC IMPORTED)
+    set_target_properties(${TARGET_NAME} PROPERTIES IMPORTED_LOCATION
+        ${${TARGET_NAME}_STATIC_LIBRARY})
+    set(PADDLE_THIRD_PARTY_LIBRARIES
+        ${PADDLE_THIRD_PARTY_LIBRARIES} ${TARGET_NAME} PARENT_SCOPE)
+    message(STATUS "Found ${TARGET_NAME}: " ${${TARGET_NAME}_STATIC_LIBRARY})
+  else()
+    message(WARNING "Cannot find ${TARGET_NAME} under ${THIRD_PARTY_ROOT}")
+  endif()
+endfunction()
+
+set(PADDLE_THIRD_PARTY_LIBRARIES)
+third_party_library(protobuf google/protobuf/message.h protobuf)
+third_party_library(glog glog/logging.h glog)
+third_party_library(openblas cblas.h openblas)
+third_party_library(gflags gflags/gflags.h gflags)
+third_party_library(zlib zlib.h z)
+
+set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
+