Project Overview
This project intends to deal with integrating and building a object detection framework and its dectection based on camera and peripheral micro-controller (Arduino BLE 33 mBed) connected to a NVIDIA Jetson Nano acting as a hub. For the purpose of demonstration we would be attaching a low power camera attached to the micro-controller to act as an inference engine built over Machine Learning Framework to interact with Jetson Nano and form a feedback loop to alarm a system of detection of person in frame of camera. The aim of this project is to train a machine learning model and deploy over low-power, low-memory micro-controllers and form feedback system of the Embedded Network Hub (Jetson). This would include bringing up completely independent hardware which includes the controlling unit, sensors, and software bring up, which includes Operating systems, communication protocols, coding styles, and languages.
Using low-level micro-controllers is cheap and provides more opportunity (future goal) to scale the product under development and testing providing both financial and engineering benefits.
NVIDIA Jetson acts as a server for the peripheral boards(clients). The term voting technique indicates that at any given time the reading that is received from the majority of the sensors will be considered for further actions. Two Camera feeds are available to work with one resides on the NVIDIA Jetson and other connected to the Peripheral Board.
Major Goals:
- Yocto to compile meta-tegra.
- Running Tegra-demo-distro with Arduino Setup.
- Setup Camera board and Communication.
- Build deep-stream or opencv (hardware accelerated) running on Jetson Nano.
- Run AI Image recognition project working with on image on Host/Hub (Jetson) and/or Arduino BLE 33 with Camera.
System works as a hard real time system, which captures camera feed at set time interval irrespective of how much time the sensors take to send data over to Nano. The overaching goal of this project would be create a Hub and Client mechanism such that machine learning can be deployed on micro-controllers with and without GPU's.
See Software Block Diagram
We intend to use Yocto as the bring up tool to generate Linux image on NVIDIA Jetson Nano.
Intended hardware platforms for this project:
- Yocto
- Signal handlers
- Clocks
- Init Scripts and Rootfs Overlays
- Yocto Image for Jetson Nano Markdown
- Creating Modules and scripts for Arduino BLE 33
- Interface for Jetson and Arduino Markdown
- Deplying Tiny Machine Learning (TinyML)
- Case study for TinyML Markdown
- Project Webpage: Dynamically Generated Documentation
- Project Overview: Project Synopsis
- Project Schedule
- Caleb's Repository: One-click-build script Focused
- Arpit's Repository: Image Development Focused
- Zach's Repository: Driver & Module Code Focused
- Caleb: Yocto Image Setup and build for Jetson Tegra, Dockerfile 1-click-image
- Zach: Serial communication Between Subsystems
- Arpit: Machine Learning Framework and TinyML Setup and yocto compatibility.
- Dive Deep into Tiny Machine Learning deployment
- Integrate Camera and peripheral to detect Person in frame.
- Code Porting and Tensorflow dependencies.