layout	title	permalink
default	Home	/Overview.html

Overview

This project intends to deal with integrating and building a object detection framework and its dectection based on camera and peripheral micro-controller (Arduino BLE 33 mBed) connected to a NVIDIA Jetson Nano acting as a hub. For the purpose of demonstration we would be attaching a low power camera attached to the micro-controller to act as an inference engine built over Machine Learning Framework to interact with Jetson Nano and form a feedback loop to alarm a system of detection of person in frame of camera. The aim of this project is to train a machine learning model and deploy over low-power, low-memory micro-controllers and form feedback system of the Embedded Network Hub (Jetson). This would include bringing up completely independent hardware which includes the controlling unit, sensors, and software bring up, which includes Operating systems, communication protocols, coding styles, and languages.

Using low-level micro-controllers is cheap and provides more opportunity (future goal) to scale the product under development and testing providing both financial and engineering benefits.

NVIDIA Jetson acts as a server for the peripheral boards(clients). The term voting technique indicates that at any given time the reading that is received from the majority of the sensors will be considered for further actions.

Major Goals:

• Yocto to compile meta-tegra.
• Running Tegra-demo-distro with Arduino Setup.
• Setup Camera board and Communication.
• Create custom UART-server, API for integration and communication with externel micro-controller.
• Run AI Image recognition project working with on image on Host/Hub (Jetson) and/or Arduino BLE 33 with Camera.

System works as a hard real time system, which captures camera feed at set time interval irrespective of how much time the sensors take to send data over to Nano. The overaching goal of this project would be create a Hub and Client mechanism such that machine learning can be deployed on micro-controllers with and without GPU's.

Block Diagrams

Hardware Block Diagrams

Software Block Diagrams

Functional Block Diagrams

See Software Block Diagram

API

API

Target Build System

We intend to use Yocto as the bring up tool to generate Linux image on NVIDIA Jetson Nano.

Hardware Platform

Intended hardware platforms for this project:

• Jetson Nano 2GB Board
• Arduino Nano 33 BLE
• OV7675 Camera

Open Source Projects

• Tensorflow
• Tensorflow Lite
• Tensorflow Micro
• Meta Tegra for Yocto
• Arduino CLI
• Arduino IDE
• TinyML

Previously Used Content

• Yocto
• Signal handlers
• Clocks
• Init Scripts and Rootfs Overlays

New Content

• Yocto Image for Jetson Nano
• Yocto Image for Jetson Docker
• Creating Modules and scripts for Arduino BLE 33
• Custom UART-Server
• Interface for Jetson and Arduino
• Case study for TinyML
• Deploying TinyML

Shared Material

• Flashing Image to SD Card for Jetson Nano

Source Code Organization

• Project Webpage: Dynamically Generated Documentation
• Project Overview: Project Synopsis
• Project Schedule: Project Schedule
• Caleb's Repository: One-click-build script Focused
• Arpit's Repository: Image Development Focused
• Zach's Repository: Driver & Module Code Focused

Group's Overview

Team Project Members

• Caleb: Yocto Image Setup and build for Jetson Tegra, Dockerfile 1-click-image
• Zach: Serial communication Between Subsystems
• Arpit: Machine Learning Framework and TinyML Setup and yocto compatibility.
  • Tiny Machine Learning deployment - [Model creation, Neural network training, Tensorflow Micro]
  • Computer Vision : Integrate Camera and peripheral to detect Person in frame.
  • Code Porting and Tensorflow dependencies.
  • { Side Project }: Dummy linux wifi driver for cfg80211 USB Wifi Module

Schedule Page

• Project Schedule: Project Schedule

Student Video Demos

• Arpit Savarkar's Video
• Caleb Provost's Video
• Zach Turner's Video