Added Gesture Control with OpenVINO

[vijaykr458]

added Gesture Control with OpenVINO

This project is a part of Google Summer of Code 2025

[adrianboguszewski]

Thanks, @vijaykr458

We're very close to merge it, just please look at the following comments:

• update the dir name to "palm_pilot" and update all the paths in this PR

Is it possible to run your app on Linux? Have you tried it?

[Copilot]

Pull Request Overview

This pull request adds a comprehensive gesture control system with OpenVINO as part of Google Summer of Code 2025. The system provides real-time hand gesture recognition using MediaPipe models running on OpenVINO inference, supporting multiple application modes including PowerPoint control, media player control, browser navigation, and racing game controls.

Key changes include:

• Complete gesture recognition pipeline with palm detection, hand landmarks, and gesture classification
• Modern PyQt6-based GUI with configuration management and live video preview
• Multi-modal application system supporting different interaction paradigms
• Performance benchmarking tools with device-specific optimization
• Cross-platform installation and deployment scripts

Reviewed Changes

Copilot reviewed 24 out of 33 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
src/gui_main.py	Main PyQt6 GUI application with real-time video, mode switching, and configuration dialogs
src/gesture_engine.py	Core gesture processing engine with OpenVINO model management and rendering
src/openvino_models.py	OpenVINO model manager with device selection and compilation
src/hand_landmark.py	Hand detection and landmark processing with MediaPipe-style algorithms
src/config_manager.py	Configuration management system with JSON persistence and validation
src/application_modes.py	Application mode manager for different gesture interaction contexts
src/game_controller.py	Two-handed racing game controller with steering wheel simulation
src/volume_controller.py	System volume control using Windows pycaw library
src/benchmark_dialog.py	Performance benchmarking tools with device configuration
utils/demo_utils.py	Utility functions for video processing and OpenVINO watermarks
setup/*.sh	Cross-platform installation and run scripts

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[adrianboguszewski]

Please verify on a fresh machine if it works, because I get "No module named PyQt6. Make sure all required modules are in the src/ directory"

[vijaykr458]

Thanks, @vijaykr458

We're very close to merge it, just please look at the following comments:

• update the dir name to "palm_pilot" and update all the paths in this PR

Is it possible to run your app on Linux? Have you tried it?

I haven’t really tested it yet. the whole application was developed on windows — pyautogui is cross-platform, so it should work fine for normal modes like ppt, browser, media player, or any other custom mode.

however, some parts use windows-only apis:
racing mode relies on pydirectinput since some older games don’t support pyautogui
volume controller mode uses pycaw for handling volume

[vijaykr458]

Please verify on a fresh machine if it works, because I get "No module named PyQt6. Make sure all required modules are in the src/ directory"

I installed it on a fresh machine, and it runs.

I am not sure how that error came,
please try doing a manual installation of creating a virtual environment, then installing the requirements and running it again

[adrianboguszewski]

Could we also update pywin32? Version 306 is very old and thus limiting the python version (I cannot use python 3.13)