A real‑time Unreal Engine 5 digital twin for simulating cannula / needle insertion using haptic‑augmented physical inputs. The system connects an Arduino-based touch sensor array to UE5, where sensor signals act as hard constraints on whether insertion is allowed, mimicking correct vs incorrect clinical technique.
This project explores the intersection of:
- Medical simulation
- Digital twins
- Haptics & soft‑body electronics
- Human–machine interaction
A physical setup (Arduino + touch sensors) represents the real-world needle / cannula interaction.
- Correct sensor activation pattern → UE5 allows cannula insertion
- Incorrect / premature / misaligned touch → insertion is blocked
This enforces procedural correctness rather than simple animation playback, turning UE5 into a logic‑validated twin, not just a visual one.
| Capability | Description |
|---|---|
| Logic-Gated Insertion | Cannula insertion is allowed only when sensor activation patterns satisfy predefined correctness constraints. |
| Real-Time Digital Twin | Physical touch events from Arduino are mirrored instantly inside Unreal Engine 5. |
| Haptic-Driven Validation | Touch correctness governs possibility, not animation state or scripted flow. |
| Research-First Architecture | Designed to study soft electronics, tactile semantics, and human–machine coupling. |
| Focus Area | Exploration |
|---|---|
| Soft-Body Electronics | Flexible / deformable sensing materials for intent detection. |
| Constraint-Based Simulation | Enforcing physical correctness via logic gates. |
| Medical Training Systems | Procedural correctness over visual realism. |
| Human–Machine Interaction | Translating touch semantics into digital control. |
[ Touch Sensors ]
│
▼
[ Arduino ] ── Serial / USB ──► [ Unreal Engine 5 ]
│
▼
Digital Twin + Logic Gate
- Touch sensors detect pressure/contact
- Arduino samples & encodes sensor state
- Data streamed over Serial
- UE5 parses input in real time
- Insertion logic validates sensor pattern
- Needle/cannula state updates accordingly
-
Arduino (Uno / Nano / compatible)
-
Touch / pressure sensors (capacitive or resistive)
-
Optional:
- Flexible or soft‑body conductive materials
- Custom sensor matrices for research
Sensor placement and timing are treated as semantic input, not binary buttons.
-
Engine: Unreal Engine 5.x
-
Input: Serial communication (custom reader / plugin)
-
Logic Layer:
- Sensor validation gates
- Insertion permission checks
- Error state handling
- All required sensors must be active
- Activation order must match expected sequence
- No forbidden sensor may trigger
Failing any rule immediately blocks insertion.
This project is not about simple VR animation.
It investigates:
- Soft‑body electronics as intent sensors
- Real‑time constraint enforcement in digital twins
- Translating tactile correctness into simulation logic
- Training systems where wrong touch = physical impossibility
- Medical training simulators
- Cannula / needle insertion practice
- Haptic‑augmented digital twins
- Research into soft electronic sensing
- Arduino ↔ UE5 serial communication established
- Touch sensor input mapped to UE logic
- Cannula insertion gated by sensor correctness
- Ongoing experimentation with soft‑body sensing
- Physics‑based soft tissue response
- Force feedback / haptic resistance
- Higher‑resolution sensor matrices
- ML‑based pattern recognition for touch validation
This is a research & simulation project, not a certified medical device. It is intended for experimentation, learning, and prototyping only.