Skip to content
ahmedvictor507Public

About

Paper ECG photo → digital signal → interpretation, running entirely in the browser (YOLOv8, 1D U-Net, Pyodide, ONNX Runtime Web)

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

9 Commits

Folders and files

Repository files navigation

🫀 ECGMate

Turn a photo of a paper 12-lead ECG back into a digital signal, then measure and interpret it, entirely in the browser.

▶ Try the live demo: click one of the example images. (First visit downloads ~60 MB: the Python runtime plus two neural networks. After that it's cached, and each analysis takes ~7 s on a laptop.)

ECGMate web app

Why

Millions of ECGs still exist only on paper or as scanned images, so they can't be used by modern software or ML models. ECGMate recovers the underlying signal (in millivolts, at 500 Hz) from an image and runs an automated first-pass analysis on it.

Privacy by design: the whole pipeline, including both neural networks, runs client-side. The ECG image never leaves the visitor's device, and the site is just static files.

How it works

flowchart LR
    A[ECG photo / scan] --> B[1. Lead detection<br/>YOLOv8]
    B --> C[2. Digitization<br/>grid removal + DP tracing]
    C --> D[3. Wave segmentation<br/>1D U-Net + QRS detector]
    D --> E[4. Interpretation<br/>rule-based measurements]
    E --> F[Report + CSV]
Loading
Stage What happens
1. Lead detection A fine-tuned YOLOv8s model finds the 12 standard leads plus the 10 s rhythm strip and crops each one.
2. Digitization Grid lines are removed with colour filtering and a morphological black-hat transform, which keeps thin dark strokes (the trace) and ignores large stains and shadows. The trace is then followed column by column with dynamic programming, bridging short gaps. The mm scale comes from the paper grid itself (FFT of the grid pattern), so no manual calibration is needed. Calibration pulses and separator ticks are trimmed off automatically.
3. Segmentation A 1D U-Net labels every sample as P wave, T wave or background. QRS complexes are anchored by a Pan-Tompkins-style detector (band-pass, slope energy, adaptive peaks), since the network is weakest on short, highly variable QRS shapes.
4. Interpretation Heart rate (a robust vote across all leads), PR, QRS, QT/QTc (Bazett and Fridericia), electrical axis and ST deviation, then plain-language findings graded info / moderate / severe.

Running Python and PyTorch models in a static web page

flowchart LR
    UI[index.html + app.js<br/>UI, canvas charts] <-->|postMessage| W[Web Worker]
    W --> P[Pyodide<br/>the same ecgmate Python package:<br/>NumPy, SciPy, OpenCV]
    W --> O[ONNX Runtime Web<br/>YOLOv8 + U-Net]
    P <-->|async calls| O
Loading
  • The models were trained in PyTorch / Ultralytics and exported to ONNX (scripts/export_onnx.py). Weights are stored as float16 with a cast back to float32 in the graph: half the download (22 MB + 5 MB) with float32 accuracy, and it runs on any WebAssembly backend.
  • The same Python package (ecgmate/) runs server-side and in the browser. ecgmate/lite.py re-implements YOLO's letterbox preprocessing and box decoding in NumPy, so the browser needs neither PyTorch nor Ultralytics. The two networks are passed in as async functions that call ONNX Runtime Web from JavaScript.
  • Everything runs in a Web Worker, so the page stays responsive while it analyses.
  • Parity check: the ONNX/browser path and the PyTorch path scored the same on all 158 evaluation records (heart rate within 5 bpm: 73.4% vs 74.1%, measured before the tracing fixes below).

Results

Evaluated on 158 synthetic ECG images rendered from real PTB-XL recordings, most with realistic damage (creases, shadows, stains, tint, noise, handwriting). Because the true signal behind each image is known, every stage can be scored. The numbers below are for the exact code path the website runs:

Metric Result
Lead detection 2053 / 2054 leads (99.95%)
Digitized signal vs. true signal median correlation 0.88, median RMSE 0.08 mV
Heart rate median error 1.5 bpm; 75% within 5 bpm
QRS duration median error 8 ms; 95% within 20 ms
QT interval median error 10 ms; 68% within 20 ms
PR interval median error 12 ms; 74% within 20 ms

Interval errors compare measurements taken from the digitized signal with the same measurements taken from the true signal, which isolates the error introduced by reading the ECG off the image. Reproduce with:

python scripts/evaluate.py "path/to/images_with_wfdb_ground_truth" --limit 1000            # browser (ONNX) path
python scripts/evaluate.py "path/to/images_with_wfdb_ground_truth" --backend torch          # PyTorch path

Engineering notes

Things I found and fixed while turning the research prototype (four separate desktop apps) into this demo:

  • A model loading bug. The segmentation weights were trained on 1 input channel, but the original app built the network with 2 (signal + lead ID), so it could never load them.
  • An off-by-one in the tracing algorithm. Each point took its y-position from the previous column, and the peak refinement read the wrong pixel cluster. Both flattened sharp R/S peaks.
  • Overlapping windows now average. Segmentation used to let the last overlapping window overwrite earlier ones, and it never labelled the end of the signal.
  • A hybrid instead of all-deep-learning. Comparing against ground truth showed the U-Net missing most QRS complexes; on a clean test image it reported 43 bpm instead of 60. Anchoring QRS with a classical detector fixed that.
  • Heart rate by vote. Each lead votes with its own median RR interval, so one badly traced lead can't skew the result. On the same 40 test records, heart rate within 5 bpm went from 57% to 65%.
  • Robust to dirty images. Replacing a global threshold with a black-hat filter plus colour-based grid removal stopped stains and dark grid lines from being traced as signal.
  • Tracing fixes driven by failure analysis. Plotting traced vs. true signals lead by lead showed three distinct failures: the calibration-pulse trimmer mistaking flat T-wave tops for pulses and cutting real signal, the tracer dipping into printed lead labels ("V6", "aVF"), and faint crease shadows being traced as ink. Fixing each one, and measuring every change on the evaluation set, raised median signal correlation from 0.81 to 0.88 and QRS accuracy (within 20 ms) from 87% to 95%.
  • No manual calibration. The original workflow needed baseline/peak sliders for each lead; the scale now comes from the paper grid.
  • Clinical sanity. ST elevation uses guideline thresholds per lead group (0.2 mV in V1-V4, where some J-point elevation is normal). RR irregularity ignores intervals that are clearly missed or extra detections.

Limitations

  • Assumes the standard layout: 25 mm/s, 10 mm/mV, 3 × 4 leads of 2.5 s plus a lead II rhythm strip.
  • Crease shadows that run directly through the trace can still pull it off course; a learned trace-segmentation model (trainable on the synthetic images' pixel-level labels) is the natural next step.
  • Measured on synthetic images. Real phone photos with perspective distortion, or unusual printer layouts, will do worse.
  • On heavily degraded images, borderline PR/QTc findings can flip with small changes such as JPEG compression. Heart rate is stable.
  • Not assessed: bundle branch blocks, 2nd/3rd degree AV block, hypertrophy, AF/flutter, old MI, electrolyte abnormalities.
  • This is a portfolio/research project, not a medical device. Do not use it for diagnosis.

Run locally

The website is static, so any file server works:

python -m http.server 8000      # then open http://localhost:8000

For the Python tooling (Gradio app, evaluation, re-exporting the models):

python -m venv .venv
# Windows: .venv\Scripts\activate    macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
python app.py                   # Gradio UI using PyTorch, http://127.0.0.1:7860
import cv2
from ecgmate import ECGPipeline

result = ECGPipeline().run(cv2.imread("examples/1_clean_printout.png"))
print(result.summary["heart_rate_bpm"], result.findings)

Project structure

index.html, web/       static website: UI (app.js), Web Worker running Pyodide + ONNX Runtime (worker.js)
ecgmate/
  pipeline.py          shared orchestration + PyTorch front-end
  lite.py              ONNX front-end used in the browser
  digitize.py          grid removal, trace following, pixel -> mV calibration
  qrs.py               Pan-Tompkins-style QRS detection
  windowing.py         sliding windows for the U-Net
  segment.py           1D U-Net (PyTorch)
  analysis.py          measurements and rule-based interpretation
models/                ONNX models for the web, PyTorch originals
scripts/
  export_onnx.py       PyTorch -> ONNX (float16 weights)
  evaluate.py          accuracy evaluation against ground-truth signals
app.py                 optional local Gradio UI
examples/              synthetic example ECGs

Tech

Python · PyTorch · Ultralytics YOLOv8 · ONNX / ONNX Runtime Web · Pyodide (WebAssembly) · OpenCV · NumPy / SciPy · JavaScript (Web Workers, Canvas) · Vercel

Data

Example and evaluation images are synthetic ECGs rendered from the PTB-XL database (Wagner et al., 2020, CC BY 4.0). No real patient images are included.

About

Paper ECG photo → digital signal → interpretation, running entirely in the browser (YOLOv8, 1D U-Net, Pyodide, ONNX Runtime Web)

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages