Skip to content

Commit aa3b4bd

Browse files
ArturSkowronskiArturSkowronski
committed
feat: add TornadoVM 3.0 and 4.0 demos with version comparison CI
Add alternative TornadoVM demos targeting newer versions alongside the existing 2.2.0 demo, plus a GitHub Actions workflow that benchmarks all three versions on VectorAdd and GPULlama3 LLM inference. TornadoVM 3.0.0 (JDK 25, OpenCL, Metal on macOS) TornadoVM 4.0.0 (JDK 21, OpenCL/Metal, CUDA Graphs) CI uses Intel oneAPI OpenCL CPU runtime to run GPU workloads on standard GitHub Actions runners (same approach as llama3-java-hat).
1 parent 4d1c5d1 commit aa3b4bd

20 files changed

Lines changed: 1989 additions & 0 deletions

File tree

‎.github/workflows/tornadovm-comparison.yml‎

Lines changed: 452 additions & 0 deletions
Large diffs are not rendered by default.

‎demos/tornadovm-3/README.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# TornadoVM 3.0 Demo - GPU Acceleration with JDK 25 & LLM Inference
2+
3+
Alternative version of the TornadoVM demo targeting **TornadoVM 3.0.0-jdk25** with:
4+
- **JDK 25** requirement (first TornadoVM release targeting JDK 25)
5+
- **Metal backend** for native Apple Silicon acceleration
6+
- **GPULlama3.java** for GPU-accelerated LLM inference
7+
- **Auto-detection** of best backend (Metal/OpenCL/PTX)
8+
9+
## Quick Start
10+
11+
### Baseline (CPU) - Works Everywhere
12+
13+
```bash
14+
./gradlew :demos:tornadovm-3:run
15+
```
16+
17+
### TornadoVM 3.0 (GPU) - Requires JDK 25
18+
19+
```bash
20+
cd demos/tornadovm-3
21+
./scripts/run-tornado.sh --size 10000000 --iters 5
22+
```
23+
24+
### GPU LLM Inference (GPULlama3.java)
25+
26+
```bash
27+
cd demos/tornadovm-3
28+
./scripts/run-gpullama3.sh --model ~/.llama/models/Llama-3.2-1B-Instruct-f16.gguf --prompt "Hello"
29+
```
30+
31+
## Version Comparison
32+
33+
| Feature | TornadoVM 2.2.0 | TornadoVM 3.0.0 | TornadoVM 4.0.0 |
34+
|---------|-----------------|-----------------|-----------------|
35+
| **JDK** | 21 (GraalVM CE) | **25** | 21, 25 |
36+
| **Backends** | OpenCL, PTX, SPIR-V | OpenCL, PTX, SPIR-V, **Metal** | OpenCL, PTX, SPIR-V, Metal |
37+
| **macOS Apple Silicon** | OpenCL (deprecated) | **Metal (native)** | Metal (native) |
38+
| **Download format** | .tar.gz | .zip | .zip |
39+
| **ImmutableTaskGraph** | Yes | Yes (enhanced) | Yes (enhanced) |
40+
| **CUDA Graphs** | No | No | `withCUDAGraph()` |
41+
42+
## What This Demo Shows
43+
44+
**VectorAddBaseline** (Gradle task):
45+
- Simple vector addition on CPU
46+
- Baseline performance measurement
47+
- Works on any JDK 21+
48+
49+
**VectorAddTornado** (script-based):
50+
- Same operation on GPU via TornadoVM 3.0
51+
- TaskGraph API for GPU offload
52+
- @Parallel annotation for parallelization
53+
- Auto-detects Metal on macOS Apple Silicon
54+
55+
**GPULlama3** (script-based):
56+
- Full LLM inference on GPU
57+
- Uses GPULlama3.java with TornadoVM 3.0
58+
- Supports Llama 3.2, Qwen, Phi3, Granite models
59+
60+
## Requirements
61+
62+
**Baseline demo:**
63+
- JDK 21+ (any distribution)
64+
65+
**TornadoVM 3.0 GPU demo:**
66+
- **JDK 25** (required by TornadoVM 3.0)
67+
- TornadoVM 3.0 SDK (auto-downloaded by script)
68+
- Metal (macOS) or OpenCL/CUDA drivers (Linux)
69+
70+
**GPU LLM inference:**
71+
- All of the above
72+
- Model file: `~/.llama/models/Llama-3.2-1B-Instruct-f16.gguf` (~2.5 GB)
73+
74+
## Running
75+
76+
```bash
77+
# CPU baseline (Gradle)
78+
./gradlew :demos:tornadovm-3:run
79+
./gradlew :demos:tornadovm-3:runtimeCheck
80+
81+
# GPU version (script - auto-downloads TornadoVM 3.0)
82+
cd demos/tornadovm-3
83+
./scripts/run-tornado.sh --size 10000000 --iters 5
84+
85+
# Force specific backend
86+
TORNADOVM_BACKEND=metal ./scripts/run-tornado.sh
87+
TORNADOVM_BACKEND=opencl ./scripts/run-tornado.sh
88+
TORNADOVM_BACKEND=ptx ./scripts/run-tornado.sh
89+
90+
# GPU LLM inference
91+
./scripts/run-gpullama3.sh --model ~/.llama/models/Llama-3.2-1B-Instruct-f16.gguf --prompt "Hello"
92+
```
93+
94+
## Code Structure
95+
96+
```
97+
demos/tornadovm-3/
98+
├── src/
99+
│ ├── main/java/com/skowronski/talk/jvmai/
100+
│ │ └── VectorAddBaseline.java # CPU version (Gradle)
101+
│ └── tornado/java/demo/tornadovm/
102+
│ └── VectorAddTornado.java # GPU version (script)
103+
├── scripts/
104+
│ ├── run-baseline.sh # CPU demo
105+
│ ├── run-tornado.sh # GPU demo (TornadoVM 3.0, auto-downloads)
106+
│ └── run-gpullama3.sh # GPU LLM (TornadoVM 3.0)
107+
├── build.gradle.kts # Gradle build (baseline only)
108+
└── README.md # This file
109+
```
110+
111+
## See Also
112+
113+
- **`demos/tornadovm/`** - Original TornadoVM 2.2.0 demo (JDK 21)
114+
- **`demos/tornadovm-4/`** - TornadoVM 4.0.0 demo (JDK 21/25, CUDA Graphs)
115+
- **`demos/llama3-java/`** - Pure Java LLM (CPU, Vector API)
116+
- **`demos/java-llama-cpp/`** - JNI LLM (~50 tok/s)
117+
- **`demos/babylon/`** - Future GPU approach (Code Reflection)

‎demos/tornadovm-3/build.gradle.kts‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
plugins {
2+
application
3+
java
4+
}
5+
6+
repositories {
7+
mavenCentral()
8+
}
9+
10+
// Note: TornadoVM 3.0 requires JDK 25
11+
// This build provides baseline (CPU) demo only
12+
// For TornadoVM GPU demo, use scripts/run-tornado.sh
13+
// For GPU LLM inference, use scripts/run-gpullama3.sh
14+
15+
dependencies {
16+
// TornadoVM dependencies would go here if running GPU version
17+
// For baseline demo, no dependencies needed
18+
}
19+
20+
application {
21+
mainClass.set("com.skowronski.talk.jvmai.VectorAddBaseline")
22+
}
23+
24+
// Runtime check demo - runs on any JDK (CPU only)
25+
tasks.register<JavaExec>("runtimeCheck") {
26+
group = "application"
27+
description = "Run baseline vector add (CPU, no TornadoVM)"
28+
29+
classpath = sourceSets.main.get().runtimeClasspath
30+
mainClass.set("com.skowronski.talk.jvmai.VectorAddBaseline")
31+
args = listOf("--size", "10000000", "--iters", "5")
32+
}
33+
34+
// Alias 'run' to 'runtimeCheck'
35+
tasks.named<JavaExec>("run") {
36+
args = listOf("--size", "10000000", "--iters", "5")
37+
}
38+
39+
// Note: TornadoVM 3.0 GPU demo requires:
40+
// 1. TornadoVM 3.0.0-jdk25 SDK installed
41+
// 2. OpenCL/Metal/CUDA drivers
42+
// 3. Use: ./scripts/run-tornado.sh
43+
// 4. Or: ./scripts/run-gpullama3.sh for LLM demo

‎demos/tornadovm-3/scripts/run-baseline.sh‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
ROOT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
5+
BUILD_DIR="$ROOT_DIR/build/baseline"
6+
SRC_DIR="$ROOT_DIR/src/baseline/java"
7+
8+
JAVA_BIN="${JAVA_HOME:-}/bin/java"
9+
JAVAC_BIN="${JAVA_HOME:-}/bin/javac"
10+
if [[ ! -x "$JAVAC_BIN" ]]; then
11+
JAVA_BIN="java"
12+
JAVAC_BIN="javac"
13+
fi
14+
15+
rm -rf "$BUILD_DIR"
16+
mkdir -p "$BUILD_DIR"
17+
18+
find "$SRC_DIR" -name "*.java" > "$BUILD_DIR/sources.txt"
19+
"$JAVAC_BIN" -d "$BUILD_DIR" @"$BUILD_DIR/sources.txt"
20+
21+
exec "$JAVA_BIN" -cp "$BUILD_DIR" demo.baseline.VectorAddBaseline "$@"

‎demos/tornadovm-3/scripts/run-gpullama3.sh‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# GPULlama3.java Runner for TornadoVM 3.0
5+
# GPU-accelerated LLM inference using TornadoVM 3.0 (JDK 25) with Metal/OpenCL/PTX backends
6+
7+
ROOT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
8+
SRC_DIR="$ROOT_DIR/build/gpullama3-src"
9+
REPO_URL="https://github.com/beehive-lab/GPULlama3.java.git"
10+
11+
usage() {
12+
cat <<'EOF'
13+
Usage:
14+
./scripts/run-gpullama3.sh --model /path/to/model.gguf --prompt "tell me a joke" [extra args...]
15+
16+
Notes:
17+
- Requires TornadoVM 3.0 (JDK 25) installed.
18+
- Set TORNADOVM_HOME to your TornadoVM 3.0 installation directory.
19+
- Model must be in FP16 format (Q4_K_M and other quantized formats not supported).
20+
- The first run clones + builds https://github.com/beehive-lab/GPULlama3.java into demos/tornadovm-3/build/.
21+
- On macOS Apple Silicon, Metal backend is auto-detected.
22+
23+
Examples:
24+
# Using auto-downloaded SDK (run-tornado.sh downloads it to build/tornadovm-sdk/):
25+
export TORNADOVM_HOME=./demos/tornadovm-3/build/tornadovm-sdk/tornadovm-3.0.0-jdk25-metal
26+
./scripts/run-gpullama3.sh --model ~/.llama/models/Llama-3.2-1B-Instruct-f16.gguf --prompt "say hello" --heap-max 6g
27+
EOF
28+
}
29+
30+
if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
31+
usage
32+
exit 0
33+
fi
34+
35+
# Support legacy TORNADO_SDK env var
36+
if [[ -n "${TORNADO_SDK:-}" && -z "${TORNADOVM_HOME:-}" ]]; then
37+
export TORNADOVM_HOME="$TORNADO_SDK"
38+
fi
39+
40+
# Auto-setup TornadoVM 3.0 if not set
41+
if [[ -z "${TORNADOVM_HOME:-}" ]]; then
42+
echo "TORNADOVM_HOME not set. Attempting auto-setup..."
43+
44+
export TORNADOVM_VERSION="${TORNADOVM_VERSION:-3.0.0}"
45+
export TORNADOVM_JDK="${TORNADOVM_JDK:-jdk25}"
46+
47+
detect_platform() {
48+
local os arch
49+
case "$(uname -s)" in
50+
Linux*) os="linux" ;;
51+
Darwin*) os="mac" ;;
52+
*) echo "Unsupported OS: $(uname -s)" >&2; exit 1 ;;
53+
esac
54+
case "$(uname -m)" in
55+
x86_64|amd64) arch="amd64" ;;
56+
arm64|aarch64) arch="aarch64" ;;
57+
*) echo "Unsupported architecture: $(uname -m)" >&2; exit 1 ;;
58+
esac
59+
echo "${os}-${arch}"
60+
}
61+
62+
# TornadoVM 3.0 only has opencl on macOS (no Metal)
63+
backend="opencl"
64+
sdk_dir="$ROOT_DIR/build/tornadovm-sdk"
65+
sdk_path="$sdk_dir/tornadovm-${TORNADOVM_VERSION}-${TORNADOVM_JDK}-${backend}"
66+
alt_sdk_path="$sdk_dir/tornadovm-${TORNADOVM_VERSION}-${backend}"
67+
68+
if [[ -d "$sdk_path" ]]; then
69+
export TORNADOVM_HOME="$sdk_path"
70+
echo "Using cached TornadoVM 3.0 SDK: $TORNADOVM_HOME"
71+
elif [[ -d "$alt_sdk_path" ]]; then
72+
export TORNADOVM_HOME="$alt_sdk_path"
73+
echo "Using cached TornadoVM 3.0 SDK: $TORNADOVM_HOME"
74+
else
75+
echo "TornadoVM 3.0 SDK not found. Run ./scripts/run-tornado.sh first to download it." >&2
76+
echo "Or set TORNADOVM_HOME manually." >&2
77+
exit 2
78+
fi
79+
fi
80+
81+
# Use TORNADOVM_HOME as JAVA_HOME if not set (for bundled JDK distributions)
82+
if [[ -z "${JAVA_HOME:-}" ]]; then
83+
export JAVA_HOME="$TORNADOVM_HOME"
84+
fi
85+
86+
# Workaround for JVMCI compatibility issues between JDK versions
87+
export JVMCI_CONFIG_CHECK="${JVMCI_CONFIG_CHECK:-ignore}"
88+
89+
# Device selection for multi-backend builds: 0:0=OpenCL/Metal, 1:0=PTX/CUDA
90+
if [[ -n "${TORNADO_DEVICE:-}" ]]; then
91+
export JAVA_OPTS="${JAVA_OPTS:-} -Dtornado.device=$TORNADO_DEVICE"
92+
echo "Using TornadoVM device: $TORNADO_DEVICE"
93+
fi
94+
95+
if [[ ! -x "$JAVA_HOME/bin/java" ]]; then
96+
echo "JAVA_HOME does not look like a JDK: $JAVA_HOME" >&2
97+
exit 2
98+
fi
99+
100+
if [[ ! -d "$SRC_DIR/.git" ]]; then
101+
rm -rf "$SRC_DIR"
102+
mkdir -p "$(dirname -- "$SRC_DIR")"
103+
git clone --depth 1 "$REPO_URL" "$SRC_DIR"
104+
fi
105+
106+
(
107+
cd "$SRC_DIR"
108+
./mvnw -q -DskipTests package
109+
)
110+
111+
export LLAMA_ROOT="$SRC_DIR"
112+
113+
DEFAULT_ARGS=()
114+
115+
has_flag() {
116+
local flag="$1"
117+
shift
118+
for arg in "$@"; do
119+
if [[ "$arg" == "$flag" ]]; then
120+
return 0
121+
fi
122+
done
123+
return 1
124+
}
125+
126+
if ! has_flag --gpu "$@"; then
127+
DEFAULT_ARGS+=(--gpu)
128+
fi
129+
if ! has_flag --heap-min "$@"; then
130+
DEFAULT_ARGS+=(--heap-min "${GPULLAMA3_HEAP_MIN:-4g}")
131+
fi
132+
if ! has_flag --heap-max "$@"; then
133+
DEFAULT_ARGS+=(--heap-max "${GPULLAMA3_HEAP_MAX:-4g}")
134+
fi
135+
if ! has_flag --gpu-memory "$@"; then
136+
DEFAULT_ARGS+=(--gpu-memory "${GPULLAMA3_GPU_MEMORY:-7GB}")
137+
fi
138+
139+
exec "$SRC_DIR/llama-tornado" "${DEFAULT_ARGS[@]}" "$@"

0 commit comments

Comments
 (0)