modify some script and test and add build_llvm script by XingerZhu · Pull Request #4 · ROCm/FlyDSL

XingerZhu · 2025-12-01T10:26:34Z

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Copilot

Pull request overview

This PR modernizes the rocDSL test infrastructure by standardizing path setup across test files, improving GPU compilation pipelines, and adding a new LLVM build script. The changes remove hardcoded developer-specific paths in favor of dynamic project root detection, update MLIR transform APIs to use newer functions with folding support, and implement workarounds for MLIR Python binding issues. Additionally, a new build_llvm.sh script simplifies the LLVM/MLIR build process for new contributors.

Key changes include:

Standardized path setup pattern across 14 Python test files for portability
Enhanced GPU compilation pipelines with explicit lowering pass ordering and reconciliation
New build_llvm.sh script for automated LLVM/MLIR building
C++ API updates to use applyPatternsAndFoldGreedily in transform passes
Operation.create() workarounds for MLIR Python binding recursion bugs
Polyfills for missing ROCDL operations in Python bindings

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
tests/python/ir/test_rocir_coord_ops.py	Updated to use standardized project root detection for sys.path setup
tests/python/ir/test_product_divide.py	Updated to use standardized project root detection for sys.path setup
tests/python/ir/test_local_ops.py	Updated to use standardized project root detection for sys.path setup
tests/python/ir/test_basic_ops.py	Updated to use standardized project root detection for sys.path setup
tests/python/gpu/utils.py	Expanded GPU compilation pipeline with explicit control flow lowering passes before GPU-to-ROCDL conversion
tests/python/gpu/test_shared_working.py	Updated to use standardized project root detection for sys.path setup
tests/python/gpu/test_mfma_simple_working.py	Updated to use standardized project root detection for sys.path setup
tests/python/gpu/test_mfma_gemm_real_python_api.py	Added Operation.create() workaround for MFMA operations, enhanced compilation pipeline, added gpu.ModuleEndOp() terminator
tests/python/gpu/test_mfma_gemm_fp8_rocir.py	Added Operation.create() workaround for FP8 MFMA operations, enhanced compilation pipeline with additional lowering passes, added gpu.ModuleEndOp() terminator
tests/python/gpu/test_mfma_gemm_fp8_python_api.py	Added Operation.create() workaround for FP8 MFMA operations, enhanced compilation pipeline, added gpu.ModuleEndOp() terminator
tests/python/gpu/test_gpu_with_rocir_coords.py	Updated to use standardized project root detection for sys.path setup
tests/python/gpu/test_gpu_simple.py	Updated to use standardized project root detection for sys.path setup, added missing os import
tests/python/gpu/test_gpu_rocdsl.py	Updated to use standardized project root detection for sys.path setup
tests/python/gpu/test_gpu_layout.py	Updated to use standardized project root detection for sys.path setup
tests/python/examples/test_rocdl_ops.py	Updated to use standardized project root detection, added comments about potentially missing ROCDL operations
tests/python/conftest.py	Updated library paths to use dynamic project root detection instead of hardcoded absolute paths
python/rocdsl/rocir_opt_runner.py	Updated rocir-opt path to use dynamic project root detection
python/rocdsl/dialects/ext/rocir.py	Refactored operations to use Operation.create() API for crd2idx, size, cosize, and rank operations
python/rocdsl/dialects/ext/rocdl.py	Added polyfills for missing ROCDL operations (wavefrontsize, s_waitcnt, readlane, readfirstlane)
python/rocdsl/dialects/ext/gpu.py	Added automatic ModuleEndOp terminator to gpu.module region_op
python/rocdsl/compiler/pipeline.py	Updated rocir-opt path to use dynamic project root detection
lib/Transforms/RocirToStandard.cpp	Updated to use applyPatternsAndFoldGreedily instead of applyPatternsGreedily
lib/Transforms/RocirCoordLowering.cpp	Updated to use applyPatternsAndFoldGreedily instead of applyPatternsGreedily
build_llvm.sh	New script to automate LLVM/MLIR build process with release/19.x branch
README.md	Updated build instructions and test paths to reflect project structure changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-01T10:34:16Z

+                            loc=loc
+                        )
+                        # Insert at current insertion point (inside the loop)
+                        # ir.InsertionPoint.current.insert(op)


The commented-out lines # ir.InsertionPoint.current.insert(op) are present but unnecessary. When using Operation.create() within an active InsertionPoint context, the operation is automatically inserted. These commented lines should be removed to clean up the code.

Suggested change

# ir.InsertionPoint.current.insert(op)

Copilot · 2025-12-01T10:34:16Z

+    def s_waitcnt(simm16):
+        if isinstance(simm16, int):
+            simm16 = arith.ConstantOp(IntegerType.get_signless(32), simm16).result
+        return Operation.create("rocdl.waitcnt", operands=[simm16]).results


The s_waitcnt polyfill returns .results (a list), while other polyfills like wavefrontsize, readlane, and readfirstlane return .results[0] (a single result). This inconsistency may cause issues if the function is expected to return a single result value. Consider returning .results[0] for consistency, or document why it returns a list.

Suggested change

return Operation.create("rocdl.waitcnt", operands=[simm16]).results

return Operation.create("rocdl.waitcnt", operands=[simm16]).results[0]

Copilot · 2025-12-01T10:34:16Z

+# Or manually:
 cd rocdsl
 mkdir -p build && cd build
 cmake .. -DMLIR_DIR=$MLIR_PATH//lib/cmake/mlir


The path contains a double slash // which is incorrect. Should be -DMLIR_DIR=$MLIR_PATH/lib/cmake/mlir with only a single slash separator.

Suggested change

cmake .. -DMLIR_DIR=$MLIR_PATH//lib/cmake/mlir

cmake .. -DMLIR_DIR=$MLIR_PATH/lib/cmake/mlir

Copilot · 2025-12-01T10:34:17Z


-    with ip or InsertionPoint.current:
-        op = rocir_ops.Crd2IdxOp(_unwrap_value(coord), _unwrap_value(layout), results=[result_type], loc=loc, ip=ip)
+    with ip or InsertionPoint.current as insertion_point:


The insertion_point variable captured from the with statement is unused. Consider either removing the variable name (use just with ip or InsertionPoint.current:), or document why it's captured if there's a specific reason.

Suggested change

with ip or InsertionPoint.current as insertion_point:

with ip or InsertionPoint.current:

Copilot · 2025-12-01T10:34:17Z


-    with ip or InsertionPoint.current:
-        op = rocir_ops.CosizeOp(_unwrap_value(layout), results=[result_type], loc=loc, ip=ip)
+    with ip or InsertionPoint.current as insertion_point:


The insertion_point variable captured from the with statement is unused. Consider either removing the variable name (use just with ip or InsertionPoint.current:), or document why it's captured if there's a specific reason.

Copilot · 2025-12-01T10:34:17Z

+            operands=[_unwrap_value(coord), _unwrap_value(layout)],
+            loc=loc
+        )
+        # insertion_point.insert(op)


The commented-out line insertion_point.insert(op) suggests uncertainty about whether manual insertion is needed. When using Operation.create() within an active InsertionPoint context (from the with statement), the operation is automatically inserted. The comment should be removed for clarity, or if there was a specific reason to keep it commented, that reason should be documented.

Suggested change

# insertion_point.insert(op)

Copilot · 2025-12-01T10:34:18Z


-    with ip or InsertionPoint.current:
-        op = rocir_ops.SizeOp(_unwrap_value(shape_or_layout), results=[result_type], loc=loc, ip=ip)
+    with ip or InsertionPoint.current as insertion_point:


The insertion_point variable captured from the with statement is unused. Consider either removing the variable name (use just with ip or InsertionPoint.current:), or document why it's captured if there's a specific reason.

Copilot · 2025-12-01T10:34:18Z


-    with ip or InsertionPoint.current:
-        op = rocir_ops.RankOp(_unwrap_value(shape_or_layout), results=[result_type], loc=loc, ip=ip)
+    with ip or InsertionPoint.current as insertion_point:


The insertion_point variable captured from the with statement is unused. Consider either removing the variable name (use just with ip or InsertionPoint.current:), or document why it's captured if there's a specific reason.

Copilot · 2025-12-01T10:34:18Z

+                        loc=loc
+                    )
+                    # Insert at current insertion point
+                    # ir.InsertionPoint.current.insert(op)


The commented-out lines ir.InsertionPoint.current.insert(op) are present but unnecessary. When using Operation.create() within an active InsertionPoint context, the operation is automatically inserted. These commented lines should be removed to clean up the code.

Suggested change

# ir.InsertionPoint.current.insert(op)

Copilot · 2025-12-01T10:34:18Z

+                            loc=loc
+                        )
+                        # Insert at current insertion point (inside the loop)
+                        # ir.InsertionPoint.current.insert(op)


The commented-out lines # ir.InsertionPoint.current.insert(op) are present but unnecessary. When using Operation.create() within an active InsertionPoint context, the operation is automatically inserted. These commented lines should be removed to clean up the code.

Suggested change

# ir.InsertionPoint.current.insert(op)

…ve (lever exhausted) First profile-directed secondary lever (DEC-11b #4), targeting the stage2 LDS-wait bottleneck identified by the R6-R9 profiling. Swept stage2 tile_n2 in {128,512,1024} vs the 256 baseline at DS V3 a4w4 tokens 32/64/16384/32768 (pinned+idle, reps=3, fail-closed CLI, kernel-path metric; MFU derived). Result: NO tile_n2 wins. n2=128 regresses (kp +5..+7%, mfu -3..-6%); n2=1024 regresses hard (t32 kp +15.8%, 16384 mfu -8.4%); n2=512 is baseline-neutral (-1.2%..+1.0%, within the DEC-9 band). The baseline tile_n2=256 is already near-optimal for this lever; tile_k2 is not a lever for DS V3 (inter_dim=256 pins it to 256). The stage2 LDS-wait bottleneck needs a DIFFERENT lever. NEXT: persist_m / xcd_swizzle, which require threading those knobs through moe_tuning_harness + legality first. Honest loss recorded with full provenance; all 4 ledger scans clean. 109 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Replace the last raw-dialect sites in the moe_gemm_2stage package with the current fx.* API. Zero codegen change (ISA byte-identical for fp8/int8/int4 gemm1, fp8/int8/int4 gemm2 f16-atomic, and gfx942 bf16-atomic compile). - gemm1/gemm2 launch wrappers: arith.index_cast(T.index, ...) -> fx.Int64(...) for the host-side grid math (matches moe_reduce.py). - gemm2 scf.for induction cast: arith.index_cast(T.i32, iv) -> fx.Int32(iv). - gemm2 atomic buffer resource: buffer_ops.create_buffer_resource(...) -> fx.rocdl.get_buffer_rsrc(fx.get_iter(fx.rocdl.make_buffer_tensor(...))). - layout_helpers atomic guards: scf.IfOp + _if_then -> plain Python `if` (the transformed _TensorWithIndex.copy traces both into the same scf.if; every atomic stays inside its predicate, BUG GUARD #4). - Drop now-dead imports (arith, scf, T, buffer_ops, _if_then). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

modify some script and test and add build_llvm script

4a1fb36

XingerZhu requested a review from Copilot December 1, 2025 10:26

Copilot started reviewing on behalf of XingerZhu December 1, 2025 10:27

View session

XingerZhu requested a review from coderfeli December 1, 2025 10:27

Copilot finished reviewing on behalf of XingerZhu December 1, 2025 10:32

Copilot AI reviewed Dec 1, 2025

View reviewed changes

XingerZhu closed this Dec 2, 2025

Navigation Menu

Repository navigation

modify some script and test and add build_llvm script - #4

modify some script and test and add build_llvm script#4
XingerZhu wants to merge 1 commit into
mainfrom
zxe_dev

XingerZhu commented Dec 1, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Copilot AI Dec 1, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	return Operation.create("rocdl.waitcnt", operands=[simm16]).results
	return Operation.create("rocdl.waitcnt", operands=[simm16]).results[0]

	cmake .. -DMLIR_DIR=$MLIR_PATH//lib/cmake/mlir
	cmake .. -DMLIR_DIR=$MLIR_PATH/lib/cmake/mlir

	with ip or InsertionPoint.current as insertion_point:
	with ip or InsertionPoint.current:

Conversation

XingerZhu commented Dec 1, 2025

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants