Skip to content

Complete dpctl tensor removal#3091

Merged
ethanglaser merged 5 commits intouxlfoundation:mainfrom
ethanglaser:dev/eglaser-dpctl-tensor-removal
Apr 1, 2026
Merged

Complete dpctl tensor removal#3091
ethanglaser merged 5 commits intouxlfoundation:mainfrom
ethanglaser:dev/eglaser-dpctl-tensor-removal

Conversation

@ethanglaser
Copy link
Copy Markdown
Contributor

@ethanglaser ethanglaser commented Mar 31, 2026

Description

Carrying out entirety of #2764


Checklist:

Completeness and readability

  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with updates and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

@ethanglaser
Copy link
Copy Markdown
Contributor Author

/intelci: run

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 31, 2026

Codecov Report

❌ Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
onedal/datatypes/_data_conversion.py 50.00% 0 Missing and 1 partial ⚠️
onedal/datatypes/_sycl_usm.py 80.00% 1 Missing ⚠️
onedal/utils/_array_api.py 0.00% 0 Missing and 1 partial ⚠️
Flag Coverage Δ
azure 75.89% <82.35%> (?)
github 81.35% <82.35%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
onedal/_device_offload.py 81.72% <100.00%> (ø)
onedal/datatypes/__init__.py 100.00% <100.00%> (ø)
onedal/datatypes/_dlpack.py 69.56% <ø> (ø)
onedal/datatypes/tests/common.py 84.31% <100.00%> (-0.88%) ⬇️
onedal/utils/_third_party.py 88.46% <ø> (-0.43%) ⬇️
sklearnex/_device_offload.py 77.41% <100.00%> (ø)
sklearnex/covariance/incremental_covariance.py 79.41% <ø> (ø)
sklearnex/decomposition/pca.py 92.67% <ø> (ø)
sklearnex/dummy/_dummy.py 82.55% <ø> (ø)
sklearnex/neighbors/_lof.py 100.00% <ø> (ø)
... and 6 more

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ethanglaser
Copy link
Copy Markdown
Contributor Author

/intelci: run

@ethanglaser
Copy link
Copy Markdown
Contributor Author

CI Failure Triage Report

Run: 23814899308 | Date: 2026-03-31

Failed Jobs: LinuxNightly/venv Python3.14_Sklearn1.8

Analysis

CI Failure Analysis

1. Failure Classification: PR-specific

2. Root Cause:

The failure is caused by the removal of the copy_to_usm function and related dpctl tensor support code. The test is likely hitting a code path where copy_to_usm is still being called but no longer exists.

3. Evidence:

  • Similar failure pattern: The same test (test_ridge_regression_check_arguments_validity) failed on another branch with a TypeError: 'dict' object is not callable, suggesting this specific test is sensitive to array handling changes
  • Incomplete log: The provided log cuts off abruptly in the middle of test execution, which often indicates a crash or unhandled exception
  • Code removal impact: The PR removes significant functionality around dpctl tensor handling without updating all call sites

4. Relevant Code Changes:

The most problematic changes in the diff are:

# In onedal/_device_offload.py - Line 162-166
# BEFORE:
return (
    copy_to_dpnp(queue, result)
    if is_dpnp_ndarray(data)
    else copy_to_usm(queue, result)  # This function was removed!
)
# AFTER:
return copy_to_dpnp(queue, result)

And the complete removal of copy_to_usm from:

  • onedal/datatypes/__init__.py (removed from exports)
  • onedal/datatypes/_sycl_usm.py (function deleted)

5. Recommendation:

Immediate actions needed:

  1. Find remaining references: Search the entire codebase for any remaining calls to copy_to_usm() that weren't updated in this PR
  2. Update device offload logic: The simplified logic in _device_offload.py may be too aggressive - verify that copy_to_dpnp() can handle all the cases that copy_to_usm() previously handled
  3. Test with dpctl arrays: Ensure the removal of dpctl tensor support doesn't break existing functionality that still relies on it during the deprecation period
  4. Run local tests: Execute the failing test locally with the PR changes to reproduce and debug the specific error

Debugging steps:

# Run the specific failing test locally
pytest linear_model/tests/test_ridge.py::test_ridge_regression_check_arguments_validity -v

The issue likely stems from incomplete migration from the old copy_to_usm/copy_to_dpnp dual-path approach to the simplified dpnp-only approach.


Similar failures in recent runs
Job Source Date Matching Errors
LinuxNightly/venv Python3.14_Sklearn1.8 dev/eglaser-dpctl-tensor-removal 2026-03-31 TypeError: 'dict' object is not callable, FAILED linear_model/tests/test_ridge.py::test_ridg

Generated by CI Triage Bot

@david-cortes-intel
Copy link
Copy Markdown
Contributor

@ethanglaser You also need to remove here:

- GPU: dpnp/dpctl arrays with newton-cg solver

Copy link
Copy Markdown
Contributor

@david-cortes-intel david-cortes-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, pending CI results.

@ethanglaser
Copy link
Copy Markdown
Contributor Author

/intelci: run

@ethanglaser ethanglaser marked this pull request as ready for review April 1, 2026 14:53
Copilot AI review requested due to automatic review settings April 1, 2026 14:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes the removal of dpctl.tensor support across sklearnex and onedal, aligning tests, utilities, and data conversion/offload paths to use dpnp (and/or Array API) instead.

Changes:

  • Remove dpctl.tensor handling from array-namespace detection and table/data conversion utilities.
  • Update GPU/SPMD and validation test matrices to stop parametrizing dpctl tensor inputs and focus on dpnp + Array API.
  • Simplify SYCL output re-wrapping paths to return dpnp outputs for SYCL-USM inputs.

Reviewed changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
sklearnex/utils/tests/test_validation.py Update supported dataframe list and comments to replace dpctl with dpnp.
sklearnex/utils/tests/test_class_weight.py Remove dpctl from array API test parametrization.
sklearnex/utils/_array_api.py Update comments/docstring wording to remove dpctl references.
sklearnex/tests/test_patching.py Remove dpctl branches/skips and update comments for dpnp-only testing.
sklearnex/tests/test_common.py Treat dpnp as an actively-loaded framework in lazy-import checks.
sklearnex/spmd/neighbors/tests/test_neighbors_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/linear_model/tests/test_logistic_regression_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/linear_model/tests/test_linear_regression_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/linear_model/tests/test_incremental_linear_spmd.py Drop dpctl from GPU SPMD dataframe filters and update comments.
sklearnex/spmd/ensemble/tests/test_forest_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/decomposition/tests/test_pca_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/decomposition/tests/test_incremental_pca_spmd.py Drop dpctl from GPU SPMD dataframe filters and update comments.
sklearnex/spmd/covariance/tests/test_incremental_covariance_spmd.py Drop dpctl from GPU SPMD dataframe filters and update comments.
sklearnex/spmd/covariance/tests/test_covariance_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/cluster/tests/test_kmeans_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/cluster/tests/test_dbscan_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/spmd/basic_statistics/tests/test_incremental_basic_statistics_spmd.py Drop dpctl from GPU SPMD dataframe filters and update comments.
sklearnex/spmd/basic_statistics/tests/test_basic_statistics_spmd.py Drop dpctl from GPU SPMD dataframe filters.
sklearnex/preview/decomposition/tests/test_incremental_pca.py Remove dpctl from dataframe parametrization.
sklearnex/preview/covariance/covariance.py Update comments referring to dpnp/dpctl behavior to dpnp-only.
sklearnex/neighbors/knn_classification.py Update comment examples to remove dpctl.
sklearnex/neighbors/common.py Update comment about device-aware allocation to remove dpctl.
sklearnex/neighbors/_lof.py Update comment about queue retention to remove dpctl.
sklearnex/manifold/tests/test_tsne.py Update comment about numpy conversion to remove dpctl.tensor.
sklearnex/linear_model/tests/test_logreg.py Update test docstring to remove dpctl mention.
sklearnex/dummy/tests/test_dummy.py Remove dpctl from dataframe parametrization.
sklearnex/dummy/_dummy.py Update module-level documentation comments to remove dpctl references.
sklearnex/decomposition/tests/test_pca.py Remove dpctl from dataframe parametrization.
sklearnex/decomposition/pca.py Update comment to remove dpctl mention.
sklearnex/covariance/incremental_covariance.py Update comments referring to dpnp/dpctl behavior to dpnp-only.
sklearnex/_device_offload.py Remove copy_to_usm usage and always rewrap SYCL outputs via copy_to_dpnp.
onedal/utils/tests/test_validation.py Remove dpctl from dataframe parametrization in validation tests.
onedal/utils/_third_party.py Remove is_dpctl_tensor helper and update dpctl-related comments.
onedal/utils/_array_api.py Remove dpctl.tensor namespace mapping; SYCL namespace mapping now dpnp-only.
onedal/tests/utils/_dataframes_support.py Remove dpctl dataframe support and default framework list entry.
onedal/datatypes/tests/test_data.py Remove dpctl from dataframe parametrization; switch dlpack test to dpnp.
onedal/datatypes/tests/common.py Update USM attribute assertion helper to operate on dpnp arrays only.
onedal/datatypes/_sycl_usm.py Remove dpctl-based USM allocation path; implement device copy via dpnp.asarray.
onedal/datatypes/_dlpack.py Update comment to remove dpctl reference.
onedal/datatypes/_data_conversion.py Remove dpctl tensor return-type construction path and warnings.
onedal/datatypes/init.py Stop exporting copy_to_usm; keep copy_to_dpnp.
onedal/_device_offload.py Remove copy_to_usm usage and always rewrap SYCL outputs via copy_to_dpnp.

Comment thread onedal/datatypes/_data_conversion.py
Comment thread sklearnex/tests/test_patching.py
@ethanglaser ethanglaser merged commit 31a814e into uxlfoundation:main Apr 1, 2026
23 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants