introcuding copilot isntructions

Author: napetrov

.github/instructions/build-config.instructions.md

@@ -0,0 +1,81 @@
+# Build Configuration Files
+
+## Core Build Files
+- `setup.py`: Main build script (500+ lines, complex configuration)
+- `pyproject.toml`: Python project metadata + linting configuration
+- `dependencies-dev`: Build-time dependencies (Cython, numpy, pybind11, cmake)
+- `requirements-test.txt`: Test dependencies with version constraints
+- `conda-recipe/meta.yaml`: Conda package build configuration
+
+## Environment Variables (Critical)
+```bash
+# MANDATORY for building
+export DALROOT=/path/to/onedal               # oneDAL installation path (required)
+
+# OPTIONAL but commonly needed
+export MPIROOT=/path/to/mpi                  # MPI for distributed features
+export NO_DIST=1                             # Disable distributed mode
+export NO_DPC=1                              # Disable GPU/SYCL support
+export NO_STREAM=1                           # Disable streaming mode
+export DEBUG_BUILD=1                         # Debug symbols + no optimization
+export MAKEFLAGS=-j$(nproc)                  # Parallel build threads
+```
+
+## Build Process (4 Stages)
+1. **Code Generation**: oneDAL C++ headers → Python/Cython sources
+2. **oneDAL Bindings**: cmake + pybind11 compilation
+3. **Cython Processing**: .pyx files → C++ sources
+4. **Final Compilation**: Link everything into Python extensions
+
+## Dependencies
+**Build Dependencies (dependencies-dev):**
+- Cython==3.1.1 (exact version required)
+- numpy>=2.0 (version varies by Python version)
+- pybind11==2.13.6
+- cmake==4.0.2
+- setuptools==79.0.1
+
+**Runtime Dependencies:**
+- Intel oneDAL 2021.1+ (backwards compatible)
+- numpy (version-specific, see requirements-test.txt)
+- scikit-learn 1.0-1.7 (see compatibility matrix)
+
+## Build Commands
+```bash
+# Development build (RECOMMENDED)
+python setup.py develop                       # Creates .egg-link, editable
+
+# Production builds
+python setup.py install                       # Full install
+python setup.py build_ext --inplace --force   # Extensions only
+
+# Special flags (Linux)
+python setup.py build --abs-rpath             # Absolute RPATH for custom oneDAL
+
+# Conda build
+conda build .                                 # Uses conda-recipe/meta.yaml
+```
+
+## Common Build Issues
+```bash
+# oneDAL not found
+RuntimeError: "Not set DALROOT variable"
+→ Solution: export DALROOT=/path/to/onedal
+
+# MPI required but missing
+ValueError: "'MPIROOT' is not set, cannot build with distributed mode"
+→ Solution: export NO_DIST=1 or set MPIROOT
+
+# Cython version mismatch
+→ Solution: pip install Cython==3.1.1 (exact version)
+
+# Linking issues (Linux)
+→ Solution: Use --abs-rpath flag
+```
+
+## CI/CD Configuration
+- **GitHub Actions**: `.github/workflows/ci.yml`
+- **Azure DevOps**: `.ci/pipeline/ci.yml` (main CI system)
+- **Pre-commit**: `.pre-commit-config.yaml` (code quality)
+
+Build timeouts: 120 minutes in CI (can be slow due to oneDAL compilation)

.github/instructions/daal4py.instructions.md

@@ -0,0 +1,70 @@
+# daal4py/* - Direct oneDAL Python Bindings
+
+## Purpose
+Direct Python bindings to Intel oneDAL for maximum performance and model builders for XGBoost/LightGBM conversion.
+
+## Three Sub-APIs
+1. **Native oneDAL**: `import daal4py as d4p` - Direct algorithm access
+2. **sklearn-compatible**: `from daal4py.sklearn import ...` - sklearn API with oneDAL backend
+3. **Model Builders**: `from daal4py.mb import convert_model` - External model conversion
+
+## Native oneDAL Pattern
+```python
+import daal4py as d4p
+import numpy as np
+
+# Create algorithm with parameters
+algorithm = d4p.dbscan(epsilon=0.5, minObservations=5)
+
+# Run computation
+result = algorithm.compute(data)
+
+# Access results (algorithm-specific attributes)
+cluster_labels = result.assignments
+core_indices = result.coreIndices
+```
+
+## Common Native Algorithms
+```python
+# Clustering
+d4p.dbscan(epsilon=0.5, minObservations=5)
+d4p.kmeans(nClusters=3, maxIterations=300)
+
+# Decomposition
+d4p.pca(method="defaultDense")
+d4p.svd(method="defaultDense")
+
+# Linear Models
+d4p.linear_regression_training()
+d4p.ridge_regression_training(ridgeParameters=1.0)
+```
+
+## Model Builders (mb/)
+```python
+from daal4py.mb import convert_model
+
+# Convert external models to oneDAL format
+d4p_model = convert_model(xgb_model)      # XGBoost → oneDAL
+d4p_model = convert_model(lgb_model)      # LightGBM → oneDAL
+d4p_model = convert_model(catboost_model) # CatBoost → oneDAL
+
+# Use converted model for fast inference
+predictions = d4p_model.compute(test_data)
+```
+
+## Testing
+```bash
+# Native daal4py tests
+pytest --verbose --pyargs daal4py
+pytest tests/test_daal4py_examples.py         # Native API examples
+pytest tests/test_model_builders.py           # Model conversion tests
+
+# sklearn compatibility in daal4py
+pytest daal4py/sklearn/tests/                 # sklearn-compatible API
+```
+
+## Development Notes
+- Native API provides direct oneDAL algorithm access (fastest performance)
+- sklearn-compatible API in `daal4py/sklearn/` maintains full sklearn compatibility
+- Model builders enable oneDAL inference for models trained with other frameworks
+- See `daal4py/AGENTS.md` for detailed algorithm usage patterns

.github/instructions/general.instructions.md

@@ -0,0 +1,62 @@
+# General Repository Instructions - Intel Extension for Scikit-learn
+
+## Repository Overview
+
+**Intel Extension for Scikit-learn** (scikit-learn-intelex) accelerates scikit-learn by 10-100x using Intel oneDAL. Zero code changes required for existing sklearn applications.
+
+- **Languages**: Python (70%), C++ (25%), Cython (5%)
+- **Architecture**: 4-layer system (sklearnex → daal4py → onedal → Intel oneDAL C++)
+- **Platforms**: Linux, Windows, macOS; CPU (x86_64, ARM), GPU (Intel via SYCL)
+- **Python**: 3.9-3.13 supported
+
+## Critical Build Requirements (ALWAYS REQUIRED)
+
+```bash
+# Environment variables (MANDATORY)
+export DALROOT=/path/to/onedal       # Required by setup.py:53-59
+export MPIROOT=/path/to/mpi          # For distributed support
+
+# Build dependencies (INSTALL FIRST)
+pip install -r dependencies-dev     # Cython==3.1.1, numpy>=2.0, pybind11==2.13.6
+
+# Development build (RECOMMENDED)
+python setup.py develop             # Creates editable install
+```
+
+## Testing & Validation (Run in Order)
+
+```bash
+# 1. Install test dependencies
+pip install -r requirements-test.txt
+
+# 2. Core test suites
+pytest --verbose -s tests/                    # Legacy tests
+pytest --verbose --pyargs daal4py             # Native oneDAL tests
+pytest --verbose --pyargs sklearnex           # sklearn compatibility
+
+# 3. Code quality (REQUIRED before commit)
+pre-commit install
+pre-commit run --all-files --show-diff-on-failure
+```
+
+## Code Standards
+
+- **Python**: Black (line-length=90) + isort
+- **C++**: clang-format version ≥14
+- **Commits**: Must be signed-off (`git commit -s`)
+- **Documentation**: numpydoc format
+
+## Common Issues & Solutions
+
+```bash
+# Build failures
+export NO_DIST=1                    # Disable distributed mode if MPI issues
+export NO_DPC=1                     # Disable GPU if driver issues
+python setup.py build_ext --inplace --force --abs-rpath  # Linux linking
+
+# Import/path issues
+export PYTHONPATH=$(pwd)            # Add repo to path
+python setup.py develop             # Ensure editable install
+```
+
+For module-specific details, see the corresponding AGENTS.md files in each directory.

.github/instructions/onedal.instructions.md

@@ -0,0 +1,59 @@
+# onedal/* - Low-Level C++ Bindings
+
+## Purpose
+Pybind11-based C++ bindings providing the bridge between Python and Intel oneDAL C++ library.
+
+## Key Components
+- `datatypes/`: Memory management and array conversions (NumPy, SYCL USM, DLPack)
+- `common/`: Policy management, device selection, serialization
+- `*/`: Algorithm-specific implementations (cluster/, decomposition/, linear_model/, etc.)
+- `spmd/`: Distributed computing interfaces
+
+## Memory Management
+```python
+# Zero-copy conversions handled automatically
+import numpy as np
+from onedal.cluster import DBSCAN
+
+# NumPy arrays converted to oneDAL tables without copying
+X = np.random.random((1000, 10))
+model = DBSCAN().fit(X)  # Automatic NumPy → oneDAL conversion
+```
+
+## Device Context
+```python
+# Device selection handled through dpctl integration
+import dpctl
+from onedal.cluster import DBSCAN
+
+# GPU execution (requires Intel GPU)
+device = dpctl.SyclDevice("gpu:0")
+with dpctl.device_context(device):
+    model = DBSCAN().fit(X)
+```
+
+## Algorithm Structure
+- Each algorithm module follows consistent pattern:
+  - `fit()` method for training
+  - `predict()` method for inference (where applicable)
+  - Parameters match oneDAL C++ API
+  - Results as Python objects with named attributes
+
+## Testing
+```bash
+# Low-level onedal tests
+pytest onedal/tests/                           # Core functionality
+pytest onedal/datatypes/tests/                 # Memory management
+pytest onedal/common/tests/                    # Device/policy tests
+
+# Algorithm-specific tests
+pytest onedal/cluster/tests/test_dbscan.py     # DBSCAN implementation
+pytest onedal/linear_model/tests/              # Linear models
+```
+
+## Development Notes
+- Direct interface to oneDAL C++ API through pybind11
+- Handles memory management between Python/C++ automatically
+- Provides foundation for both daal4py and sklearnex layers
+- SPMD module enables distributed computing with MPI
+- See `onedal/AGENTS.md` for detailed technical implementation

.github/instructions/sklearnex.instructions.md

@@ -0,0 +1,56 @@
+# sklearnex/* - Primary sklearn-compatible Interface
+
+## Purpose
+Primary user interface for sklearn acceleration with patching system and device offloading.
+
+## Key Files & Functions
+- `dispatcher.py`: Patching system (`get_patch_map_core` line 36)
+- `_device_offload.py`: GPU/CPU dispatch (`dispatch` function line 72)
+- `_config.py`: Global configuration (target_offload, allow_fallback_to_host)
+- `base.py`: oneDALEstimator base class for all accelerated algorithms
+
+## Usage Patterns
+
+**Global Patching (Most Common):**
+```python
+from sklearnex import patch_sklearn
+patch_sklearn()                      # All sklearn imports now accelerated
+from sklearn.cluster import DBSCAN   # Uses oneDAL implementation
+```
+
+**Selective Patching:**
+```python
+patch_sklearn(["DBSCAN", "KMeans"])  # Only specific algorithms
+```
+
+**Direct Import (No Patching):**
+```python
+from sklearnex.cluster import DBSCAN  # Always oneDAL implementation
+```
+
+**Device Control:**
+```python
+from sklearnex import config_context
+
+# GPU acceleration (requires Intel GPU + drivers)
+with config_context(target_offload="gpu:0"):
+    model.fit(X, y)
+
+# Force CPU
+with config_context(target_offload="cpu"):
+    model.fit(X, y)
+```
+
+## Testing
+```bash
+# sklearnex-specific tests
+pytest --verbose --pyargs sklearnex
+pytest sklearnex/tests/test_patching.py       # Core patching functionality
+pytest sklearnex/tests/test_config.py         # Configuration system
+```
+
+## Development Notes
+- All sklearn-compatible algorithms inherit from `base.oneDALEstimator`
+- Fallback to original sklearn if oneDAL implementation unavailable
+- Device offloading requires Intel GPU drivers and SYCL runtime
+- See `sklearnex/AGENTS.md` for detailed module information