Skip to content

en-US_agent.md

Latest commit

 

History

History
458 lines (352 loc) · 14.8 KB

en-US_agent.md

File metadata and controls

458 lines (352 loc) · 14.8 KB
Raw

Agent Development Guide

This guide provides AI agents (and developers) with comprehensive information to contribute to and maintain the deep-solutions project.

Table of Contents

  1. Project Engineering Structure
  2. Code Format & Development Specifications
  3. Local Testing Guide
  4. Maintenance & Release Guide

Project Engineering Structure

0. Directory Layout

deep-solutions/
├── src/deep_solutions/       # Main package source code
├── tests/                    # Unit tests
├── docs/                     # Documentation (bilingual)
│   ├── en-US/                # English docs (devs/, user-guide/, design/)
│   └── zh-CN/                # Chinese docs (same structure as en-US)
├── scripts/                  # Development & release scripts
├── .github/                  # GitHub config (workflows, templates, dependabot)
├── pyproject.toml            # Project metadata & tool config
├── environment.yml           # Conda environment specification
├── tox.ini                   # Tox multi-version testing config
├── README.md                 # English README
├── README.zh-CN.md           # Chinese README
└── CHANGELOG.md              # Release history
Directory Purpose
src/deep_solutions/ Main package modules (core, utils, etc.)
tests/ pytest unit tests
docs/en-US/ English documentation (devs/, user-guide/, design/)
docs/zh-CN/ Chinese documentation (mirrors en-US/ structure)
scripts/ Dev scripts: check.sh, ci-local.sh, check_language.py, document_checker.py, test_release.sh, final_release.sh
.github/workflows/ GitHub Actions: CI, publish, test-report
.github/ISSUE_TEMPLATE/ Issue templates (bug, feature, docs, custom)
pyproject.toml Project config: dependencies, tool settings (ruff, mypy, pytest)
environment.yml Conda environment file for dev setup
tox.ini Multi-Python-version testing (3.8-3.12)

0.1. Package Distribution: src/deep_solutions/

src/deep_solutions/
├── __init__.py               # Public API: hello_world, DeepSolution, format_output, get_library_version
├── _version.py               # Auto-generated by setuptools-scm (do not edit)
├── tools/                    # User-facing utility tools (public API)
│   ├── helloworld/           # Template/example tool
│   │   ├── core.py           # Core functionality (@public_api)
│   │   ├── utils.py          # Utility functions (@public_api)
│   │   └── __init__.py       # Public API exports
│   └── parameter_search/     # Parameter search library
│       ├── core/             # Core engine (ParamSearcher, SearchResult)
│       ├── epochs/           # Built-in epoch implementations (timed_epoch, simple_epoch)
│       ├── analyzers/        # Result analyzers (BestParamAnalyzer, ChartAnalyzer)
│       └── pytorch/          # PyTorch wrapper (DataLoaderParamSelector)
└── _utils/                   # Internal utilities (NOT public API)
    ├── timer.py              # Performance timing
    ├── metrics.py            # Metrics collection
    ├── decorators.py         # @public_api decorator
    └── __init__.py           # Internal exports

Structure Principles:

  • tools/: User-facing utilities (public API). Each tool is a complete, task-oriented package. Use @public_api decorator to mark stable APIs.
  • _utils/: Internal, project-agnostic utilities. NOT public API. Users should not import from _utils.
  • Tests: Match structure under tests/test_tools/ and tests/test__utils/.
Module Contents Purpose
tools/helloworld/ hello_world(), DeepSolution, format_output() Template tool demonstrating structure
tools/parameter_search/ ParamSearcher, DataLoaderParamSelector Parameter space exploration
_utils/ Timer, MetricsCollector, @public_api Internal utilities for all modules

0.2. Development Environment

Project Overview

  • Language: Python 3.8+ (minimum: 3.8, maximum tested: 3.13)
  • Package Manager: Conda + pip
  • Build System: setuptools with setuptools-scm (auto-versioning from Git tags)
  • Code Quality: Ruff (format + lint), mypy (type checking), pytest (testing)
  • CI/CD: GitHub Actions (CI, publish, test-report)

Dependencies

Core Dependencies (runtime):

numpy >= 1.17.0
scipy >= 1.5.0

Development Dependencies (optional):

pytest >= 7.0.0          # Unit testing
pytest-cov >= 4.0.0      # Coverage reporting
ruff >= 0.4.0            # Formatting + linting
mypy >= 1.0.0            # Type checking
build >= 0.10.0          # Building packages
twine >= 4.0.0           # Publishing to PyPI
tox >= 4.0.0             # Multi-version testing
commitizen >= 3.0.0      # Conventional commits
pre-commit >= 3.0.0      # Git hooks

Setup Development Environment

Step 1: Clone the repository

git clone https://github.com/FrostyHec/deep-solutions.git
cd deep-solutions

Step 2: Create Conda environment

conda env create -f environment.yml
conda activate deep-solutions

Step 3: Install the package in editable mode

pip install -e ".[dev]"

Step 4: Verify setup

python -c "import deep_solutions; print(f'Version: {deep_solutions.__version__}')"
python -m pytest tests/ -v

Dependency Management

Core Principle: All pip dependencies managed in pyproject.toml only.

  • Runtime deps → [project].dependencies
  • Dev deps → [project.optional-dependencies].dev
  • Local dev environment must strictly match pyproject.toml

Update Strategy:

  • Runtime deps: Conservative updates (impacts users). Increase minimum version only when needed for features.
  • Dev deps: Can update more aggressively (only affects contributors & CI).

Adding a new dependency:

  1. Create branch: git checkout -b chore/add-<pkg>
  2. Edit pyproject.toml (add version like "package>=x.y")
  3. Reinstall: pip install -U pip && pip install -e ".[dev]"
  4. Verify: bash scripts/check.sh && tox
  5. Commit with reason (e.g., "build(deps): add requests for HTTP support")

Upgrading existing dependency:

When you need a new feature (available in version X.Y):

  1. Update pyproject.toml: change >=old to >=x.y
  2. Reinstall: pip install -U pip && pip install -e ".[dev]"
  3. Verify: bash scripts/check.sh && tox

Example:

# Before: numpy>=1.17
# After (need new API): numpy>=1.23

Verification checklist (after any dependency change):

  • pip install -e ".[dev]" succeeds
  • bash scripts/check.sh passes
  • tox passes (at least Python 3.8 + latest)

Code Format & Development Specifications

Language Policy

  • Source Code: English only (no Chinese)

    • Comments, docstrings, variable names must be in English
    • Type hints required for function parameters and returns

  • Documentation: Bilingual (English base + Chinese translations)

    • English docs in docs/en-US/ (organized by devs/, user-guide/, design/)
    • Chinese docs in docs/zh-CN/ (mirrors en-US/ structure)
    • See Language Guidelines for details

  • Git Commits: English only (Conventional Commits format)

    • Format: type(scope): subject (e.g., feat(core): add new solver)
    • See Commit Conventions for details

Code Style & Standards

Formatting (Ruff):

  • Line length: 88 characters
  • Quote style: double quotes (")
  • Indent: 4 spaces

Linting (Ruff):

  • No unused imports
  • PEP 8 compliance
  • Bugbear checks enabled
  • Comprehension checks enabled

Type Hints (mypy):

  • Required for all public functions
  • Target Python 3.8 compatibility
  • Example: 
    def process_data(x: List[int]) -> Dict[str, float]:
    """Process a list of integers."""
    return {"mean": sum(x) / len(x)}

Docstrings:

  • Use Google-style docstrings
  • Include Args, Returns, Raises sections
  • Example: 
    def hello_world() -> str:
    """Return a greeting message.
    
    Returns:
        A simple greeting string.
    """
    return "Hello from deep-solutions!"

When Chinese is Allowed

✅ Allowed:

  • Documentation in docs/zh-CN/ directory (files prefixed with zh-CN_)
  • Comments in Chinese documentation files
  • User-facing error messages (can be localized), but must use appropriate i18n framework

❌ Not Allowed:

  • Source code (.py files)
  • Git commit messages
  • GitHub issue/PR titles
  • Code comments

Local Testing Guide

One-Command: Run All Checks

# Using bash script
bash scripts/check.sh

# Or manually, in order:
ruff format --check src/ tests/     # 1. Format check
ruff check src/ tests/               # 2. Lint check
mypy src/                            # 3. Type check
python scripts/check_language.py     # 4. Language check
pytest                               # 5. Unit tests

Run Unit Tests

# Run all tests with coverage
pytest

# Run specific test file
pytest tests/test_core.py

# Run with verbose output
pytest -v

# Run only tests matching a pattern
pytest -k "test_hello" -v

Run Code Formatting Check (Ruff)

# Check formatting
ruff format --check src/ tests/

# Auto-fix formatting
ruff format src/ tests/

# Lint check (code quality)
ruff check src/ tests/

# Auto-fix linting issues
ruff check --fix src/ tests/

Run Type Checking (mypy)

# Type check
mypy src/

# Type check with verbose output
mypy src/ --show-error-codes

Run Language Check

# Check for Chinese characters in source code
python scripts/check_language.py -v

Multi-Version Testing (tox)

Test against multiple Python versions (3.8-3.12):

# Run all Python versions
tox

# Run specific Python version
tox -e py38

# Run and generate coverage report
tox -- --cov=deep_solutions

Coverage Report

View test coverage:

pytest --cov=deep_solutions --cov-report=html
# Open htmlcov/index.html in browser

Maintenance & Release Guide

Version Management

The project uses setuptools-scm for automatic versioning from Git tags.

Version Format: v<major>.<minor>.<patch> (e.g., v1.0.0, v0.2.1)

Release Checklist

Prerequisites (one-time setup):

conda install -c conda-forge gh
gh auth login

Standard Release Flow (Recommended)

To release version v0.1.1:

Step A — Test on TestPyPI (test_release.sh):

bash scripts/test_release.sh
# Enter tag: v0.1.1.dev1
# If it fails, fix and re-run with v0.1.1.dev2, etc.

Step B — Publish to PyPI (final_release.sh):

bash scripts/final_release.sh
# Choose rename mode (y): rename v0.1.1.dev1 → v0.1.1
# Optionally clean up leftover .dev tags
# Script triggers publish.yml and waits for result

Shortcut: If you're confident, you can test directly with v0.1.1 in Step A (no .dev suffix), then use direct mode (n) in Step B.

test_release.sh — Test Release

Automates the TestPyPI validation pipeline:

  1. Reminds you to update CHANGELOG.md
  2. Switches to main and pulls latest code
  3. Prompts for version tag (e.g., v0.1.1.dev1)
  4. Creates and pushes git tag
  5. Triggers publish-test.yml via gh CLI
  6. Waits for completion and displays results

final_release.sh — Production Release

Publishes to production PyPI with two modes:

  • Rename mode (typical): Renames a tested tag to a clean version (e.g., v0.1.1.dev3 → v0.1.1), optionally deletes leftover .dev tags.
  • Direct mode: Uses an existing tag as-is (when you tested with the final version number directly).

Then triggers publish.yml, waits for completion, and reports the result.

CI Workflows

Workflow Trigger Purpose
CI PR / push to main Lint, type-check, test (3.8 for PR, full matrix for main)
Test Report After CI completes Parse test results, post sticky PR comment
Publish to TestPyPI Manual dispatch Build & publish to TestPyPI for testing
Publish to PyPI Manual dispatch Build & publish to production PyPI, create Release

Useful Commands Reference

Command Purpose
conda activate deep-solutions Activate dev environment
pip install -e ".[dev]" Install package in editable mode with dev deps
bash scripts/check.sh Run all local checks (format, lint, type, language, tests)
bash scripts/ci-local.sh Simulate CI pipeline locally
python scripts/check_language.py Check English-only code (calls document_checker)
python scripts/document_checker.py Check bilingual docs, broken refs, structure
pytest Run unit tests
ruff format src/ tests/ Auto-fix code formatting
ruff check --fix src/ tests/ Auto-fix linting issues
mypy src/ Type check
tox Test on Python 3.8-3.12
git tag -a v1.0.0 -m "Release v1.0.0" Create release tag
python -m build Build distribution (wheel + sdist)
twine upload dist/* Upload to PyPI

Documentation & Structure Sync

When the project file structure changes significantly, the following documents must be updated to stay in sync:

Document What to Update
en-US_project_structure.md + zh-CN_project_structure.md Directory tree, file descriptions
en-US_agent.md + zh-CN_agent.md Directory layout (§0), scripts table
docs/{en-US,zh-CN}/index.md Top-level doc index if new subdirs added
docs/{en-US,zh-CN}/{subdir}/index.md Subdir index if files added/removed
README.md + README.zh-CN.md Documentation links table

Documentation File Naming Convention

  • English docs: docs/en-US/{subdir}/en-US_filename.md
  • Chinese docs: docs/zh-CN/{subdir}/zh-CN_filename.md
  • Every new doc must have both EN and ZH versions
  • Every directory must contain an index.md
  • check_language.py enforces bilingual parity automatically

Additional Resources