Skip to content

TESTING.md

Latest commit

 

History

History
219 lines (170 loc) · 9.04 KB

TESTING.md

File metadata and controls

219 lines (170 loc) · 9.04 KB

Test Execution Guide

Prerequisites

uv must be installed.

# Installing uv (if not already installed)
curl -LsSf https://astral.sh/uv/install.sh | sh   # macOS / Linux
# or
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"  # Windows

Running Tests

Run all tests (Basic)

uvx --with-requirements requirements.txt --with-editable . pytest tests/ -v --timeout=30

Show summary only (Concise)

uvx --with-requirements requirements.txt --with-editable . pytest tests/ -q --timeout=30

Dependency Management

requirements.txt is generated from requirements.in. When updating or regenerating dependencies:

uv pip compile requirements.in -o requirements.txt
File Role
requirements.in Dependency library definition (version constraints only)
requirements.txt Pinned lockfile generated by uv pip compile

Test Structure

tests/
├── unit/               # Local specification tests for each module
│   ├── test_lock.py
│   ├── test_quota.py
│   ├── test_path_normalize.py
│   ├── test_files_sequential.py
│   ├── test_files_randomaccess.py
│   ├── test_handle_io.py
│   ├── test_fs_coverage.py         # v11: Comprehensive test for FS public API
│   ├── test_timestamp.py           # v11: Timestamp / stat() API
│   ├── test_default_storage.py
│   ├── test_text_handle.py
│   ├── test_package_metadata.py
│   ├── test_memory_info.py         # v14: OS dependent memory detection
│   └── test_memory_guard.py        # v14: MemoryGuard strategy
├── integration/        # Integration tests for MemoryFileSystem public API
│   ├── test_open_modes.py
│   ├── test_mkdir_listdir.py
│   ├── test_rename_move.py       # v10: Added move() / copy_tree()
│   ├── test_remove_rmtree.py
│   ├── test_export_import.py
│   ├── test_stats.py
│   ├── test_concurrency.py
│   ├── test_async.py             # v11: AsyncMemoryFileSystem
│   └── test_memory_guard_integration.py  # v14: MemoryGuard integration test
├── scenarios/          # Scenario tests for representative use cases
│   ├── test_usecase_sqlite_snapshot.py
│   ├── test_usecase_etl_staging.py
│   ├── test_usecase_archive_like.py
│   └── test_usecase_restricted_env.py
├── stress/             # Load and stress tests
│   └── test_threaded_stress.py
├── property/           # Hypothesis property-based tests
│   └── test_hypothesis.py
└── helpers/            # Common test utilities
    ├── fixtures.py
    ├── concurrency.py
    └── asserts.py

Commonly Used Options

Run specific files/tests

# Specify a file
uvx --with-requirements requirements.txt --with-editable . pytest tests/unit/test_lock.py -v

# Filter by test function name (-k option)
uvx --with-requirements requirements.txt --with-editable . pytest tests/ -k "quota" -v

# Specific markers only (defined in pytest.ini: p0 / p1 / p2)
uvx --with-requirements requirements.txt --with-editable . pytest tests/ -m p0 -v

Parallel execution (pytest-xdist)

uvx --with-requirements requirements.txt --with-editable . pytest tests/ -n auto --timeout=30

Stop immediately on failure

uvx --with-requirements requirements.txt --with-editable . pytest tests/ -x --timeout=30

Change timeout

# Default is 30 seconds (set in pytest.ini)
uvx --with-requirements requirements.txt --with-editable . pytest tests/ --timeout=60

Coverage Measurement

# Show missing lines in terminal
uvx --with-requirements requirements.txt --with-editable . pytest tests/ --cov=dmemfs --cov-report=term-missing --timeout=30

# Generate XML report similar to CI (coverage.xml)
uvx --with-requirements requirements.txt --with-editable . pytest tests/ --cov=dmemfs --cov-report=xml --cov-report=term-missing --timeout=30

# Generate HTML report (htmlcov/index.html)
uvx --with-requirements requirements.txt --with-editable . pytest tests/ --cov=dmemfs --cov-report=html --timeout=30

Testing with free-threaded Python (GIL disabled)

Verify that D-MemFS lock mechanisms work correctly without depending on the GIL. Run stress tests under tests/stress/ (high load pattern: 50 threads * 1000 times) using free-threaded Python 3.13t.

# 1. Install free-threaded Python (if not already installed)
uv python install cpython-3.13+freethreaded

# 2. Run stress tests only (use free-threaded build with GIL=0)
$env:PYTHON_GIL = "0"
uvx --python cpython-3.13+freethreaded --with-requirements requirements.txt --with-editable . pytest tests/stress/ -v --timeout=30

# 3. Run all tests with free-threaded Python (Optional/Recommended)
uvx --python cpython-3.13+freethreaded --with-requirements requirements.txt --with-editable . pytest tests/ -v --timeout=30

⚠️ $env:PYTHON_GIL = "0" is an environment variable to explicitly disable the GIL. While it is disabled by default in the free-threaded build, it is recommended to set it explicitly.

Generating API Documentation (pdoc)

# Generate HTML API documentation (under docs/api)
uvx --with-requirements requirements.txt pdoc dmemfs -o docs/api

Test File Mapping Table

File breakdown of all tests (436 items):

File Path Test Count Covered Area
tests/unit/test_lock.py 14 ReadWriteLock basic behavior, timeout, reentrancy, thread contention
tests/unit/test_quota.py 12 QuotaManager reserve/release, quota exceeded
tests/unit/test_path_normalize.py 12 Path normalization, invalid path detection
tests/unit/test_files_sequential.py 15 SequentialMemoryFile read/write, promotion triggers
tests/unit/test_files_randomaccess.py 17 RandomAccessMemoryFile seek/truncate, random write
tests/unit/test_handle_io.py 30 MemoryFileHandle mode-specific I/O, flush/close
tests/unit/test_fs_coverage.py 19 Comprehensive test for FS public API (added in v12)
tests/unit/test_timestamp.py 16 stat() / timestamps / generation
tests/unit/test_default_storage.py 9 Default storage settings / switching
tests/unit/test_text_handle.py 14 MFSTextHandle encode/decode, readline, iteration
tests/unit/test_package_metadata.py 1 Package metadata consistency
tests/unit/test_memory_info.py 8 v14: get_available_memory_bytes() OS dependent memory detection
tests/unit/test_memory_guard.py 8 v14: NullGuard/InitGuard/PerWriteGuard strategy patterns
tests/unit/test_archive_sanitize.py 12 v15: _sanitize_archive_path() Zip Slip prevention
tests/unit/test_archive_adapter.py 13 v15: ArchiveAdapter base class, can_handle(), _detect()
tests/integration/test_open_modes.py 21 open() modes behavior (rb/wb/ab/r+b/xb)
tests/integration/test_mkdir_listdir.py 32 mkdir, listdir, exists, is_dir, glob, walk
tests/integration/test_rename_move.py 34 rename, move, copy, copy_tree
tests/integration/test_remove_rmtree.py 11 remove, rmtree
tests/integration/test_export_import.py 19 export_tree, import_tree, export_as_bytesio
tests/integration/test_stats.py 10 stats(), get_size()
tests/integration/test_concurrency.py 7 Multi-threaded concurrent access
tests/integration/test_async.py 22 AsyncMemoryFileSystem asynchronous API
tests/integration/test_memory_guard_integration.py 12 v14: MemoryGuard integration tests, MemoryError messages
tests/integration/test_archive_expand.py 19 v15: expand_archive() atomic extraction integration tests
tests/integration/test_archive_streaming.py 20 v15: expand_archive_streaming() streaming extraction integration tests
tests/scenarios/test_usecase_etl_staging.py 4 ETL staging, incremental updates, concurrent writing
tests/scenarios/test_usecase_archive_like.py 5 Archive operations, import_tree/export_tree roundtrip
tests/scenarios/test_usecase_sqlite_snapshot.py 3 SQLite serialize/deserialize, quota limits
tests/scenarios/test_usecase_restricted_env.py 3 Use cases in quota-restricted environments
tests/scenarios/test_usecase_archive_expand.py 3 v15: UC-1 archive expand → transform → repack scenarios
tests/stress/test_threaded_stress.py 6 Multi-threaded stress tests
tests/property/test_hypothesis.py 5 Hypothesis property-based tests
Total 436

See .github/workflows/test.yml for GitHub Actions settings. If you want to reproduce the identical environment locally as CI:

# Regenerate requirements.txt before running tests
uv pip compile requirements.in -o requirements.txt
uvx --with-requirements requirements.txt --with-editable . pytest tests/ -v --timeout=30

# Reproduce API documentation generation
uvx --with-requirements requirements.txt pdoc dmemfs -o docs/api