Skip to content

test(picklescan): cover startup hook source refresh#1228

Merged
mldangelo-oai merged 1 commit intomainfrom
test/call-graph-source-cache-refresh
May 8, 2026
Merged

test(picklescan): cover startup hook source refresh#1228
mldangelo-oai merged 1 commit intomainfrom
test/call-graph-source-cache-refresh

Conversation

@mldangelo-oai
Copy link
Copy Markdown
Contributor

Summary

  • add a focused call-graph regression for startup hook file-open/file-write source rewrites
  • verify rewritten module source is re-read instead of using stale clean cache state

Tests

  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest packages/modelaudit-picklescan/tests/test_call_graph_import_statements.py::test_startup_hook_write_call_graph_refreshes_after_source_rewrite
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run ruff format modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run ruff check --fix modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache uv run mypy modelaudit/ packages/modelaudit-picklescan/src packages/modelaudit-picklescan/tests tests/
  • UV_CACHE_DIR=/tmp/modelaudit-uv-cache PROMPTFOO_DISABLE_TELEMETRY=1 uv run pytest -n auto -m "not slow and not integration" --maxfail=1

Note: the first full pytest attempt hit an unrelated load-sensitive timing assertion in tests/scanners/test_pytorch_zip_scanner.py::test_pytorch_zip_skips_numeric_data_files; that test passed in isolation and the full rerun passed.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

Workflow run and artifacts

Performance Benchmarks

Compared 12 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 12 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 562.00ms -> 555.76ms (-1.1%).

Workload Benchmark Target Size Files Baseline Current Change Status
clean-training-checkpoint tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint safe_large 278.2 KiB 1 11.53ms 10.30ms -10.6% stable
suspicious-pickle-intake tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake suspicious-intake 183.8 KiB 4 72.11ms 68.72ms -4.7% stable
padded-multi-stream-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload multi_stream_padded 4.1 KiB 1 398.1us 382.2us -4.0% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64] nested_base64 98 B 1 117.1us 120.6us +3.0% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex] nested_hex 130 B 1 128.0us 124.5us -2.7% stable
chunked-upload-stream tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream chunked_stream 278.2 KiB 1 13.81ms 13.52ms -2.1% stable
direct-malicious-upload tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload malicious_reduce 52 B 1 335.1us 328.9us -1.8% stable
single-checkpoint-preflight tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load single_checkpoint.pkl 183.0 KiB 1 33.85ms 33.51ms -1.0% stable
nested-payload-review tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw] nested_raw 78 B 1 114.4us 113.3us -0.9% stable
duplicate-heavy-registry tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot registry-snapshot 915.2 KiB 13 174.53ms 173.48ms -0.6% stable
warm-cache-rescan tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan release-candidate 547.3 KiB 32 30.29ms 30.39ms +0.3% stable
mixed-model-repository tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository release-candidate 547.3 KiB 32 224.79ms 224.77ms -0.0% stable

@mldangelo-oai mldangelo-oai merged commit c930fda into main May 8, 2026
30 checks passed
@mldangelo-oai mldangelo-oai deleted the test/call-graph-source-cache-refresh branch May 8, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant