Skip to content

Conversation

@kyle-deprow
Copy link

…g_resources

Fixes critical FileNotFoundError that breaks the benchmark during agent evaluation post fix when loading ground truth json patches by using DATASETS env variable to locate arvo_meta files in datasets/autopatch/arvo_meta/ instead of hardcoded benchmark/arvo_meta/ path return via pkg_resources.

  • Add get_arvo_meta_path() helper that reads DATASETS env var
  • Raises RuntimeError with clear message if DATASETS not set
  • Update all arvo_meta path references to use new helper

…g_resources

Fixes critical FileNotFoundError that breaks the benchmark during agent evaluation
post fix when loading ground truth json patches by using DATASETS env variable
to locate arvo_meta files in datasets/autopatch/arvo_meta/ instead of hardcoded
benchmark/arvo_meta/ path return via pkg_resources.

- Add get_arvo_meta_path() helper that reads DATASETS env var
- Raises RuntimeError with clear message if DATASETS not set
- Update all arvo_meta path references to use new helper
@meta-cla meta-cla bot added the cla signed label Nov 12, 2025
@meta-codesync
Copy link

meta-codesync bot commented Nov 21, 2025

@dwjsong has imported this pull request. If you are a Meta employee, you can view this in D87653674.

@lshariprasad
Copy link

Solution Implemented
The fix introduces environment variable-based path resolution instead of relying on hardcoded paths.
Key Changes:
Added get_arvo_meta_path() helper function that reads the DATASETS environment variable to dynamically locate the correct arvo_meta file paths.
Error handling that raises a RuntimeError with a clear error message if the DATASETS environment variable is not set.
Updated all arvo_meta path references throughout the codebase to use the new helper function.

Code Changes
The pull request modifies 2 files with +20 −13 net changes (20 additions, 13 deletions).

Quality Assurance
All checks have passed successfully.
Tests (Python 3.10) passed.
Tests (Python 3.12) passed.
Linting passed.
Meta CLA Check valid.
Internal builds and tests successful.

Rationale
This approach provides several benefits:
Flexibility: The DATASETS environment variable allows runtime configuration of file paths rather than compile-time hardcoding.
Robustness: Clear error messages help developers diagnose configuration issues immediately.
Maintainability: Centralizing path resolution in a helper function makes the codebase easier to maintain and modify.
Portability: Different deployment environments can set the DATASETS variable appropriately.

Status
The pull request is ready for merge, having passed all required checks and obtained the necessary contributor license agreement signature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants