-
Notifications
You must be signed in to change notification settings - Fork 5.6k
Enhance mod_data to Use Global Loader Extensions in salt-ssh #68497
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
twangboy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a changelog and write a test
## Problem The `mod_data` function in `salt/client/ssh/__init__.py` was only discovering modules from the fileserver backend (e.g., `_modules`, `_states` directories accessed via `fsclient.cache_file()`). This caused salt-ssh to ignore: - Custom modules registered via entry-points - Modules from `extension_modules` configuration - Modules from `module_dirs` CLI options - Custom fileserver backends and other loader plugins This resulted in inconsistent behavior between `salt-call` (which loads all extensions) and `salt-ssh` (which ignores them). ## Solution Modified `mod_data()` to collect modules from three sources: ### 1. Global Loader Paths Uses `salt.loader._module_dirs()` to discover all module directories, including: - Entry-point registered plugins - `extension_modules` from configuration - `module_dirs` from CLI - System-wide module paths ### 2. File Roots Directly scans `file_roots` directories for `_modules`, `_states`, etc., avoiding network-dependent fileserver operations that require master connectivity. ### 3. Error Handling Wraps all discovery operations in try-except blocks to ensure salt-ssh continues working even if: - Some module paths are inaccessible - Entry-point plugins fail to load - File roots are misconfigured ## Benefits - **Consistency**: salt-ssh now uses the same module discovery logic as salt-call - **Entry-point support**: Custom plugins registered via setuptools entry-points are now available - **No master dependency**: Avoids `KeyError: 'master_uri'` by not using fileserver client operations - **Resilience**: Graceful error handling ensures partial failures don't break salt-ssh ## Technical Details **Before**: Only scanned fileserver via `fsclient.cache_file()` → required master connection **After**: 1. Calls `salt.loader._module_dirs(opts, ref, tag=...)` for each module type 2. Scans directories directly from filesystem 3. Collects modules from `file_roots` configuration 4. Builds tarball with all discovered modules for deployment to targets **Key Changes**: - Removed dependency on `fsclient.cache_file()` which triggers master authentication - Added direct filesystem scanning of `file_roots` - Added comprehensive exception handling - Preserved backward compatibility with existing module discovery ## Impact All salt-ssh deployments now have access to: - Custom execution modules - Custom states - Custom grains - Custom renderers - Custom returners - Custom fileserver backends (when registered via entry-points) This resolves the inconsistency where `salt-call --local sys.list_fileserver_backends` would show custom backends but `salt-ssh <target> sys.list_fileserver_backends` would not.
87dd0c5 to
4d11ac9
Compare
Done |
|
The documentation is here: Basically, create a file in the |
Fixed |
|
If you install pre-commit ( Pre-commit will run and make all the changes you're seeing in the failing pre-commit. Repeat those 2 commands until pre-commit comes back clean. |
|
Hello @twangboy. Please re-run tests for Windows. I see he has failed because some services are sent |
|
Congratulations on your first PR being merged! 🎉 |
Problem
The
mod_datafunction insalt/client/ssh/__init__.pywas only discovering modules from the fileserver backend (e.g.,_modules,_statesdirectories accessed viafsclient.cache_file()). This caused salt-ssh to ignore:extension_modulesconfigurationmodule_dirsCLI optionsThis resulted in inconsistent behavior between
salt-call(which loads all extensions) andsalt-ssh(which ignores them).Solution
Modified
mod_data()to collect modules from three sources:1. Global Loader Paths
Uses
salt.loader._module_dirs()to discover all module directories, including:extension_modulesfrom configurationmodule_dirsfrom CLI2. File Roots
Directly scans
file_rootsdirectories for_modules,_states, etc., avoiding network-dependent fileserver operations that require master connectivity.3. Error Handling
Wraps all discovery operations in try-except blocks to ensure salt-ssh continues working even if:
Benefits
KeyError: 'master_uri'by not using fileserver client operationsTechnical Details
Before: Only scanned fileserver via
fsclient.cache_file()→ required master connectionAfter:
salt.loader._module_dirs(opts, ref, tag=...)for each module typefile_rootsconfigurationKey Changes:
fsclient.cache_file()which triggers master authenticationfile_rootsImpact
All salt-ssh deployments now have access to:
This resolves the inconsistency where
salt-call --local sys.list_fileserver_backendswould show custom backends butsalt-ssh <target> sys.list_fileserver_backendswould not.Commits signed with GPG?
Yes
Linked issue: #68496