Xrootd https remote #4822
Open
Xrootd https remote #4822
+361
−87
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix http.exthandler path prefix (use /ssi to match client URL) - Ensure config generation and server startup use same working directory - Add better error handling and debugging output to startup scripts - Remove suppression of openssl errors to aid debugging Co-Authored-By: Claude Opus 4.5 <[email protected]>
The spack-installed OpenSSL has a corrupted config file path. Work around this by setting OPENSSL_CONF=/dev/null when generating the self-signed certificate, since -subj provides all necessary information. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Use direct exit code check with 'if command; then' instead of checking $? indirectly, as recommended by shellcheck. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Generate unique ports based on MD5 hash of build directory to avoid conflicts when multiple CI jobs run in parallel on the same machine - Pass XRD and HTTP ports as arguments to generateXRootDHttpConfig.sh - Set XRootDHttpsHost environment variable in tests with correct port - Improve start script to check for daemon/nobody user availability when running as root, and show log on startup failure - Fix shellcheck SC2181 by using direct exit code check Co-Authored-By: Claude Opus 4.5 <[email protected]>
XRootD requires SSL keys to have restricted permissions (600), but when running as root with -R to switch to daemon/nobody user, that user needs read access to the key. - Set key to 600 in config script (XRootD requirement for non-root) - Change ownership of cert directory to target user in start script when running as root with user switching Co-Authored-By: Claude Opus 4.5 <[email protected]>
When XRootD loads plugins, it doesn't use RTLD_GLOBAL, so the XrdSsiProviderServer symbol from the SSI plugin isn't visible via dlopen(NULL). Fix by: - Pass SSI library path as 'ssilib=' parameter to HTTP handler - Use RTLD_NOLOAD to get handle to already-loaded library - Fall back to loading the library if not pre-loaded This allows the HTTP handler to find the SSI provider symbol regardless of how XRootD loads its plugins. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Rename XrootdHttpsRemote to XrootdHttpRemote to reflect support for both HTTP and HTTPS protocols - Add m_UseHttps flag (default true) to configure protocol at runtime - Add DoXRootDHttp environment variable for plain HTTP mode in BP5Reader - HTTP mode is needed for NERSC Spin deployment where Spin Ingress terminates TLS and forwards plain HTTP to containers Docker files for Spin deployment (scripts/docker/spin-xrootd/): - Dockerfile: Builds x86_64 image with ADIOS2, XRootD, CURL, and HTTP-to-SSI bridge handler - xrootd-http.cfg: XRootD config for HTTP mode on port 8080 - docker-entrypoint.sh: Container startup script - build.sh: Build script for cross-platform builds on ARM Macs Co-Authored-By: Claude Opus 4.5 <[email protected]>
eisenhauer
force-pushed
the
xrootd-https-remote
branch
from
cae889f to
626cd7b
Compare
- Set ownership of SSL certs to xrootd user (194:194) so non-root container can read the private key - Set ownership of /var/spool/xrootd, /run/xrootd, /var/log/xrootd, and /data directories for xrootd user - Required for NERSC Spin deployment with security policies requiring non-root containers with dropped capabilities Co-Authored-By: Claude Opus 4.5 <[email protected]>
eisenhauer
force-pushed
the
xrootd-https-remote
branch
from
626cd7b to
48c57ee
Compare
Member
Author
|
The docker stuff here will need to change after this is merged and we've can build ADIOS from an official source, but including it in this PR because it was necessary for testing. |
Collaborator
|
@eisenhauer is this intended to run in the CI? |
Member
Author
Clarifying. The docker scripts are not for CI, but just to help build what we would need to have an ADIOS/XRootD server in a docker image for deployment to environments like NERSC Spin. The new transports would be built and tested in CI if we had a build that had both xrootd and libcurl. I haven't looked to see if we do. (And actually I guess this would be like the other situation where we might have to add CURL to the spack after this is integrated?) |
Collaborator
|
sounds good! I suggest to make these scripts also podman compatible (so they can build without root) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.