Skip to content

sync with main + bump version to 5.0.1dev #4825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1,414 commits into from
Mar 18, 2025
Merged

Conversation

boegel
Copy link
Member

@boegel boegel commented Mar 18, 2025

No description provided.

Flamefire and others added 30 commits February 4, 2025 18:10
Small improvements to `ModuleLoadEnvironment`
Both branches are identical so the function is no longer required.
…lready prints at info level the stdout and stderr of those commands
The create_unused_dir function distinguishes between the parent
directory and the name of the target subdirectory in 2 separate
arguments. This interface requires additional parameter management when
combined with newer path management interfaces such as pathlib.

- Add a function create_unused_dirs accepting the target paths as a
  single argument.
- Support the simultaneous creation of multiple unused paths with all of
  them having the same suffix if any of the actually requested paths is
  unavailable.
Packages are be built in some selected build path (--buildpath), and
the logs of successful compilation are then concentrated to some other
location for permanent storage (--logfile-format). Logs of failed builds
remain in the build path location so that they can be inspected.

However, this setup is problematic when building software in HPC jobs. Quite
often in HPC systems the build path is set to some fast storage local to
the node, like NVME raid mounted on `/tmp` or `/dev/shm` (as suggested
in the documentation: https://docs.easybuild.io/configuration/#buildpath).
The node storage is often wiped out after the end of a job, so the log
files and the artifacts are no longer available after the termination of
the job.

This commit adds options (--log-error-path and --artifact-error-path) to
accumulate error logs and artifacts in some more permanent locations, so
that the can be easily inspected after a failed build. By default,
neither the logs nor the build artifacts are copied in the permanent
location.
Flags for persisting logs and artifacts are now checked during the
validation of input options. In case of invalid settings the program
terminates early with an error.

Integration tests for internal logging functions are deactivated as the
logging functions are never reached in normal operation.
- Avoid single character variables names outside list comprehension.
- Use a consistent pattern in function calls throughout the pull request.
- Use context manager to mock stdout and stderr.
enable non-blocking reads for streaming outputs
When copying logs and artifacts from failed builds, mention the full
target path instead of its parent. This reduces the number of actions
required to access the artifacts (select, paste, and enter).
Adapt `module show` command run to cope with non-zero exit code for non-existing module (required for Environment Modules v5.5+ and Lmod 8.7.56+)
add extra test case for make_module_req
@boegel boegel added the update label Mar 18, 2025
@boegel boegel added this to the release after 5.0.0 milestone Mar 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.