-
Notifications
You must be signed in to change notification settings - Fork 960
Open
Labels
cuIOcuIO issuecuIO issuefeature requestNew feature or requestNew feature or requestlibcudfAffects libcudf (C++/CUDA) code.Affects libcudf (C++/CUDA) code.
Milestone
Description
Is your feature request related to a problem? Please describe.
Currently, benchmarking tools like PDS include some pre-loading work to help complete cuIO setup that is done lazily.
cuIO setup steps include:
- loading cuFile (if enabled)
- loading nvCOMP (if host compression, decompression is set to AUTO or OFF)
- allocating KvikIO pinned buffers
Describe the solution you'd like
We should add an API to complete the buffer allocation and library loading, so that the first cuIO read has the same latency as the second read. The API would need to be public and have Python bindings (perhaps cudf::io::setup()
or some other design would be fine)
Describe alternatives you've considered
Continue adding dummy IO steps to benchmarks.
Additional context
We talked about doing this automatically during import cudf
, but it would add pressure to the overall import time. See #627
Metadata
Metadata
Assignees
Labels
cuIOcuIO issuecuIO issuefeature requestNew feature or requestNew feature or requestlibcudfAffects libcudf (C++/CUDA) code.Affects libcudf (C++/CUDA) code.