Skip to content

Use CMake's FILE_SET for better modeling project. #1412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 30, 2024

Conversation

HungMingWu
Copy link
Contributor

@HungMingWu HungMingWu commented Sep 13, 2024

No description provided.

Copy link

copy-pr-bot bot commented Sep 13, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@HungMingWu HungMingWu changed the title Use CMake's FILE_SET to replace target_include_directories for better convience Use CMake's FILE_SET to replace target_include_directories for better convenience Sep 13, 2024
@ericniebler
Copy link
Collaborator

how is this more convenient, and for whom? sorry, i only know enough cmake to get by. this pr is beyond my understanding.

@HungMingWu
Copy link
Contributor Author

HungMingWu commented Sep 13, 2024

how is this more convenient, and for whom? sorry, i only know enough cmake to get by. this pr is beyond my understanding.

From now on there is no difference , but if we want to transfer the library into C++20 module, FILE_SET is necessary.

Please refer
C++20 Modules, CMake, And Shared Libraries
And
my little experimental repo

Mix FILE_SET and target_include_directories is OK, but I hope it is much identical..

@ericniebler
Copy link
Collaborator

/ok to test

@AlexisTM
Copy link

Is the hardcoded list of includes necessary?
That looks like a pain to maintain.

@HungMingWu
Copy link
Contributor Author

It is all right to use file(GLOB_RECURSE xxx)
But from stackoverflow's answer1 and answer2

It seems the idea is not recommend.

Copy link

@bretbrownjr bretbrownjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am in favor of the direction of this PR, assuming stdexec is willing to set its minimum required CMake version to at least 3.23 (almost three years old). Better modeling of how headers relate to libraries will improve interop, tooling, and project clarity going forward.

To illustrate, as an enhancement to this PR, I also suggest:

... possibly as a separate PR.

I would do this in the package build logic (like conanfile.py) and in the CI verification logic (like the GitHub Actions yaml). I would also consider making that change in any IDE configuration logic (like vscode JSON)

This will create and build a C++ source file for each stdexec header file, ensuring that each stdexec header is free of missing includes. Those source files would also be added to compile_commands.json when that is created, more directly exposing stdexec headers to tools that consume compile_commands.json such as clang-tidy.

@HungMingWu HungMingWu changed the title Use CMake's FILE_SET to replace target_include_directories for better convenience Use CMake's FILE_SET for better modeling project. Sep 21, 2024
@ericniebler
Copy link
Collaborator

#1403 added new headers to stdexec. i am guessing something here needs to change to accommodate them.

that highlights the thing about this PR that makes me uncomfortable: it adds a header list to the build files that needs to be kept in sync with the rest of the project. i anticipate problems arising from headers getting added to the project but not added to the FILE_SET.

i think @bretbrownjr has thoughts about how to remove the maintenance burden. Bret?

@ericniebler
Copy link
Collaborator

/ok to test

@bretbrownjr
Copy link

You're probably looking for the file(GLOB ...) API. Make sure to use the CONFIGURE_DEPENDS flag to make sure CMake re-executes your CMakeLists.txt when the modification time of the relevant directory or directories change. Otherwise, you'll need to make new CMake build directories (for instance) to force CMake to detect file creation, deletion, or renaming. No one wants that.

Anyway, that API takes a glob, returns a CMake list of paths to files, and then you can use that to populate relevant FILE_SETs.

Note the text in the linked CMake docs on the drawbacks of using globbing instead of explicit file lists, but it's valid to be more concerned about maintaining file manifests explicitly.

@HungMingWu
Copy link
Contributor Author

I am not sure which direction do you want (file(GLOB) or explicit FILE_SET), but I can modify to another version if needed.

@HungMingWu
Copy link
Contributor Author

Hi @bretbrownjr , I create an experimental CMakeLists.txt based on #1403
But I met some problem

cmake_minimum_required(VERSION 3.25.0)

if(POLICY CMP0135)
  # make the timestamps of ExternalProject_ADD match the download time
  # https://cmake.org/cmake/help/latest/policy/CMP0135.html
  cmake_policy(SET CMP0135 NEW)
  set(CMAKE_POLICY_DEFAULT_CMP0135 NEW)
endif()

if(POLICY CMP0141)
  # Enables the use of the MSVC_DEBUG_INFORMATION_FORMAT target property
  # https://cmake.org/cmake/help/latest/policy/CMP0141.html
  cmake_policy(SET CMP0141 NEW)
  set(CMAKE_POLICY_DEFAULT_CMP0141 NEW)
endif()

##############################################################################
# - Download and initialize RAPIDS CMake helpers -----------------------------

# Fetch rapids-cmake
if(NOT EXISTS ${CMAKE_BINARY_DIR}/RAPIDS.cmake)
  file(DOWNLOAD https://raw.githubusercontent.com/rapidsai/rapids-cmake/branch-24.02/RAPIDS.cmake
       ${CMAKE_BINARY_DIR}/RAPIDS.cmake)
endif()
# Initialize rapids-cmake
include(${CMAKE_BINARY_DIR}/RAPIDS.cmake)
# utilities for generating export set package metadata
include(rapids-export)
# utilities for finding packages
include(rapids-find)
# utilities for defining project defaults
include(rapids-cmake)
# utilities for using CPM
include(rapids-cpm)

##############################################################################
# - Project definition -------------------------------------------------------

# Define the project and set the version and languages
if(NOT EXISTS ${CMAKE_BINARY_DIR}/execution.bs)
  file(DOWNLOAD "https://raw.githubusercontent.com/cplusplus/sender-receiver/main/execution.bs"
      ${CMAKE_BINARY_DIR}/execution.bs)
endif()
file(STRINGS "${CMAKE_BINARY_DIR}/execution.bs" STD_EXECUTION_BS_REVISION_LINE REGEX "Revision: [0-9]+")
string(REGEX REPLACE "Revision: ([0-9]+)" "\\1" STD_EXECUTION_BS_REVISION ${STD_EXECUTION_BS_REVISION_LINE})

# nvc++ isn't supported by (s)ccache yet, so unset these before the `project()`
# call so CMake's CXX compiler detection doesn't throw attempting to use it
include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules/CompilerHelpers.cmake)

project(STDEXEC VERSION "0.${STD_EXECUTION_BS_REVISION}.0" LANGUAGES CXX)

# Print CMake configuration
message(STATUS "System           : ${CMAKE_SYSTEM}")
message(STATUS "System name      : ${CMAKE_SYSTEM_NAME}")
message(STATUS "System ver       : ${CMAKE_SYSTEM_VERSION}")
message(STATUS)

# Set the version and current build date
set(STDEXEC_VERSION "${PROJECT_VERSION}")
set(STDEXEC_VERSION_MAJOR "${PROJECT_VERSION_MAJOR}")
string(TIMESTAMP STDEXEC_BUILD_DATE "%Y-%m-%d")
string(TIMESTAMP STDEXEC_BUILD_YEAR "%Y")

message(STATUS "Library ver      : ${STDEXEC_VERSION}")
message(STATUS "Build date       : ${STDEXEC_BUILD_DATE}")
message(STATUS "Build year       : ${STDEXEC_BUILD_YEAR}")
message(STATUS)

# Integrate with LLVM/clang tooling
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

# Write the version header
rapids_cmake_write_version_file(include/stdexec_version_config.hpp)

# Set CMAKE_BUILD_TYPE=Release the default if none provided
rapids_cmake_build_type(Release)

list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules")

# Don't build tests if configuring stdexec as a submodule of another
# CMake project, unless they explicitly set STDEXEC_BUILD_TESTS=TRUE,
# or they enabled CTest's BUILD_TESTING option
if ((CMAKE_PROJECT_NAME STREQUAL PROJECT_NAME) OR BUILD_TESTING)
    set(STDEXEC_BUILD_TESTS_DEFAULT ON)
else()
    set(STDEXEC_BUILD_TESTS_DEFAULT OFF)
endif()
option(STDEXEC_BUILD_TESTS "Build stdexec tests" ${STDEXEC_BUILD_TESTS_DEFAULT})

# STDEXEC_BUILD_TESTS is used solely to configure CTest's BUILD_TESTING option,
# which is CMake's preferred option for enabling testing when using CTest.
set(BUILD_TESTING ${STDEXEC_BUILD_TESTS})

if (BUILD_TESTING)
  # CTest automatically calls enable_testing() if BUILD_TESTING is ON
  # https://cmake.org/cmake/help/latest/module/CTest.html#module:CTest
  include(CTest)
endif()

##############################################################################
# - Dependencies -------------------------------------------------------------

# Initialize CPM
rapids_cpm_init(OVERRIDE ${CMAKE_CURRENT_SOURCE_DIR}/cmake/versions.json)

if (BUILD_TESTING)
    # Add Catch2
    set(Catch2_VERSION 2.13.6)
    # Always download it, don't attempt to do `find_package(Catch2)` first
    set(CPM_DOWNLOAD_Catch2 TRUE)
    rapids_cpm_find(Catch2 ${Catch2_VERSION}
      GLOBAL_TARGETS Catch2::Catch2
      BUILD_EXPORT_SET stdexec-exports
      CPM_ARGS
        URL https://github.com/catchorg/Catch2/archive/refs/tags/v${Catch2_VERSION}.zip
    )
endif()

# Add ICM
set(icm_VERSION 1.5.0)
# Always download it, don't attempt to do `find_package(ICM)` first
set(CPM_DOWNLOAD_icm TRUE)
rapids_cpm_find(icm ${icm_VERSION}
  CPM_ARGS
    GITHUB_REPOSITORY iboB/icm
    GIT_TAG v${icm_VERSION}
    VERSION ${icm_VERSION}
    PATCH_COMMAND git restore -- . && git apply ${CMAKE_CURRENT_LIST_DIR}/cmake/cpm/patches/icm/regex-build-error.diff
)

# Ensure that we link with the threading library
set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
rapids_find_package(Threads REQUIRED
  BUILD_EXPORT_SET stdexec-exports
  INSTALL_EXPORT_SET stdexec-exports
)

##############################################################################
# - Main library targets -----------------------------------------------------

# Detect the compiler frontend (GNU, Clang, MSVC, etc.)
if(DEFINED CMAKE_CXX_COMPILER_FRONTEND_VARIANT)
  set(stdexec_compiler_frontend ${CMAKE_CXX_COMPILER_FRONTEND_VARIANT})
else()
  set(stdexec_compiler_frontend ${CMAKE_CXX_COMPILER_ID})
endif()

set(stdexec_export_targets)

# Define the main library
add_library(stdexec INTERFACE)

target_sources(stdexec
PUBLIC
  FILE_SET headers
  TYPE HEADERS
  BASE_DIRS include
  FILES
    include/exec/__detail/__atomic_intrusive_queue.hpp
    include/exec/__detail/__atomic_ref.hpp
    include/exec/__detail/__basic_sequence.hpp
    include/exec/__detail/__bit_cast.hpp
    include/exec/__detail/__bwos_lifo_queue.hpp
    include/exec/__detail/__numa.hpp
    include/exec/__detail/__system_context_default_impl.hpp
    include/exec/__detail/__system_context_default_impl_entry.hpp
    include/exec/__detail/__system_context_replaceability_api.hpp
    include/exec/__detail/__xorshift.hpp
    include/exec/__detail/intrusive_heap.hpp
    include/exec/any_sender_of.hpp
    include/exec/async_scope.hpp
    include/exec/at_coroutine_exit.hpp
    include/exec/create.hpp
    include/exec/env.hpp
    include/exec/finally.hpp
    include/exec/inline_scheduler.hpp
    include/exec/into_tuple.hpp
    include/exec/just_from.hpp
    include/exec/libdispatch_queue.hpp
    include/exec/linux/__detail/memory_mapped_region.hpp
    include/exec/linux/__detail/safe_file_descriptor.hpp
    include/exec/linux/io_uring_context.hpp
    include/exec/linux/memory_mapped_region.hpp
    include/exec/linux/safe_file_descriptor.hpp
    include/exec/materialize.hpp
    include/exec/on.hpp
    include/exec/on_coro_disposition.hpp
    include/exec/repeat_effect_until.hpp
    include/exec/repeat_n.hpp
    include/exec/reschedule.hpp
    include/exec/scope.hpp
    include/exec/sequence.hpp
    include/exec/sequence/any_sequence_of.hpp
    include/exec/sequence/empty_sequence.hpp
    include/exec/sequence/ignore_all_values.hpp
    include/exec/sequence/iterate.hpp
    include/exec/sequence/transform_each.hpp
    include/exec/sequence_senders.hpp
    include/exec/single_thread_context.hpp
    include/exec/start_now.hpp
    include/exec/static_thread_pool.hpp
    include/exec/system_context.hpp
    include/exec/task.hpp
    include/exec/timed_scheduler.hpp
    include/exec/timed_thread_scheduler.hpp
    include/exec/trampoline_scheduler.hpp
    include/exec/variant_sender.hpp
    include/exec/when_any.hpp
    include/execpools/thread_pool_base.hpp
    include/stdexec/__detail/__as_awaitable.hpp
    include/stdexec/__detail/__awaitable.hpp
    include/stdexec/__detail/__basic_sender.hpp
    include/stdexec/__detail/__bulk.hpp
    include/stdexec/__detail/__completion_signatures.hpp
    include/stdexec/__detail/__concepts.hpp
    include/stdexec/__detail/__config.hpp
    include/stdexec/__detail/__connect_awaitable.hpp
    include/stdexec/__detail/__continues_on.hpp
    include/stdexec/__detail/__cpo.hpp
    include/stdexec/__detail/__debug.hpp
    include/stdexec/__detail/__diagnostics.hpp
    include/stdexec/__detail/__domain.hpp
    include/stdexec/__detail/__ensure_started.hpp
    include/stdexec/__detail/__env.hpp
    include/stdexec/__detail/__execute.hpp
    include/stdexec/__detail/__execution_fwd.hpp
    include/stdexec/__detail/__force_include.hpp
    include/stdexec/__detail/__inline_scheduler.hpp
    include/stdexec/__detail/__into_variant.hpp
    include/stdexec/__detail/__intrusive_mpsc_queue.hpp
    include/stdexec/__detail/__intrusive_ptr.hpp
    include/stdexec/__detail/__intrusive_queue.hpp
    include/stdexec/__detail/__intrusive_slist.hpp
    include/stdexec/__detail/__just.hpp
    include/stdexec/__detail/__let.hpp
    include/stdexec/__detail/__manual_lifetime.hpp
    include/stdexec/__detail/__meta.hpp
    include/stdexec/__detail/__on.hpp
    include/stdexec/__detail/__operation_states.hpp
    include/stdexec/__detail/__optional.hpp
    include/stdexec/__detail/__ranges.hpp
    include/stdexec/__detail/__read_env.hpp
    include/stdexec/__detail/__receiver_adaptor.hpp
    include/stdexec/__detail/__receiver_ref.hpp
    include/stdexec/__detail/__receivers.hpp
    include/stdexec/__detail/__run_loop.hpp
    include/stdexec/__detail/__schedule_from.hpp
    include/stdexec/__detail/__schedulers.hpp
    include/stdexec/__detail/__scope.hpp
    include/stdexec/__detail/__sender_adaptor_closure.hpp
    include/stdexec/__detail/__sender_introspection.hpp
    include/stdexec/__detail/__senders.hpp
    include/stdexec/__detail/__senders_core.hpp
    include/stdexec/__detail/__shared.hpp
    include/stdexec/__detail/__spin_loop_pause.hpp
    include/stdexec/__detail/__split.hpp
    include/stdexec/__detail/__start_detached.hpp
    include/stdexec/__detail/__starts_on.hpp
    include/stdexec/__detail/__stop_token.hpp
    include/stdexec/__detail/__stopped_as_error.hpp
    include/stdexec/__detail/__stopped_as_optional.hpp
    include/stdexec/__detail/__submit.hpp
    include/stdexec/__detail/__sync_wait.hpp
    include/stdexec/__detail/__tag_invoke.hpp
    include/stdexec/__detail/__then.hpp
    include/stdexec/__detail/__transfer_just.hpp
    include/stdexec/__detail/__transform_completion_signatures.hpp
    include/stdexec/__detail/__transform_sender.hpp
    include/stdexec/__detail/__tuple.hpp
    include/stdexec/__detail/__type_traits.hpp
    include/stdexec/__detail/__upon_error.hpp
    include/stdexec/__detail/__upon_stopped.hpp
    include/stdexec/__detail/__utility.hpp
    include/stdexec/__detail/__variant.hpp
    include/stdexec/__detail/__when_all.hpp
    include/stdexec/__detail/__with_awaitable_senders.hpp
    include/stdexec/__detail/__write_env.hpp
    include/stdexec/concepts.hpp
    include/stdexec/coroutine.hpp
    include/stdexec/execution.hpp
    include/stdexec/functional.hpp
    include/stdexec/stop_token.hpp
  # stdexec_version_config.hpp is generated by raipds' script
  FILE_SET version_config
  TYPE HEADERS
  BASE_DIRS ${CMAKE_BINARY_DIR}/include
  FILES
    ${CMAKE_BINARY_DIR}/include/stdexec_version_config.hpp
)
list(APPEND stdexec_export_targets stdexec)

# Set library version
set_target_properties(stdexec PROPERTIES
                      VERSION "${STDEXEC_VERSION}"
                      SOVERSION "${STDEXEC_VERSION_MAJOR}")

# Declare the public include directories
include(GNUInstallDirs)

target_link_libraries(stdexec INTERFACE Threads::Threads)

# Use C++20 standard
target_compile_features(stdexec INTERFACE cxx_std_20)

# # Enable GPU compilation when using NVHPC compiler
# target_compile_options(stdexec INTERFACE
#                        $<$<COMPILE_LANG_AND_ID:CXX,NVHPC>:-stdpar=gpu>
#                        )
# target_link_options(stdexec INTERFACE
#                        $<$<LINK_LANG_AND_ID:CXX,NVHPC>:-stdpar=gpu>
#                        )

# Enable coroutines for GCC
target_compile_options(stdexec INTERFACE
                       $<$<COMPILE_LANG_AND_ID:CXX,GNU>:-fcoroutines>
                       )

# Increase the concepts diagnostics depth for GCC
target_compile_options(stdexec INTERFACE
                       $<$<COMPILE_LANG_AND_ID:CXX,GNU>:-fconcepts-diagnostics-depth=10>
                       )

# Do you want a preprocessor that works? Picky, picky.
target_compile_options(stdexec INTERFACE
                       $<$<COMPILE_LANG_AND_ID:CXX,MSVC>:/Zc:__cplusplus /Zc:preprocessor>
                       )

option(STDEXEC_ENABLE_EXTRA_TYPE_CHECKING "Enable extra type checking that is costly at compile-time" OFF)

if (STDEXEC_ENABLE_EXTRA_TYPE_CHECKING)
    target_compile_definitions(stdexec INTERFACE STDEXEC_ENABLE_EXTRA_TYPE_CHECKING)
endif()

add_library(STDEXEC::stdexec ALIAS stdexec)

# Don't require building everything when installing
set(CMAKE_SKIP_INSTALL_ALL_DEPENDENCY ON)

# Support target for examples and tests
add_library(stdexec_executable_flags INTERFACE)

# Enable warnings
target_compile_options(stdexec_executable_flags INTERFACE
                       $<$<STREQUAL:${stdexec_compiler_frontend},GNU>:-Wall>
                       $<$<STREQUAL:${stdexec_compiler_frontend},AppleClang>:-Wall>
                       $<$<STREQUAL:${stdexec_compiler_frontend},MSVC>:/W4>)

# Increase the error limit with NVC++
target_compile_options(stdexec_executable_flags INTERFACE
                       $<$<CXX_COMPILER_ID:NVHPC>:-e1000>)

# Silence warnings
target_compile_options(stdexec_executable_flags INTERFACE
                       $<$<CXX_COMPILER_ID:GNU>:-Wno-non-template-friend>
                       $<$<CXX_COMPILER_ID:NVHPC>:--diag_suppress177,550,111,497,554>
                       $<$<CXX_COMPILER_ID:MSVC>:/wd4100 /wd4101 /wd4127 /wd4324 /wd4456 /wd4459>)

# Template backtrace limit
target_compile_options(stdexec_executable_flags INTERFACE
                       $<$<OR:$<CXX_COMPILER_ID:Clang>,$<CXX_COMPILER_ID:AppleClang>>:
                          $<$<STREQUAL:${CMAKE_CXX_COMPILER_FRONTEND_VARIANT},MSVC>:/clang:>-ferror-limit=0
                          $<$<STREQUAL:${CMAKE_CXX_COMPILER_FRONTEND_VARIANT},MSVC>:/clang:>-fmacro-backtrace-limit=0
                          $<$<STREQUAL:${CMAKE_CXX_COMPILER_FRONTEND_VARIANT},MSVC>:/clang:>-ftemplate-backtrace-limit=0>
                       $<$<AND:$<CXX_COMPILER_ID:NVHPC>,$<VERSION_GREATER:$<CXX_COMPILER_VERSION>,23.3.0>>:
                       -ftemplate-backtrace-limit=0>)

# # Always enable colored output
# target_compile_options(stdexec_executable_flags INTERFACE
#                        $<$<OR:$<CXX_COMPILER_ID:Clang>,$<CXX_COMPILER_ID:AppleClang>>:
#                        -fcolor-diagnostics>
#                        $<$<CXX_COMPILER_ID:GNU>:-fdiagnostics-color=always>
#                        )

# Clang CUDA options
target_compile_options(stdexec_executable_flags INTERFACE
                       $<$<COMPILE_LANG_AND_ID:CUDA,Clang>:
                       -Wno-unknown-cuda-version
                       -Xclang=-fcuda-allow-variadic-functions
                       -D_GLIBCXX_USE_TBB_PAR_BACKEND=0
                       -include stdexec/__detail/__force_include.hpp>
                       )

target_compile_definitions(
  stdexec_executable_flags INTERFACE
  $<$<NOT:$<AND:$<CXX_COMPILER_ID:NVHPC>,$<COMPILE_LANGUAGE:CXX>>>:STDEXEC_ENABLE_EXTRA_TYPE_CHECKING>)

# Support target for examples and tests
add_library(nvexec_executable_flags INTERFACE)

target_compile_options(nvexec_executable_flags INTERFACE
                       $<$<AND:$<CXX_COMPILER_ID:NVHPC>,$<COMPILE_LANGUAGE:CXX>>:-gpu=nomanaged>)
target_link_options(nvexec_executable_flags INTERFACE
                    $<$<AND:$<CXX_COMPILER_ID:NVHPC>,$<COMPILE_LANGUAGE:CXX>>:-gpu=nomanaged>)

# Set up nvexec library
option(STDEXEC_ENABLE_CUDA "Enable CUDA targets for non-nvc++ compilers" OFF)
if(CMAKE_CXX_COMPILER_ID STREQUAL "NVHPC")
    set(STDEXEC_ENABLE_CUDA ON)
    # Unset these if using nvc++
    disable_compiler(LANG CUDA)
endif()

if(STDEXEC_ENABLE_CUDA)

    add_library(nvexec INTERFACE)
    target_sources(nvexec
    PUBLIC
      FILE_SET headers
      TYPE HEADERS
      BASE_DIRS include
      FILES
        include/nvexec/detail/config.cuh
        include/nvexec/detail/cuda_atomic.cuh
        include/nvexec/detail/cuda_fwd.cuh
        include/nvexec/detail/memory.cuh
        include/nvexec/detail/queue.cuh
        include/nvexec/detail/throw_on_cuda_error.cuh
        include/nvexec/detail/variant.cuh
        include/nvexec/multi_gpu_context.cuh
        include/nvexec/nvtx.cuh
        include/nvexec/stream/algorithm_base.cuh
        include/nvexec/stream/bulk.cuh
        include/nvexec/stream/common.cuh
        include/nvexec/stream/continues_on.cuh
        include/nvexec/stream/ensure_started.cuh
        include/nvexec/stream/launch.cuh
        include/nvexec/stream/let_xxx.cuh
        include/nvexec/stream/reduce.cuh
        include/nvexec/stream/schedule_from.cuh
        include/nvexec/stream/split.cuh
        include/nvexec/stream/start_detached.cuh
        include/nvexec/stream/submit.cuh
        include/nvexec/stream/sync_wait.cuh
        include/nvexec/stream/then.cuh
        include/nvexec/stream/upon_error.cuh
        include/nvexec/stream/upon_stopped.cuh
        include/nvexec/stream/when_all.cuh
        include/nvexec/stream_context.cuh
    )
    list(APPEND stdexec_export_targets nvexec)
    add_library(STDEXEC::nvexec ALIAS nvexec)

    target_compile_features(nvexec INTERFACE cuda_std_20)
    target_link_libraries(nvexec INTERFACE STDEXEC::stdexec)

    target_compile_options(nvexec INTERFACE
      $<$<AND:$<CXX_COMPILER_ID:NVHPC>,$<COMPILE_LANGUAGE:CXX>>:-stdpar -gpu=cc${CMAKE_CUDA_ARCHITECTURES}>)
    target_link_options(nvexec INTERFACE
      $<$<AND:$<CXX_COMPILER_ID:NVHPC>,$<COMPILE_LANGUAGE:CXX>>:-stdpar -gpu=cc${CMAKE_CUDA_ARCHITECTURES}>)

    install(TARGETS nvexec
        EXPORT stdexec-exports
        FILE_SET headers)

    if(NOT (CMAKE_CXX_COMPILER_ID STREQUAL "NVHPC"))
        include(rapids-cuda)
        # Needs to run before `enable_language(CUDA)`
        rapids_cuda_init_architectures(STDEXEC)
        enable_language(CUDA)
        # Since stdexec only enables CUDA optionally we need to manually include
        # the file that rapids_cuda_init_architectures relies on `project` calling
        if(CMAKE_PROJECT_STDEXEC_INCLUDE)
            include("${CMAKE_PROJECT_STDEXEC_INCLUDE}")
        endif()
        # Must come after enable_language(CUDA)
        # Use `-isystem <path>` instead of `-isystem=<path>`
        # because the former works with clangd intellisense
        set(CMAKE_INCLUDE_SYSTEM_FLAG_CUDA "-isystem ")

        set_source_files_properties(${nvexec_sources} PROPERTIES LANGUAGE CUDA)

        include(rapids-find)
        rapids_find_package(
            CUDAToolkit REQUIRED
            BUILD_EXPORT_SET stdexec-exports
            INSTALL_EXPORT_SET stdexec-exports
        )

        target_link_libraries(nvexec INTERFACE CUDA::cudart)

        include(${rapids-cmake-dir}/cpm/cccl.cmake)
        rapids_cpm_cccl(
            BUILD_EXPORT_SET stdexec-exports
            INSTALL_EXPORT_SET stdexec-exports
            SYSTEM TRUE)

        target_link_libraries(stdexec INTERFACE CCCL::CCCL)
    endif ()
endif ()


option(STDEXEC_ENABLE_TBB "Enable TBB targets" OFF)

if (STDEXEC_ENABLE_TBB)
    include(rapids-find)
    rapids_find_package(
        TBB REQUIRED
        BUILD_EXPORT_SET stdexec-exports
        INSTALL_EXPORT_SET stdexec-exports
    )

    add_library(tbbpool INTERFACE)
    target_sources(tbbpool
    PUBLIC
      FILE_SET headers
      TYPE HEADERS
      BASE_DIRS include
      FILES
        include/tbbexec/tbb_thread_pool.hpp
	include/execpools/tbb/tbb_thread_pool.hpp
    )
    list(APPEND stdexec_export_targets tbbpool)
    add_library(STDEXEC::tbbpool ALIAS tbbpool)

    target_link_libraries(tbbpool
        INTERFACE
        STDEXEC::stdexec
        TBB::tbb
        )

    install(TARGETS tbbpool
        EXPORT stdexec-exports
        FILE_SET headers)
endif()

option(STDEXEC_ENABLE_TASKFLOW "Enable TaskFlow targets" OFF)

if(STDEXEC_ENABLE_TASKFLOW)
    include(rapids-find)
    rapids_cpm_find(Taskflow 3.7.0
    CPM_ARGS
      GITHUB_REPOSITORY taskflow/taskflow
      GIT_TAG v3.7.0
    )
    file(GLOB_RECURSE taskflow_pool include/execpools/taskflow/*.hpp)
    add_library(taskflow_pool INTERFACE)
    target_sources(taskflow_pool
    PUBLIC
      FILE_SET headers
      TYPE HEADERS
      BASE_DIRS include
      FILES
        include/execpools/taskflow/taskflow_thread_pool.hpp
    )
    list(APPEND stdexec_export_targets taskflow_pool)
    add_library(STDEXEC::taskflow_pool ALIAS taskflow_pool)
    target_link_libraries(taskflow_pool
        INTERFACE
        STDEXEC::stdexec
        Taskflow
    )
    install(TARGETS taskflow_pool
        EXPORT stdexec-exports
        FILE_SET headers)
endif()

option(STDEXEC_ENABLE_ASIO "Enable Boost targets" OFF)
set(STDEXEC_ASIO_IMPLEMENTATION "boost" CACHE STRING "boost")
set_property(CACHE STDEXEC_ASIO_IMPLEMENTATION PROPERTY STRINGS boost standalone)

if(STDEXEC_ENABLE_ASIO)
    set(STDEXEC_ASIO_USES_ASIO FALSE)
    set(STDEXEC_ASIO_USES_STANDALONE FALSE)

    include(rapids-find)
    if(${STDEXEC_ASIO_IMPLEMENTATION} STREQUAL "boost")
      set(STDEXEC_ASIO_USES_BOOST TRUE)
    elseif(${STDEXEC_ASIO_IMPLEMENTATION} STREQUAL "standalone")
      set(STDEXEC_ASIO_USES_STANDALONE TRUE)
    else()
      message(FATAL_ERROR "Unknown configuration for ASIO implementation: " ${STDEXEC_ASIO_IMPLEMENTATION})
    endif()

    file(GLOB_RECURSE boost_pool_sources include/execpools/asio/*.hpp)
    set(STDEXEC_ASIO_CONFIG_FILE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/include/execpools/asio)
    configure_file(include/execpools/asio/asio_config.hpp.in ${STDEXEC_ASIO_CONFIG_FILE_PATH}/asio_config.hpp)

    if(${STDEXEC_ASIO_USES_BOOST})
      set(BOOST_ENABLE_COMPATIBILITY_TARGETS TRUE)
      rapids_cpm_find(Boost 1.86.0
      CPM_ARGS
        GITHUB_REPOSITORY boostorg/boost
        GIT_TAG boost-1.86.0
      )
      add_library(stdexec_boost_pool INTERFACE)
      target_sources(stdexec_boost_pool
      PUBLIC
        FILE_SET headers
        TYPE HEADERS
        BASE_DIRS include
        FILES
          include/execpools/asio/asio_config.hpp.in
          include/execpools/asio/asio_thread_pool.hpp
      )

      list(APPEND stdexec_export_targets stdexec_boost_pool)
      add_library(STDEXEC::asio_pool ALIAS stdexec_boost_pool)
      target_link_libraries(stdexec_boost_pool
          INTERFACE
          STDEXEC::stdexec
          Boost::boost
      )
      install(TARGETS stdexec_boost_pool
          EXPORT stdexec-exports
          FILE_SET headers)
    elseif(${STDEXEC_ASIO_USES_STANDALONE})
      include(cmake/import_standalone_asio.cmake)
      import_standalone_asio(
        TAG "asio-1-31-0"
        VERSION "1.31.0")

      add_library(stdexec_asio_pool INTERFACE)
      target_sources(stdexec_asio_pool
      PUBLIC
        FILE_SET headers
        TYPE HEADERS
        BASE_DIRS include
        FILES
          include/execpools/asio/asio_config.hpp.in
          include/execpools/asio/asio_thread_pool.hpp
      )

      list(APPEND stdexec_export_targets stdexec_asio_pool)
      add_library(STDEXEC::asio_pool ALIAS stdexec_asio_pool)

      target_link_libraries(stdexec_asio_pool
          INTERFACE
          STDEXEC::stdexec
          asio
      )
      install(TARGETS stdexec_asio_pool
          EXPORT stdexec-exports
          FILE_SET headers)
    else()
      message(FATAL_ERROR "ASIO implementation is not configured")
    endif()
endif()

include(CheckIncludeFileCXX)
if (CMAKE_SYSTEM_NAME STREQUAL "Darwin")
  CHECK_INCLUDE_FILE_CXX("dispatch/dispatch.h" STDEXEC_FOUND_LIBDISPATCH)
  option(STDEXEC_ENABLE_LIBDISPATCH "Enable the tests for the Grand Central Dispatch scheduler" ${STDEXEC_FOUND_LIBDISPATCH})
endif()

option (STDEXEC_ENABLE_NUMA "Enable NUMA affinity for static_thread_pool" OFF)
if (STDEXEC_ENABLE_NUMA)
  find_package(numa REQUIRED)
  target_link_libraries(stdexec INTERFACE numa::numa)
  target_compile_definitions(stdexec INTERFACE STDEXEC_ENABLE_NUMA)
endif()

set(SYSTEM_CONTEXT_SOURCES src/system_context/system_context.cpp)
add_library(system_context STATIC ${SYSTEM_CONTEXT_SOURCES})
target_compile_features(system_context PUBLIC cxx_std_20)
set_target_properties(system_context PROPERTIES
    CXX_STANDARD 20
    CXX_STANDARD_REQUIRED ON
    CXX_EXTENSIONS OFF)
target_compile_options(system_context PUBLIC
    $<$<COMPILE_LANG_AND_ID:CXX,MSVC>:/Zc:__cplusplus /Zc:preprocessor>
    )
add_library(STDEXEC::system_context ALIAS system_context)
target_link_libraries(system_context PUBLIC stdexec)


if(CMAKE_CROSSCOMPILING)
  CHECK_INCLUDE_FILE_CXX("linux/io_uring.h" STDEXEC_FOUND_IO_URING)
else()
  include(CheckCXXSourceRuns)
  CHECK_CXX_SOURCE_RUNS(
    "
    #include <linux/io_uring.h>
    #include <sys/syscall.h>
    #include <unistd.h>

    #include <cstdlib>

    int main()
    {
      io_uring_params a = {};
      return syscall(__NR_io_uring_setup, 1, &a) != -1
        ? EXIT_SUCCESS
        : EXIT_FAILURE;
    }
    "
    STDEXEC_FOUND_IO_URING)
endif()
option(STDEXEC_ENABLE_IO_URING_TESTS "Enable io_uring tests" ${STDEXEC_FOUND_IO_URING})

option(STDEXEC_BUILD_DOCS "Build stdexec documentation" OFF)
option(STDEXEC_BUILD_EXAMPLES "Build stdexec examples" ON)

# Configure documentation
if(STDEXEC_BUILD_DOCS)
    add_subdirectory(docs)
endif()

# Configure test executables
if(BUILD_TESTING)
    add_subdirectory(test)
endif()

# Configure example executables
if(STDEXEC_BUILD_EXAMPLES)
    add_subdirectory(examples)
endif()

##############################################################################
# Install targets ------------------------------------------------------------

include(CPack)

install(TARGETS stdexec system_context
	EXPORT stdexec-exports
	FILE_SET headers
	FILE_SET version_config)

##############################################################################
# Install exports ------------------------------------------------------------

set(code_string "")

# Install side of the export set
rapids_export(
  INSTALL ${stdexec_export_targets}
  EXPORT_SET stdexec-exports
  GLOBAL_TARGETS ${stdexec_export_targets}
  NAMESPACE STDEXEC::
  FINAL_CODE_BLOCK code_string
)

# Build side of the export set so a user can use the build dir as a CMake package root
rapids_export(
  BUILD ${stdexec_export_targets}
  EXPORT_SET stdexec-exports
  GLOBAL_TARGETS ${stdexec_export_targets}
  NAMESPACE STDEXEC::
  FINAL_CODE_BLOCK code_string
)

The problem is about upstream libraries doesn't have FILE_SET

CMake Error: install(EXPORT "stdexec-exports" ...) includes target "stdexec_boost_pool" which requires target "boost_comptarget_boost" that is not in any export set.
CMake Error in CMakeLists.txt:
export called with target "stdexec_boost_pool" which requires target
"boost_comptarget_boost" that is not in any export set

and

CMake Error: install(EXPORT "stdexec-exports" ...) includes target "stdexec_asio_pool" which requires target "asio" that is not in any export set.
CMake Error in CMakeLists.txt:
export called with target "stdexec_asio_pool" which requires target "asio"
that is not in any export set.

I don't know how to handle it correctly. Do you have any experience?

@bretbrownjr
Copy link

Sorry for the delay, but I found a few minutes to look at this.

I took the head branch from this PR, and did a build and install procedure with the following:

  • Ubuntu @ 22.04
  • System g++ @ 11.4.0-1ubuntu1~22.04
  • CMake @ 3.30.5 from snap

I ran the following:

git clone https://github.com/HungMingWu/stdexec -b FILE_SET
cd stdexec
export CMAKE_BUILD_TYPE=Debug
export CMAKE_GENERATOR=Ninj
cmake -B build -S .
cmake --build build
ctest --test-dir build
DESTDIR=../staging cmake --install build --prefix /opt/HungMingWu

CMake succeeded (with some deprecation warnings about CPM using a deprecated CMake FetchContent API). The build succeeded (with a warning or two). And the install succeeded as best as I can tell. Here is what is installed:
stdexec-install.log.

Note that both CPM and this rapids CMake module (I'm not familiar) are not standard CMake. It's possible one or both have defects with respect to supporting exports. I wouldn't expect FILE_SETS to come into it since FILE_SETs are mostly about building against and installing headers specifically, though I don't have deep knowledge of either of those modules. They could do clever things with header files and dependencies I suppose?

@bretbrownjr
Copy link

If there's another branch to look at or a different build workflow to try, I could use those details.

Otherwise, you can get all of the detail you want from a call to cmake if you use the --trace or --trace-expand flags. They are very, very verbose, so be sure to redirect your output to a log of some sort and be prepared to search through the text for references to things like boost_comptarget_boost. Those aren't normal target names, so I'm guessing someone is doing something clever somewhere that is breaking generation of CMake modules for stdexec, at least for your workflow. Reading through the files generated for me when I build the head commit of this PR, it's likely that's the rapid CMake modules since they seem to wrap/extend/replace what CMake supports out of the box.

@HungMingWu
Copy link
Contributor Author

Sorry, but i haven't push the newest commit to GitHub, the original stdexec now support boost and asio, but my branch isn't, you may overwrite CMakeLists.txt describe above, or I create another branch to test it.

@HungMingWu
Copy link
Contributor Author

@bretbrownjr I create a new branch, now you can test it from following command

$ git clone https://github.com/HungMingWu/stdexec -b FILE_SET_BOOST
$ cmake -S . -B build/ -DCMAKE_INSTALL_PREFIX=~/test_build -DSTDEXEC_ENABLE_ASIO=ON
or 
$ cmake -S . -B build/ -DCMAKE_INSTALL_PREFIX=~/test_build  -DSTDEXEC_ENABLE_ASIO=ON -DSTDEXEC_ASIO_IMPLEMENTATION=standalone

If you remove

      install(TARGETS stdexec_boost_pool
          EXPORT stdexec-exports
          FILE_SET headers)

and

      install(TARGETS stdexec_asio_pool
          EXPORT stdexec-exports
          FILE_SET headers)

It can compiles but fail to install.

@ericniebler
Copy link
Collaborator

@HungMingWu @bretbrownjr i don't know what to do with this pr. @bretbrownjr i think you were suggesting to use a file glob to build the FILE_SET, is that correct? can you point me to a project that does that?

@bretbrownjr
Copy link

I'm unclear on what difficulties @HungMingWu is having. It appears to be some difficulty in getting the rapids library/framework to behave nicely with expected target names. I was hoping I could apply the CMakeLists.txt that @HungMingWu provided to the HEAD of this PR and reproduce the same failure. Unfortunately, I found some missing header errors instead, which looks like some other rebase details weren't provided. @HungMingWu: it makes sense to push a full branch somewhere if you still want help with that specific issue. But my instinct is that rapids has a bug or otherwise isn't supporting CMake fully somehow.

But perhaps good news is that I did manage to manually rebase the HEAD of this PR onto the main branch as of today. It seems to build, test, and install fine. I went ahead and did a best-effort GLOB_RECURSE for stdexec, exec, tbbpool and nvexec headers, incorporating them into relevant FILE_SETs. I have a diff between the HEAD of this PR and my local version attached to this comment. @ericniebler, hopefully the diff works as a demonstration of what you're looking

I also installed the project into a staging/opt/stdexec directory and ran tree. I've attached that log as well. I don't feel comfortable eyeballing the output definitively, but it looks about right? I'm unaware of any test procedure to validate stdexec as installed. I generally do that sort of validation in the context of a packaging system at $dayjob.

NB: Conan has support for test packages to help prevent regressions in these sorts of situations. Not to sidetrack this PR.

stdexec.tree.txt
stdexec-rebase-diff.txt
(Pardon the *.txt extension -- GitHub won't let me attach *.log or *.diff files but *.txt files are fine?)

@bretbrownjr
Copy link

bretbrownjr commented Oct 25, 2024

OK. Why not save a click? One out of about three places where you'd likely want to glob could look like this. It's the most complicated example, so might be the most important want to demonstrate.

file(GLOB_RECURSE exec_headers CONFIGURE_DEPENDS include/exec/*.hpp)
file(GLOB_RECURSE stdexec_headers CONFIGURE_DEPENDS include/stdexec/*.hpp)
target_sources(stdexec
PUBLIC
  FILE_SET headers
  TYPE HEADERS
  BASE_DIRS include
  FILES
    ${exec_headers}
    ${stdexec_headers}
  # stdexec_version_config.hpp is generated by raipds' script
  FILE_SET version_config
  TYPE HEADERS
  BASE_DIRS ${CMAKE_BINARY_DIR}/include
  FILES
    ${CMAKE_BINARY_DIR}/include/stdexec_version_config.hpp
)

I'm making some assumptions about how you would expect globs to be applied. You can do non-recursive globbing. It's cleaner if it works for your expectations. I also thought the one-off version file doesn't need a glob, but you can glob that as well if it's interesting. Note that it's in the binary directory and not in the source directory, so it needs its own detection logic.

@bretbrownjr
Copy link

Oops:

s/CMAKE_CONFIGURE_DEPENDS/CONFIGURE_DEPENDS/ on those examples. I already fixed the inline example, but the diff still needs that tweak. It's enough to make me wish for a build system with better syntax checking.....

@ericniebler
Copy link
Collaborator

/ok to test

@ericniebler
Copy link
Collaborator

/ok to test

@ericniebler
Copy link
Collaborator

thanks bret. i have integrated your changes. if you get a sec, could you pls look over [a1aee75] and let me know if i screwed anything up.

@HungMingWu i did a rather rude thing: i force pushed to your FILE_SET branch. beware.

@HungMingWu
Copy link
Contributor Author

HungMingWu commented Oct 26, 2024

@bretbrownjr , I rebase from stdexec main branch and use your patch and force push my FILE_SET branch.
And I find something wrong.
Here is my test steps.

$ git clone https://github.com/HungMingWu/stdexec
$ cmake -S . -B build/ -DCMAKE_INSTALL_PREFIX=~/test_build -DSTDEXEC_ENABLE_ASIO=ON
$ cmake --build build/
$ cmake --install build/

Here is the result from main branch

-- Install configuration: "Release"
-- Installing: /home/hm/test_build/lib/libsystem_context.a
-- Installing: /home/hm/test_build/include
-- Installing: /home/hm/test_build/include/execpools
-- Installing: /home/hm/test_build/include/execpools/thread_pool_base.hpp
-- Installing: /home/hm/test_build/include/execpools/asio
-- Installing: /home/hm/test_build/include/execpools/asio/asio_config.hpp
-- Installing: /home/hm/test_build/include/execpools/asio/asio_config.hpp.in
-- Installing: /home/hm/test_build/include/execpools/asio/asio_thread_pool.hpp
-- Installing: /home/hm/test_build/include/execpools/taskflow
-- Installing: /home/hm/test_build/include/execpools/taskflow/taskflow_thread_pool.hpp
-- Installing: /home/hm/test_build/include/execpools/tbb
-- Installing: /home/hm/test_build/include/execpools/tbb/tbb_thread_pool.hpp
-- Installing: /home/hm/test_build/include/stdexec
-- Installing: /home/hm/test_build/include/stdexec/__detail

Here is the result from FILE_SET branch

-- Install configuration: "Release"
-- Installing: /home/hm/test_build/include/exec/any_sender_of.hpp
-- Installing: /home/hm/test_build/include/exec/async_scope.hpp
-- Installing: /home/hm/test_build/include/exec/at_coroutine_exit.hpp
-- Installing: /home/hm/test_build/include/exec/create.hpp
-- Installing: /home/hm/test_build/include/exec/env.hpp
-- Installing: /home/hm/test_build/include/exec/finally.hpp
-- Installing: /home/hm/test_build/include/exec/inline_scheduler.hpp
-- Installing: /home/hm/test_build/include/exec/into_tuple.hpp

You can find no any execpools headers install.
But if you watch my FILE_SET_BOOST branch, I install them into three parts.

    install(TARGETS tbbpool
        EXPORT stdexec-exports
        FILE_SET headers)
      install(TARGETS stdexec_boost_pool
          EXPORT stdexec-exports
          FILE_SET headers)
      install(TARGETS stdexec_asio_pool
          EXPORT stdexec-exports
          FILE_SET headers)

install(tbbpool...) works well, but failed on install(stdexec_boost_pool...) and install(stdexec_asio_pool...) .
I can remove the install target, but the result is different from main branch.
Here is my problem meet right now.
I don't know how to handle 3rd party's FILE_SET like this

CMake Error: install(EXPORT "stdexec-exports" ...) includes target "stdexec_boost_pool" which requires target "boost_comptarget_boost" that is not in any export set.
CMake Error in CMakeLists.txt:
export called with target "stdexec_boost_pool" which requires target
"boost_comptarget_boost" that is not in any export set

@ericniebler
Copy link
Collaborator

/ok to test

@ericniebler
Copy link
Collaborator

@HungMingWu pls ignore the macos CI failures for now. they are unrelated. the other failures need to be addressed.

i'll add that this PR is looking very good now.

CMakeLists.txt Outdated
FILES
${exec_headers}
${stdexec_headers}
# stdexec_version_config.hpp is generated by raipds' script

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like typo?

Suggested change
# stdexec_version_config.hpp is generated by raipds' script
# stdexec_version_config.hpp is generated by rapids' script

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like typo?
Yes, I will fix it.

@ericniebler
Copy link
Collaborator

/ok to test

@ericniebler ericniebler merged commit c211de1 into NVIDIA:main Dec 30, 2024
18 checks passed
@ericniebler
Copy link
Collaborator

@HungMingWu thank you very much for this PR. i may reach out to you if this causes any tooling problems.

@HungMingWu HungMingWu deleted the FILE_SET branch December 31, 2024 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants