(#6891) Add nmos-cpp

* nmos-cpp recipe files copied from sony/nmos-cpp#195

* Add config.yml for current tip of garethsb:library-cmake

* Attempt to install Avahi Apple Bonjour compatibility library on Linux

* Provide with_dnssd options based on #6928, #6930, #6937

* Fix typo in prev commit

* Simplify selection of appropriate requirements for DNS-SD

* For now, keep default option as mdnsresponder on Windows and Linux

* Leave most exceptions uncaught as they really aren't expected in the test_package (cf. the best practice shown in nmos-cpp-node example app) but do catch web::http::http_exception since the most likely cause of that is a port being in use or, on Windows with cpprestsdk built with its default settings, that the test_package wasn't run as administrator, which WinHTTP enforces to listen on the wildcard hostname.
Uncaught exceptions, via std::terminate, end up as STATUS_STACK_BUFFER_OVERRUN on Windows (see https://devblogs.microsoft.com/oldnewthing/20190108-00/?p=100655), which Conan reports as "ConanException: Error 3221226505 while executing bin\test_package", which might take one a while to understand, if one didn't already know all of this...

* Update to head revision of garethsb:library-cmake and latest mdnsresponder (#6930) and avahi (#7049) from-source recipes, since those seem more likely to be approved on CCI

* Better comment

* Switch to upstream sony/nmos-cpp
(sony/nmos-cpp#195 was merged to master)

* self.options.get_safe("with_dnssd")

* For now, to build on CCI, don't require the opt-in patches

* Build requires CMake 3.17

* Hopefully using short paths will resolve "fatal error C1083: Cannot open compiler generated file: '': Invalid argument" on some Windows builds on CCI

* Test package requires C++11

* int64_t should work everywhere... I think

* Test proposed upstream fix to make relocatable package per https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#creating-relocatable-packages

* Fix handling of ${_IMPORT_PREFIX}

* Fix handling of ${_IMPORT_PREFIX} and add tools.check_min_cppstd

* No version ranges

* Log all exceptions
to debug intermittent test_package failure on arch=x86_64, build_type=Release, compiler=Visual Studio, compiler.runtime=MT, compiler.version=14, os=Windows

* Additional warning

* Take upstream latest which merged the relocatable package fix (cf. bf1388a)

* Update conanfile.py

* Apply suggestions from code review (properly)

Co-authored-by: Anonymous Maarten <[email protected]>

* Apply suggestions from code review

Co-authored-by: Anonymous Maarten <[email protected]>

* Apply suggestions from code review

Co-authored-by: Anonymous Maarten <[email protected]>

* self.options["mdnsresponder"].with_opt_patches = True

* Apply suggestions from code review

Co-authored-by: Uilian Ries <[email protected]>

* cmake/3.21.1 conflicts with openssl/1.1.1l

* Update topics

* Try bump openssl/1.1.1l and cmake/3.21.2

* Take upstream latest, now built and tested with OpenSSL v1.1.1L

Co-authored-by: Anonymous Maarten <[email protected]>
Co-authored-by: Uilian Ries <[email protected]>

Author: 3 people

recipes/nmos-cpp/all/CMakeLists.txt

@@ -0,0 +1,10 @@
+# see https://github.com/conan-io/cmake-conan/issues/249#issuecomment-737011732
+cmake_minimum_required(VERSION 3.17)
+project(cmake_wrapper)
+
+include(conanbuildinfo.cmake)
+conan_basic_setup()
+
+# conanfile.py source() method extracts to source_subfolder
+# nmos-cpp top-level CMakeLists.txt is in Development
+add_subdirectory("source_subfolder/Development")

recipes/nmos-cpp/all/conandata.yml

@@ -0,0 +1,5 @@
+sources:
+  # see https://github.com/conan-io/conan-center-index/blob/master/docs/faqs.md#what-version-should-packages-use-for-libraries-without-official-releases
+  "cci.20210902":
+    url: "https://github.com/sony/nmos-cpp/archive/d3a8c7935f80ce5de6d4727c307ddcef667b5d57.tar.gz"
+    sha256: "f5bd0b96542810ac1ed3ebaf7237d7471f3c42e68d966ae8b2d39db2601a38f9"

recipes/nmos-cpp/all/conanfile.py

@@ -0,0 +1,250 @@
+import json
+import os
+import re
+from conans import ConanFile, CMake, tools
+
+required_conan_version = ">=1.33.0"
+
+class NmosCppConan(ConanFile):
+    name = "nmos-cpp"
+    description = "An NMOS C++ Implementation"
+    license = "Apache-2.0"
+    url = "https://github.com/conan-io/conan-center-index"
+    homepage = "https://github.com/sony/nmos-cpp"
+    topics = ("amwa", "nmos", "is-04", "is-05", "is-07", "is-08", "is-09", "broadcasting", "network", "media")
+
+    settings = "os", "compiler", "build_type", "arch"
+    # for now, no "shared" option support
+    options = {
+        "fPIC": [True, False],
+        "with_dnssd": ["mdnsresponder", "avahi"],
+    }
+    # "fPIC" is handled automatically by Conan, injecting CMAKE_POSITION_INDEPENDENT_CODE
+    default_options = {
+        "fPIC": True,
+        "with_dnssd": "mdnsresponder",
+    }
+
+    # wrapper CMakeLists.txt to call conan_basic_setup()
+    exports_sources = ["CMakeLists.txt"]
+    # use cmake_find_package_multi and prefer config-file packages
+    generators = "cmake", "cmake_find_package_multi"
+
+    short_paths = True
+
+    _cmake = None
+
+    # for out-of-source build, cf. wrapper CMakeLists.txt
+    @property
+    def _source_subfolder(self):
+        return "source_subfolder"
+
+    @property
+    def _build_subfolder(self):
+        return "build_subfolder"
+
+    def config_options(self):
+        if self.settings.os == "Windows":
+            del self.options.fPIC
+
+        if self.settings.os == "Macos":
+            del self.options.with_dnssd
+        elif self.settings.os == "Linux":
+            self.options.with_dnssd = "avahi"
+        elif self.settings.os == "Windows":
+            self.options.with_dnssd = "mdnsresponder"
+
+    def requirements(self):
+        # for now, consistent with project's conanfile.txt
+        self.requires("boost/1.76.0")
+        self.requires("cpprestsdk/2.10.18")
+        self.requires("websocketpp/0.8.2")
+        self.requires("openssl/1.1.1l")
+        self.requires("json-schema-validator/2.1.0")
+
+        if self.options.get_safe("with_dnssd") == "mdnsresponder":
+            self.requires("mdnsresponder/878.200.35")
+            # The option mdnsresponder:with_opt_patches=True is recommended in order to permit the
+            # over-long service type _nmos-registration._tcp used in IS-04 v1.2, and also to enable
+            # support for unicast DNS-SD on Linux, since NMOS recommends this in preference to mDNS.
+            # See https://specs.amwa.tv/is-04/releases/v1.3.1/docs/3.1._Discovery_-_Registered_Operation.html#dns-sd-advertisement
+        elif self.options.get_safe("with_dnssd") == "avahi":
+            self.requires("avahi/0.8")
+
+    def build_requirements(self):
+        # nmos-cpp needs CMake 3.17 or higher but CCI doesn't allow version ranges
+        self.build_requires("cmake/3.21.2")
+
+    def validate(self):
+        if self.settings.compiler.get_safe("cppstd"):
+            tools.check_min_cppstd(self, 11)
+
+    def source(self):
+        tools.get(**self.conan_data["sources"][self.version],
+                  destination=self._source_subfolder, strip_root=True)
+
+    def _configure_cmake(self):
+        if self._cmake:
+            return self._cmake
+        self._cmake = CMake(self)
+        # prefer config-file packages created by cmake_find_package_multi
+        # over any system-installed find-module packages
+        self._cmake.definitions["CMAKE_FIND_PACKAGE_PREFER_CONFIG"] = True
+        # (on Linux) select Avahi or mDNSResponder
+        self._cmake.definitions["NMOS_CPP_USE_AVAHI"] = self.options.get_safe("with_dnssd") == "avahi"
+        # (on Windows) use the Conan package for DNSSD (mdnsresponder), not the project's own DLL stub library
+        self._cmake.definitions["NMOS_CPP_USE_BONJOUR_SDK"] = True
+        # no need to build unit tests
+        self._cmake.definitions["NMOS_CPP_BUILD_TESTS"] = False
+        # the examples (nmos-cpp-registry and nmos-cpp-node) are useful utilities for users
+        self._cmake.definitions["NMOS_CPP_BUILD_EXAMPLES"] = True
+        # out-of-source build
+        self._cmake.configure(build_folder=self._build_subfolder)
+        return self._cmake
+
+    def build(self):
+        cmake = self._configure_cmake()
+        cmake.build()
+
+    def package(self):
+        self.copy("LICENSE", dst="licenses", src=self._source_subfolder)
+        cmake = self._configure_cmake()
+        cmake.install()
+        cmake_folder = os.path.join(self.package_folder, "lib", "cmake")
+        self._create_components_file_from_cmake_target_file(os.path.join(cmake_folder, "nmos-cpp", "nmos-cpp-targets.cmake"))
+        # remove the project's own generated config-file package
+        tools.rmdir(cmake_folder)
+
+    # based on abseil recipe
+    # see https://github.com/conan-io/conan-center-index/blob/master/recipes/abseil/all/conanfile.py
+    def _create_components_file_from_cmake_target_file(self, target_file_path):
+        components = {}
+
+        target_content = tools.load(target_file_path)
+
+        cmake_functions = re.findall(r"(?P<func>add_library|set_target_properties)[\n|\s]*\([\n|\s]*(?P<args>[^)]*)\)", target_content)
+        for (cmake_function_name, cmake_function_args) in cmake_functions:
+            cmake_function_args = re.split(r"[\s|\n]+", cmake_function_args, maxsplit=2)
+
+            cmake_imported_target_name = cmake_function_args[0]
+            cmake_target_nonamespace = cmake_imported_target_name.replace("nmos-cpp::", "")
+            component_name = cmake_target_nonamespace.lower()
+            # Conan component name cannot be the same as the package name
+            if component_name == "nmos-cpp":
+                component_name = "nmos-cpp-lib"
+
+            components.setdefault(component_name, {"cmake_target": cmake_target_nonamespace})
+
+            if cmake_function_name == "add_library":
+                cmake_imported_target_type = cmake_function_args[1]
+                if cmake_imported_target_type in ["STATIC", "SHARED"]:
+                    # library filenames are based on the target name by default
+                    lib_name = cmake_target_nonamespace
+                    # the filename may be changed by a straightforward command:
+                    # set_property(TARGET Bonjour PROPERTY OUTPUT_NAME dnssd)
+                    # but we'd have to read the nmos-cpp-targets-<config>.cmake files
+                    # and parse the IMPORTED_LOCATION_<CONFIG> values
+                    if lib_name == "Bonjour":
+                        lib_name = "dnssd"
+                    components[component_name]["libs"] = [lib_name]
+            elif cmake_function_name == "set_target_properties":
+                target_properties = re.findall(r"(?P<property>INTERFACE_[A-Z_]+)[\n|\s]+\"(?P<values>.+)\"", cmake_function_args[2])
+                for target_property in target_properties:
+                    property_type = target_property[0]
+                    # '\', '$' and '"' are escaped; '$' especially is important here
+                    # see https://github.com/conan-io/conan/blob/release/1.39/conans/client/generators/cmake_common.py#L43-L48
+                    property_values = re.sub(r"\\(.)", r"\1", target_property[1]).split(";")
+                    if property_type == "INTERFACE_LINK_LIBRARIES":
+                        for dependency in property_values:
+                            match_private = re.fullmatch(r"\$<LINK_ONLY:(.+)>", dependency)
+                            if match_private:
+                                dependency = match_private.group(1)
+                            if "::" in dependency:
+                                dependency = dependency.replace("nmos-cpp::", "")
+                                # Conan component name cannot be the same as the package name
+                                if dependency == "nmos-cpp":
+                                    dependency = "nmos-cpp-lib"
+                                # Conan packages for Boost, cpprestsdk, websocketpp, OpenSSL and Avahi have component names that (except for being lowercase) match the CMake targets
+                                # json-schema-validator overrides cmake_find_package[_multi] names
+                                elif dependency == "nlohmann_json_schema_validator::nlohmann_json_schema_validator":
+                                    dependency = "json-schema-validator::json-schema-validator"
+                                # mdnsresponder overrides cmake_find_package[_multi] names
+                                elif dependency == "DNSSD::DNSSD":
+                                    dependency = "mdnsresponder::mdnsresponder"
+                                components[component_name].setdefault("requires" if not match_private else "requires_private", []).append(dependency.lower())
+                            elif "${_IMPORT_PREFIX}/lib/" in dependency:
+                                self.output.warn("{} recipe does not handle {} {} (yet)".format(self.name, property_type, dependency))
+                            else:
+                                components[component_name].setdefault("system_libs", []).append(dependency)
+                    elif property_type == "INTERFACE_COMPILE_DEFINITIONS":
+                        for property_value in property_values:
+                            components[component_name].setdefault("defines", []).append(property_value)
+                    elif property_type == "INTERFACE_COMPILE_FEATURES":
+                        for property_value in property_values:
+                            if property_value not in ["cxx_std_11"]:
+                                self.output.warn("{} recipe does not handle {} {} (yet)".format(self.name, property_type, property_value))
+                    elif property_type == "INTERFACE_COMPILE_OPTIONS":
+                        for property_value in property_values:
+                            # handle forced include (Visual Studio /FI, gcc -include) by relying on includedirs containing "include"
+                            property_value = property_value.replace("${_IMPORT_PREFIX}/include/", "")
+                            components[component_name].setdefault("cxxflags", []).append(property_value)
+                    elif property_type == "INTERFACE_INCLUDE_DIRECTORIES":
+                        for property_value in property_values:
+                            if property_value not in ["${_IMPORT_PREFIX}/include"]:
+                                self.output.warn("{} recipe does not handle {} {} (yet)".format(self.name, property_type, property_value))
+                    elif property_type == "INTERFACE_LINK_OPTIONS":
+                        for property_value in property_values:
+                            # workaround required because otherwise "/ignore:4099" gets converted to "\ignore:4099.obj"
+                            # thankfully the MSVC linker accepts both '/' and '-' for the option specifier and Visual Studio
+                            # handles link options appearing in Link/AdditionalDependencies rather than Link/AdditionalOptions
+                            # because the CMake generators put them in INTERFACE_LINK_LIBRARIES rather than INTERFACE_LINK_OPTIONS
+                            # see https://github.com/conan-io/conan/pull/8812
+                            # and https://docs.microsoft.com/en-us/cpp/build/reference/linking?view=msvc-160#command-line
+                            property_value = re.sub(r"^/", r"-", property_value)
+                            components[component_name].setdefault("linkflags", []).append(property_value)
+                    else:
+                        self.output.warn("{} recipe does not handle {} (yet)".format(self.name, property_type))
+
+        # Save components informations in json file
+        with open(self._components_helper_filepath, "w") as json_file:
+            json.dump(components, json_file, indent=4)
+
+    @property
+    def _components_helper_filepath(self):
+        return os.path.join(self.package_folder, "lib", "components.json")
+
+    def package_info(self):
+        bindir = "bin"
+        libdir = "lib"
+        # on Windows, cmake_install() puts the binaries in a config-specific sub-folder
+        if self.settings.os == "Windows":
+            config_install_dir = "Debug" if self.settings.build_type == "Debug" else "Release"
+            bindir = os.path.join(bindir, config_install_dir)
+            libdir = os.path.join(libdir, config_install_dir)
+
+        def _register_components():
+            components_json_file = tools.load(self._components_helper_filepath)
+            components = json.loads(components_json_file)
+            for component_name, values in components.items():
+                cmake_target = values["cmake_target"]
+                self.cpp_info.components[component_name].names["cmake_find_package"] = cmake_target
+                self.cpp_info.components[component_name].names["cmake_find_package_multi"] = cmake_target
+                self.cpp_info.components[component_name].libs = values.get("libs", [])
+                self.cpp_info.components[component_name].libdirs = [libdir]
+                self.cpp_info.components[component_name].defines = values.get("defines", [])
+                self.cpp_info.components[component_name].cxxflags = values.get("cxxflags", [])
+                linkflags = values.get("linkflags", [])
+                self.cpp_info.components[component_name].sharedlinkflags = linkflags
+                self.cpp_info.components[component_name].exelinkflags = linkflags
+                self.cpp_info.components[component_name].system_libs = values.get("system_libs", [])
+                self.cpp_info.components[component_name].frameworks = values.get("frameworks", [])
+                self.cpp_info.components[component_name].requires = values.get("requires", [])
+                # hmm, how should private requirements be indicated? this results in a string format error...
+                #self.cpp_info.components[component_name].requires.extend([(r, "private") for r in values.get("requires_private", [])])
+                self.cpp_info.components[component_name].requires.extend(values.get("requires_private", []))
+        _register_components()
+
+        # add nmos-cpp-registry and nmos-cpp-node to the path
+        bin_path = os.path.join(self.package_folder, bindir)
+        self.output.info("Appending PATH environment variable: {}".format(bin_path))
+        self.env_info.PATH.append(bin_path)

recipes/nmos-cpp/all/test_package/CMakeLists.txt

@@ -0,0 +1,11 @@
+cmake_minimum_required(VERSION 3.1)
+project(NmosCppTestPackage CXX)
+
+include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
+conan_basic_setup(TARGETS)
+
+find_package(nmos-cpp REQUIRED)
+
+add_executable(test_package main.cpp)
+target_link_libraries(test_package nmos-cpp::compile-settings nmos-cpp::nmos-cpp)
+set_target_properties(test_package PROPERTIES CXX_STANDARD 11 CXX_STANDARD_REQUIRED ON)

recipes/nmos-cpp/all/test_package/conanfile.py

@@ -0,0 +1,43 @@
+import os
+import subprocess
+from six import StringIO
+from conans import ConanFile, CMake, tools
+
+class NmosCppTestPackageConan(ConanFile):
+    settings = "os", "compiler", "build_type", "arch"
+    # use cmake_find_package_multi because the project installs a config-file package
+    generators = "cmake", "cmake_find_package_multi"
+
+    def build(self):
+        cmake = CMake(self)
+        cmake.configure()
+        cmake.build()
+
+    def test(self):
+        if not tools.cross_building(self):
+            with open("registry-config.json", "w") as config:
+                config.write('{"http_port": 10000, "domain": "local.", "pri": 51967}')
+            with open("node-config.json", "w") as config:
+                config.write('{"http_port": 20000, "domain": "local.", "highest_pri": 51967, "lowest_pri": 51967}')
+
+            # start up the installed nmos-cpp-registry to check it works
+            registry = subprocess.Popen(["nmos-cpp-registry", "registry-config.json"],
+                                        stdout=subprocess.PIPE,
+                                        stderr=subprocess.STDOUT,
+                                        universal_newlines=True)
+
+            # run the test_package node which should have time to register and then exit
+            node_out = StringIO()
+            try:
+                bin_path = os.path.join("bin", "test_package")
+                self.run(bin_path + " node-config.json", run_environment=True, output=node_out)
+            finally:
+                registry.terminate()
+            if "Adopting registered operation" not in node_out.getvalue():
+                self.output.warn("test_package node failed to register with nmos-cpp-registry\n"
+                                 "\n"
+                                 "nmos-cpp-registry log:\n"
+                                 "{}\n"
+                                 "test_package log:\n"
+                                 "{}"
+                                 .format(registry.communicate()[0], node_out.getvalue()))

recipes/nmos-cpp/all/test_package/main.cpp

@@ -0,0 +1,40 @@
+#include <fstream>
+#include <thread>
+#include "cpprest/json_utils.h"
+#include "nmos/id.h"
+#include "nmos/log_gate.h"
+#include "nmos/model.h"
+#include "nmos/node_resource.h"
+#include "nmos/node_server.h"
+#include "nmos/server.h"
+
+const web::json::field_with_default<int64_t> how_long{ U("how_long"), 2000 };
+
+int main(int argc, char* argv[])
+{
+    nmos::node_model node_model;
+    nmos::experimental::log_model log_model;
+    nmos::experimental::log_gate gate(std::cerr, std::cout, log_model);
+    nmos::experimental::node_implementation node_implementation;
+
+    if (argc > 1)
+    {
+        std::ifstream file(argv[1]);
+        node_model.settings = web::json::value::parse(file);
+    }
+    nmos::insert_node_default_settings(node_model.settings);
+
+    auto node_server = nmos::experimental::make_node_server(node_model, node_implementation, log_model, gate);
+    nmos::insert_resource(node_model.node_resources, nmos::make_node(nmos::make_id(), node_model.settings));
+
+    try
+    {
+        nmos::server_guard node_server_guard(node_server);
+        std::this_thread::sleep_for(std::chrono::milliseconds(how_long(node_model.settings)));
+    }
+    catch (const std::exception& e)
+    {
+        slog::log<slog::severities::error>(gate, SLOG_FLF) << "Exception: " << e.what();
+    }
+    return 0;
+}

recipes/nmos-cpp/config.yml

@@ -0,0 +1,3 @@
+versions:
+  "cci.20210902":
+    folder: all