Support building on macOS with MoltenVK

[Nadrin]

It would be great if someone modified projects/cmake/CMakeLists.txt to make this project build on macOS using MoltenVK. Unfortunately I don't have access to any Apple hardware to do it myself.

[melchor629]

Hi @Nadrin I managed to compile the project using MoltenVK by modifying two lines the of CMake file (it is possible that the variables are wrong on macOS?), changing VULKAN_INCLUDE_DIRS to Vulkan_INCLUDE_DIRS in CMakeLists.txt:79 and changing VULKAN_LIBRARIES to Vulkan_LIBRARIES in CMakeLists.txt:80.

Dependencies can be installed by issuing: brew install assimp glfw3 pkg-config glslang cmake and generating the makefiles by issuing:

cmake .. -DVulkan_LIBRARY=$VULKAN_SDK/macOS/libMoltenVK.dylib -DVulkan_INCLUDE_DIR=$VULKAN_SDK/include

where $VULKAN_SDK is where the MoltenVK is located.

But the demo cannot run neither in OpenGL nor Vulkan:

➜  build git:(master) ✗ ./PBR -vulkan
Error: Vulkan loader has not been found
➜  build git:(master) ✗ ./PBR -opengl
Error: Failed to create OpenGL context

Inspecting a bit I saw that for OpenGL, is trying to create a 4.5 context, something impossible to macOS, so it is expected to fail.

For Vulkan, volk is trying to load a library with name libvulkan.so or libvulkan.so.1. But on Mac, has a different name, so I guess for Mac it should have a different loader. So I end up writing a fast loader for macOS with this simple code:

#include <dlfcn.h>

VkResult volkInitialize_mac() {
    void* module = dlopen("libMoltenVK.dylib", RTLD_NOW | RTLD_LOCAL);
    if (!module)
        return VK_ERROR_INITIALIZATION_FAILED;
    volkInitializeCustom((PFN_vkGetInstanceProcAddr)dlsym(module, "vkGetInstanceProcAddr"));
    return VK_SUCCESS;
}

And modifying the Renderer::initialize() method with this code:

#if __APPLE__ && __MACH__
	if(VKFAILED(volkInitialize_mac())) {
		throw std::runtime_error("Vulkan loader has not been found");
	}
#else
	if(VKFAILED(volkInitialize())) {
		throw std::runtime_error("Vulkan loader has not been found");
	}
#endif

And it started to run, and show a little log. But then it suddenly closes, here is the log:

[mvk-info] MoltenVK version 0.19.0. Vulkan version 1.0.64.
[mvk-info] GPU device NVIDIA GeForce GT 650M supports the following Metal Feature Sets:
	OSX GPU Family 1 v2
	OSX GPU Family 1 v1
[***MoltenVK ERROR***] VK_ERROR_LAYER_NOT_PRESENT: Vulkan layer VK_LAYER_LUNARG_standard_validation is not supported.
[***MoltenVK ERROR***] VK_ERROR_EXTENSION_NOT_PRESENT: Vulkan extension VK_EXT_debug_report is not supported.
Error: Failed to create Vulkan instance
Program ended with exit code: 1

So here I end my ideas. Hope it helps clarify something about support for macOS. Anything you want to try on Mac, count on me :) I will try to test.

Note: I'm using macOS 10.12.6

[isaboll1]

Are you only linking to MoltenVK? I don't have a Mac on me atm, but i do remember that the Vulkan SDK for Mac includes 2 library files, with the first one being MoltenVK, and the second one being either vulkan.framework, or libvulkan.1.dylib, which includes the standard debugging layers. As a result, i think you're getting those errors due to only linking to MoltenVK

[melchor629]

More or less, you have a point @isaboll1 . I downloaded the wrong (and old) MoltenVK. So, downloading LunarG Vulkan SDK, setting the env variable VULKAN_SDK to the subfolder of macOS from the SDK and VK_ICD_FILENAMES=$VULKAN_SDK/etc/vulkan/icd.d/MoltenVK_icd.jsonstarted doing things. Also, from the custom macOS loader I wrote before, the library should be NULL instead of "libMoltenVK.dylib".

After that, another error occurred with GLFW. Googling a bit, I end up with this issue. Basically, use GLFW 3.3, not yet released. To do it with brew: brew install glfw3 --HEAD (this will compile from sources).

I also disabled the debugging stuff of the debugging layers you mentioned because in the output of vulkaninfo, neither both extensions are available (I guess they didn't implement them on macOS).

With that, I can see stuff being loaded in memory. Wait, another error:

...
Loading SPIR-V shader module: shaders/spirv/tonemap_vs.spv
Loading SPIR-V shader module: shaders/spirv/tonemap_fs.spv
Loading mesh: meshes/cerberus.fbx
Loading image: textures/cerberus_A.png
Loading image: textures/cerberus_N.png
Loading image: textures/cerberus_M.png
Loading image: textures/cerberus_R.png
Loading SPIR-V shader module: shaders/spirv/pbr_vs.spv
Loading SPIR-V shader module: shaders/spirv/pbr_fs.spv
[***MoltenVK ERROR***] VK_ERROR_INITIALIZATION_FAILED: Shader library compile failed (error code 3):
Compilation failed: 

<program source>:30:18: warning: 'main0' has C-linkage specified, but returns user-defined type 'main0_out' which is incompatible with C
vertex main0_out main0(main0_in in [[stage_in]], constant TransformUniforms& _19 [[buffer(0)]])
                 ^
<program source>:30:8: error: vertex function has invalid return type 'main0_out'
vertex main0_out main0(main0_in in [[stage_in]], constant TransformUniforms& _19 [[buffer(0)]])
       ^
<program source>:17:5: note: field 'Vertex_tangentBasis' of type 'float3x3' (aka 'matrix<float, 3, 3>') declared here
    float3x3 Vertex_tangentBasis [[user(locn2)]];
    ^
.
[***MoltenVK ERROR***] VK_ERROR_INITIALIZATION_FAILED: Vertex shader function could not be compiled into pipeline. See previous error.
Error: Failed to create graphics pipeline
Program ended with exit code: 1

Well, at least I could see start doing things :) As far as I know, libvulkan.dylib is linked into the app and this one will load libMoltenVK.dylib automatically. Now the error has to do with the shaders. I don't know if it is a problem of the macOS version of Vulkan - 1.0.77 is the version reported.
Hope it helps.

[Nadrin]

@melchor629 I've done a quick google search and it seems that MoltenVK, for some reason, generated invalid Metal shader code from SPIR-V representation. I have no idea, however, whether it's a bug or a limitation of MoltenVK, or if it's caused by something I've done in my Vulkan code.

Also, I just wanted to let you guys know that I really appreciate the troubleshooting that you're doing! Hope this reaches a satisfactory conclusion and in the end we'll have that macOS build. Keep it up! :)

[melchor629]

Thanks @Nadrin :)

Well, I have to say I have no idea of Vulkan so I'm not be able to try things over the shaders or any other stuff of Vulkan. But one thing that may help is the list of MoltenVK limitations, maybe there's something that your project is using and is listed here. If not, maybe it is a bug of MoltenVK and should be reported.