Auto register metafunctions (POC)

[DyXel]

This is an proof-of-concept implementation (works, though I won't call it mergeable) to illustrate the idea I was having around in regards to auto-registration of metafunctions and possibly adopt it for the main PR. It also includes the patch to remove CPPFRONT_METAFUNCTION_LIBRARY usage by @/MaxSagebaum (thanks btw!). In summary, it does the following:

• Remove global mangled names generation
• Remove the per-file entry-point generation
• Instead, generate a static global object called cpp2::meta::register_function for each metafunction, for which the fully-qualified name is passed, as well as a pointer to the metafunction as arguments to the constructor. (this works similarly to how register_flag in cppfront works today)
• The implementation of cpp2::meta::register_function is then provided separately, and its mean to be used when building/linking the final library, along a unambiguous extern "C" function which becomes the only entry-point of the DLL (all in meta_lib_impl.cpp)

Usage then becomes (writing down example with multiple TUs/files to better show how this scales):

metafunctions.cpp2:

greeter: (inout t: cpp2::meta::type_declaration) = {
  t.add_member($R"(say_hi: () = std::cout << "Hello, world!\nFrom (t.name())$\n";)");
}

metafunctions2.cpp2:

greeter2: (inout t: cpp2::meta::type_declaration) = {
  t.add_member($R"(say_hi: () = std::cout << "Hello, world! 2\nFrom (t.name())$\n";)");
}

main.cpp2:

my_class: @greeter type   = { }
my_class2: @greeter2 type = { }
main: ()                  = {
	my_class().say_hi();
	my_class2().say_hi();
}

Build cppfront:

g++ -rdynamic -std=c++20 cppfront.cpp -o cppfront

Build metafunctions lib:

./cppfront metafunctions.cpp2 metafunctions2.cpp2
g++ -std=c++20 -fPIC -shared -o libmetafunctions.so metafunctions.cpp metafunctions2.cpp meta_lib_impl.cpp

Build and run main:

CPPFRONT_METAFUNCTION_LIBRARIES=./libmetafunctions.so ./cppfront main.cpp2
g++ -std=c++20 main.cpp -o main
./main

Output:

metafunctions.cpp2... ok (all Cpp2, passes safety checks)

metafunctions2.cpp2... ok (all Cpp2, passes safety checks)

main.cpp2... ok (all Cpp2, passes safety checks)

Hello, world!
From my_class
Hello, world! 2
From my_class2

Generated Cpp1 code for completeness:
metafunctions.cpp:

//=== Cpp2 type declarations ====================================================


#include "cpp2util.h"

#line 1 "metafunctions.cpp2"


//=== Cpp2 type definitions and function declarations ===========================

#line 1 "metafunctions.cpp2"
auto greeter(cpp2::meta::type_declaration& t) -> void;

//=== Cpp2 function definitions =================================================

#line 1 "metafunctions.cpp2"
auto greeter(cpp2::meta::type_declaration& t) -> void{
#line 2 "metafunctions.cpp2"
  CPP2_UFCS(add_member)(t, R"(say_hi: () = std::cout << "Hello, world!\nFrom )" + cpp2::to_string(CPP2_UFCS(name)(t)) + R"(\n";)");
}

namespace { // Metafunctions registry
cpp2::meta::register_function rf1("::greeter",std::addressof(::greeter));
}

metafunctions2.cpp:

//=== Cpp2 type declarations ====================================================


#include "cpp2util.h"

#line 1 "metafunctions2.cpp2"


//=== Cpp2 type definitions and function declarations ===========================

#line 1 "metafunctions2.cpp2"
auto greeter2(cpp2::meta::type_declaration& t) -> void;

//=== Cpp2 function definitions =================================================

#line 1 "metafunctions2.cpp2"
auto greeter2(cpp2::meta::type_declaration& t) -> void{
#line 2 "metafunctions2.cpp2"
  CPP2_UFCS(add_member)(t, R"(say_hi: () = std::cout << "Hello, world! 2\nFrom )" + cpp2::to_string(CPP2_UFCS(name)(t)) + R"(\n";)");
}

namespace { // Metafunctions registry
cpp2::meta::register_function rf1("::greeter2",std::addressof(::greeter2));
}

main.cpp:

//=== Cpp2 type declarations ====================================================


#include "cpp2util.h"

#line 1 "main.cpp2"
class my_class;
#line 2 "main.cpp2"
class my_class2;


//=== Cpp2 type definitions and function declarations ===========================

#line 1 "main.cpp2"
class my_class             {
public: static auto say_hi() -> void;

      public: my_class() = default;
      public: my_class(my_class const&) = delete; /* No 'that' constructor, suppress copy */
      public: auto operator=(my_class const&) -> void = delete;
  };
#line 2 "main.cpp2"
class my_class2 {
public: static auto say_hi() -> void;

      public: my_class2() = default;
      public: my_class2(my_class2 const&) = delete; /* No 'that' constructor, suppress copy */
      public: auto operator=(my_class2 const&) -> void = delete;
};
#line 3 "main.cpp2"
auto main() -> int;

//=== Cpp2 function definitions =================================================

#line 1 "main.cpp2"


auto my_class::say_hi() -> void { std::cout << "Hello, world!\nFrom my_class\n"; }
auto my_class2::say_hi() -> void { std::cout << "Hello, world! 2\nFrom my_class2\n"; }
#line 3 "main.cpp2"
auto main() -> int        {
 CPP2_UFCS(say_hi)(my_class());
 CPP2_UFCS(say_hi)(my_class2());
}

Follow up

@MaxSagebaum you mentioned here hsutter#907 (comment) that you'd prefer Option 1, over 2 (what I did here), could you elaborate on why?

After drafting this code, I feel like with this approach, most of the code currently in the main PR becomes a bit redundant:

• No need for mangling implementations at all
• No need to look up multiple symbols, just an exact name is enough, which makes dll_symbol unnecessary
• No extra code generation besides the registry bit at the very end
• Simpler usage for the end-user, all they need to do is provide meta_lib_impl.cpp. It also acts as a customization point if needed.

What do you think? Would this be a direct improvement? I am not sure which caveats it might have over the existing solution, lets discuss here!

[MaxSagebaum]

I tested your implementation and it works. But meta_lib_impl.cpp is not really necessary. I modified you branch, such that meta_lib_impl.cpp could be moved to reflect_impl.h2. This would make the implementation even simpler.

simplification.patch.txt

@MaxSagebaum you mentioned here hsutter#907 (comment) that you'd prefer Option 1, over 2 (what I did here), could you elaborate on why?

The loading order of static symbols is not defined in Cpp1 and I had problems with this in the past. I thought about it a little bit more and for this kind of application it should not matter. So option 2 should be fine.

[DyXel]

I modified you branch, such that meta_lib_impl.cpp could be moved to reflect_impl.h2. This would make the implementation even simpler.

Oh, I thought about that too, but my concern with it is that it introduces extra global state to cppfront, in order to keep track where symbols belong per library (constraint 3 point c in the constraints post), as you'd need to setup global context that the register_function constructor within cppfront can use, this might or might not matter in the end, but good to keep in mind.

The loading order of static symbols is not defined in Cpp1 and I had problems with this in the past. I thought about it a little bit more and for this kind of application it should not matter. So option 2 should be fine.

Yeah, I too know how problematic order of static constructors can be, but here we defer all of that to cppfront, it can decide how exactly to build up the look-up tree from the registry, and even detect duplicate entries (so we can deal with subtle ODR violations).

[MaxSagebaum]

Oh, I thought about that too, but my concern with it is that it introduces extra global state to cppfront, in order to keep track where symbols belong per library (constraint 3 point c in hsutter#907 (comment)), as you'd need to setup global context that the register_function constructor within cppfront can use, this might or might not matter in the end, but good to keep in mind.

With your current implementation, every library needs to setup a local state.

The global state in cppfront is not really required. The register_functions could just call a function handle that is set by cppfront prior to the loading of the dll.

I do not really see the problem with constraint 3 point c. You know which dll you are loading and can use this information to tag each meta function. If a second metafunction with the same name is loaded, then you can throw a nice error. This would be nearly the same as it currently is.

[DyXel]

The register_functions could just call a function handle that is set by cppfront prior to the loading of the dll.

I do not really see the problem with constraint 3 point c. You know which dll you are loading and can use this information to tag each meta function.

yes, the "set by cppfront prior to loading the dll" part is what I mean by "global state", you'd need a global variable to setup that function handle, which needs to be seen by register_function ctor, then you can load the dll and continue forward. It is minuscule, but still there. Whereas if the register_function implementation is inside the dll, all you need to do is call the exported function, and directly process the array of items returned, without anything additional in cppfront side. Like I said, its maybe not a big deal, and probably worth putting that extra complexity on cppfront side is better for UX.