Convert std::set to frozenset

[thony-murray]

I am trying to convert the type std::map<std::set<int>, int> from C++ to Python. All of these types have built-in conversions, but the conversion fails with the error TypeError: unhashable type: 'set'. This makes sense, as the built-in conversions are tying to create the invalid type Dict[Set[int], int] in python. Is there any way I can get it to convert my std::set to a Python frozenset instead? frozenset is hashable and can be used as map keys.

EDIT: The conversion in the reverse direction is desirable as well. If I have a C++ function that takes a std::map<std::set<int>, int> I'd like to be able to bind it to Python and pass in a Dict[FrozenSet[int], int].

[jiwaszki]

Update: actually I have prototyped something like this. I will test it out more and probably create a PR with this. For now feel free to use this sample in your code. I will be happy to hear some feedback!

Here is my gist with answer:
https://gist.github.com/jiwaszki/5ab6267672929e92e6542a44a27fdaa9

And results from tests:

frozenset({1, 2, 3})
frozenset({1, 2, 3})
frozenset({5, 6, 7})
frozenset({5, 6, 7})
{frozenset({1, 2, 3}): 6}
{frozenset({1, 2, 3}): 6}
{1, 2, 3}
{1, 2, 3}
{5, 6, 7}
{5, 6, 7}
Traceback (most recent call last):
  File "./test_frozenset.py", line 13, in <module>
    print(mymodule.foo5({{1,2,3}: 6}))
TypeError: unhashable type: 'set'

So to create py::frozenset that is fully functional like py::set (that translates to std::set) you need to have two major parts of code. First is implementation of Python type that uses Python C API, this is easy part. I used modified version of py::set that is working for frozenset instead.
Now going to problematic part. ** Best case scenario is to catch both cases std::set<const type> (or even const std::set<const type>) and std::set<type>. Why is it important? It should be possible to distinguish frozenset and set.**

However now there is an issue that pybind maintainers should answer: is adding new class based on top of already covered C++ type is something that breaks existing compatibility between versions? I guess I need to find some answers... 😃

[thony-murray]

Thanks so much for working on this!! I don't have time to dig into your code right now but I will try to take a look this weekend if I have some time

[Skylion007]

Adding a new caster or pytype does not break compatibility between versions. Even if it did, we do allow ABI breaks for good reason.

[ecatmur]

I hope you don't mind - I needed this so I've expanded it into a PR handling sequence types also (as Tuple[T, ...]): #3886

[jiwaszki]

@ecatmur Not at all! I was busy last two weeks, that is great to see someone expanding on my idea! 😄

[Skylion007]

@jiwaszki If you want an example, Eigen::Ref has slightly different behavior for Eigen::Ref<MatrixType> and Eigen::Ref<const MatrixTypehttps://pybind11.readthedocs.io/en/stable/advanced/cast/eigen.html#pass-by-reference

[jiwaszki]

Thank you! I will dig into that in a meantime.

[Skylion007]

@jiwaszki More specifically, here is the check:

pybind11/include/pybind11/eigen.h

Line 273 in c4e2952

return eigen_array_cast<props>(src, parent, !std::is_const<Type>::value);