return_value_policy::reference fails if operator new is not public

[AndrewWalker]

In cases where operator new for a class is inaccessible, bound functions that return that type with the return_value_policy::reference will not compile

#include <memory>
#include <iostream>
#include <pybind11/pybind11.h>
using namespace pybind11;

class Foo {
    void *operator new(size_t bytes) throw();
};

Foo gFoo;
Foo& getstmt() {
    return gFoo;
}

PYBIND11_PLUGIN(example) {
    module m("example");
    class_<Foo>(m, "Foo");
    m.def("getstmt", getstmt, return_value_policy::reference);
    return m.ptr();
}

generating errors in type_caster_generic::cast

/usr/local/include/pybind11/cast.h:214:16: error: 'operator new' is a private member of 'Foo'
        return new type(*((const type *) arg));
               ^~~
/usr/local/include/pybind11/cast.h:202:80: note: in instantiation of function template specialization
      'pybind11::detail::type_caster<Foo, void>::copy_constructor<Foo, 0>' requested here
        return type_caster_generic::cast(&src, policy, parent, &typeid(type), &copy_constructor);
                                                                               ^
/usr/local/include/pybind11/pybind11.h:66:39: note: in instantiation of member function
      'pybind11::detail::type_caster<Foo, void>::cast' requested here
            handle result = cast_out::cast(
                                      ^
/usr/local/include/pybind11/pybind11.h:469:22: note: in instantiation of function template
      specialization 'pybind11::cpp_function::cpp_function<Foo &, pybind11::name, pybind11::sibling,
      pybind11::return_value_policy>' requested here
        cpp_function func(std::forward<Func>(f), name(name_),
                     ^
main.cpp:21:7: note: in instantiation of function template specialization 'pybind11::module::def<Foo
      &(&)(), pybind11::return_value_policy>' requested here
    m.def("getstmt", getstmt, return_value_policy::reference);
      ^
main.cpp:8:11: note: implicitly declared private here
    void *operator new(size_t bytes) throw();

The equivalent boost::python example does compile.

#include <memory>
#include <boost/python.hpp>

using namespace boost;
using namespace boost::python;

class Foo {
    void *operator new(size_t bytes) throw();
};

Foo gFoo;
Foo& getstmt() {
    return gFoo;
}

BOOST_PYTHON_MODULE(example)
{
    class_<Foo>("Foo");
    def("getstmt", getstmt, return_value_policy<reference_existing_object>());
}

[wjakob]

ok, third try!, now I think it works everywhere -- can you test on your end?

[bennybp]

I think the fix is causing build problems on my end. For classes where I don't do anything with operator new, but which are not copy constructible (copy constructor is deleted), detail::is_copy_constructible seems to be true, which breaks the build.

[wjakob]

does not reproduce here (tested on Clang and GCC), can you provide more details? (also, are you sure you are testing the latest version? I rebased a few times until getting it to work)

[bennybp]

Ok after some digging, it fails if the class is not copy constructible, but is move constructible. Then, it seems detail::is_copy_constructible is true. Just modifying the original Foo class a bit:

class Foo {
public:
    Foo() = default;
    Foo(const Foo &) = delete;
    Foo(Foo &&) = default;
};

Errors:

../pybind11/include/pybind11/cast.h: In instantiation of ‘static void* pybind11::detail::type_caster<type, Enable>::copy_constructor(const void*)
[with T = Foo; typename std::enable_if<pybind11::detail::is_copy_constructible<T>::value, int>::type <anonymous> = 0; type = Foo; Enable = void]’:
<snip>
../pybind11/include/pybind11/cast.h:214:46: error: use of deleted function ‘Foo::Foo(const Foo&)’
     return new type(*((const type *) arg));

[AndrewWalker]

Thanks so much, confirming this has fixed the problem

[wjakob]

@bennybp interesting, seems the standard still leaves some lee-way for compilers to behave differently. Do you have a proposal for changing detail::is_copy_constructible to catch this case?

[bennybp]

From looking around, std::declval<T> returns T &&, but std::declval<T &> returns T & (according to the standard, declval goes through add_rvalue_reference).

So this works for me and the original issue (with gcc 4.9.2 at least)

-    template <typename T2> static std::true_type test(decltype(new T2(std::declval<T2>())) *);
+    template <typename T2> static std::true_type test(decltype(new T2(std::declval<T2 &>())) *);

[wjakob]

ok, the latest commit should hopefully fix this, let me know if it doesn't.