How to use shared_ptr and make_shared with arrays? - cython

Question

I want to use a C++ shared_ptr as a replacement for raw C pointers. As a simple example the following code seems to work as intended:
from libcpp.memory cimport shared_ptr, allocator
cdef shared_ptr[double] spd
cdef allocator[double] allo
spd.reset(allo.allocate(13))
The size is chosen as 13 here, but in general is not know at compile time.
I'm not sure if this is correct, but I haven't had any errors (no memory leaks and segfaults yet). I'm curious if there is a more optimal solution with make_shared.
But I can't use C++11 arrays because Cython doesn't allow literals as templates, e.g. something like
cdef shared_ptr[array[double]] spd = make_shared[array[double,13]]()
and "normal" arrays which are supposed to work with C++20 compiler (e.g. gcc 10) are causing problems in one way or another:
# Cython error "Expected an identifier or literal"
cdef shared_ptr[double[]] spd = make_shared[double[]](3)
# Can't get ptr to work
ctypedef double[] darr
cdef shared_ptr[darr] spd = make_shared[darr](13)
cdef double* ptr = spd.get() # or spd.get()[0] or <double*> spd.get()[0] or ...
Is the allocator solution the correct and best one or is there another way how to do it?

Answer 1

Here is what I'm going with
cdef extern from *:
"""
template <typename T>
struct Ptr_deleter{
size_t nn;
void (*free_ptr)(T*, size_t);
Ptr_deleter(size_t nIn, void (*free_ptrIn)(T*, size_t)){
this->nn = nIn;
this->free_ptr = free_ptrIn;
};
void operator()(T* ptr){
free_ptr(ptr, nn);
};
};
template <typename T>
std::shared_ptr<T> ptr_to_sptr (T* ptr, size_t nn, void (*free_ptr)(T*, size_t)) {
Ptr_deleter dltr(nn, free_ptr);
std::shared_ptr<T> sp (ptr, dltr);
return sp;
};
"""
shared_ptr[double] d_ptr_to_sptr "ptr_to_sptr"(double* ptr, size_t nn, void (*free_ptr)(double*, size_t) nogil) nogil
cdef void free_d_ptr(double* ptr, size_t nn) nogil:
free(ptr)
cdef shared_ptr[double] sp_d_empty(size_t nn) nogil:
return d_ptr_to_sptr(<double*> nullCheckMalloc(nn*sizeof(double)), nn, &free_d_ptr)
My understanding is that the "right" way to handle malloced arrays is to use a custom deleter like I did. I personally prefer sticking with somewhat-raw C pointers (double* instead of double[] or something), since it's more natural in Cython and my projects.
I think it's reasonably easy to see how to change free_ptr for more complicated data types. For simple data types it could be done in less lines and less convoluted, but I wanted to have the same base.
I like my solution in the regard that I can just "wrap" existing Cython/C code raw pointers in a shared_ptr.
When working with C++ (especially newer standards like C++20) I think verbatim code is pretty often necessary. But I've intentionally defined free_d_ptr in Cython, so it's easy to use existing Cython code to handle the actual work done to free/clear/whatever the array.
I didn't get C++11 std::arrays to work, and it's apparently not "properly" possible in Cython in general (see Interfacing C++11 array with Cython).
I didn't get double[] or similar to work either (is possible in C++20), but with verbatim C++ code I think this should be doable in Cython. I prefer more C-like pointers/arrays anyway as I said.