How can I tell Cython that my C++ class has overloaded operator=? I tried:
cdef extern from "my_source.H":
cdef cppclass MyStatus:
void operator=(const char* status)
cdef public void setStatus(MyStatus& status):
status = "FOO"
but Cython either complains "Assignment to reference status" or (if I make status a non-reference) constructs a python object out of the string "FOO" and then tries to assign the python object to status.
The problem in your code is, that for Cython "FOO" is a Python-object. For expressions like
char *s = "FOO"
Cython is clever enough to understand, what you want and automatically interprets "FOO" as char *.
However, Cython doesn't really "understand"/interpret the signatures of wrapped c++-functions (for that it must be a c++-compiler) and thus cannot know, that you want "FOO" be a char *.
Thus you have to help Cython, for example:
status = <const char *>"FOO"
You also have to work around the problem with reference, for example via:
cdef public void setStatus(MyStatus *status):
status[0] = <const char *>"FOO"
or if you want to have keep the signature of the function intact:
cdef public void setStatus(MyStatus& status):
cdef MyStatus * as_ptr = &status
as_ptr[0] = <const char *>"FOO"
I'm not completely sure the problem with the assigment to reference isn't a bug.
Another observation: the assigment operators aren't part of the "official" wrap of the standard containers, see here or here.
Related
I want to export a python class to be used in C++:
class Object:
def method(self):
pass
cdef public object create_object() with gil:
return Object()
cdef public void py_method(object obj) with gil:
obj.method() # obj is expected to have method()
Then the generated header file will have the following functions:
__PYX_EXTERN_C PyObject *create_object(void);
__PYX_EXTERN_C void py_method(PyObject *);
My question is: should I also have a function like
from cpython cimport Py_XDECREF, PyObject
cdef public void destroy_object(PyObject* obj) with gil:
Py_XDECREF(obj)
to avoid memory leak?
Then these functions may be used in C++ code like:
auto obj = create_object()
py_method(obj)
// Finally
destroy_object(obj)
Yes you should decrement it when calling it from C++. You could do it the way you propose or you could skip the definition of destroy_object and just use Py_XDECREF in C++.
Other things you should consider:
create_object can return NULL if it's raised a Python exception so check for that
RAII wrappers are always a good idea (to handle the reference counting)
Hello i've been trying to call a python user-defined callback from c++ using cython for a while. But it looks like it's impossible without changes on the c++ side or a static function buffer.
So, is there only one option for binding a propper callback (ctypes with CFUNCTYPE)?
Cython 0.29.23
A.hpp:
typedef void (*Callback) ();
class A{
Callback callback;
public:
A(){
this->callback = nullptr;
}
void set_callback(Callback callback){
this->callback = callback;
}
void call_callback(){
this->callback();
}
};
A.pxd:
cdef extern from "A.hpp":
ctypedef void (*Callback) ()
cdef cppclass A:
A() except +
void set_callback(Callback callback)
void call_callback()
B.pyx
from A cimport A, Callback
cdef class B:
cdef A *c_self
cdef object callback
def __cinit__(self):
self.c_self = new A()
def __dealloc__(self):
del self.c_self
cdef void callback_func(self) with gil:
print("I'm here")
self.callback()
def set_callback(self, callback):
self.callback = callback
self.c_self.set_callback(<Callback>self.callback_func)
def call_callback(self):
self.c_self.call_callback()
def print_():
print("hello")
b = B()
b.set_callback(print)
b.call_callback()
Output:
I'm here
[segmentation fault]
Looks like ctypes: get the actual address of a c function is a good one work-around, but it uses ctypes.
It scares me, but works:
B.pyx
from A cimport A, Callback
import ctypes
from libc.stdint cimport uintptr_t
cdef class B:
cdef A *c_self
cdef object callback
def __cinit__(self):
self.c_self = new A()
def __dealloc__(self):
del self.c_self
def set_callback(self, callback):
f = ctypes.CFUNCTYPE(None)(callback)
self.callback = f
cdef Callback c_callback = (<Callback*><uintptr_t>ctypes.addressof(f))[0]
self.c_self.set_callback(c_callback)
def call_callback(self):
self.c_self.call_callback()
def hello():
print("hello")
b = B()
b.set_callback(hello)
b.call_callback()
A function pointer does not have any space to store extra information. Therefore it is not possible to convert a Python callable to a function pointer in pure C. Similarly a cdef function of a cdef class must store the address of the instance to be usable and that is impossible too.
You have three options:
Use ctypes as in https://stackoverflow.com/a/34900829/4657412 (the bottom half of the answer shows how to do it). ctypes accomplishes this with runtime code generation (and thus only works on processors that it explicitly supports).
Use a std::function in C++ instead of a function pointer. These can store arbitrary information. You need to write an object wrapper (or re-use one from elsewhere) so it isn't completely pure Cython. See Pass a closure from Cython to C++. What you're trying to do is probably better covered by How to use a Cython cdef class member method in a native callback.
Use the class C scheme where the callback is of type
void (CallbackT)(/* any args */, void* user_data)
and is registered with:
void register_callback(CallbackT func, void* user_data)
In this case user_data would be the address of your B instances (and you'd need to make sure it was Py_INCREFed before setting the address and Py_DECREFed before unsetting the address). Cython callback with class method provides an example.
I want to use a C++ shared_ptr as a replacement for raw C pointers. As a simple example the following code seems to work as intended:
from libcpp.memory cimport shared_ptr, allocator
cdef shared_ptr[double] spd
cdef allocator[double] allo
spd.reset(allo.allocate(13))
The size is chosen as 13 here, but in general is not know at compile time.
I'm not sure if this is correct, but I haven't had any errors (no memory leaks and segfaults yet). I'm curious if there is a more optimal solution with make_shared.
But I can't use C++11 arrays because Cython doesn't allow literals as templates, e.g. something like
cdef shared_ptr[array[double]] spd = make_shared[array[double,13]]()
and "normal" arrays which are supposed to work with C++20 compiler (e.g. gcc 10) are causing problems in one way or another:
# Cython error "Expected an identifier or literal"
cdef shared_ptr[double[]] spd = make_shared[double[]](3)
# Can't get ptr to work
ctypedef double[] darr
cdef shared_ptr[darr] spd = make_shared[darr](13)
cdef double* ptr = spd.get() # or spd.get()[0] or <double*> spd.get()[0] or ...
Is the allocator solution the correct and best one or is there another way how to do it?
Here is what I'm going with
cdef extern from *:
"""
template <typename T>
struct Ptr_deleter{
size_t nn;
void (*free_ptr)(T*, size_t);
Ptr_deleter(size_t nIn, void (*free_ptrIn)(T*, size_t)){
this->nn = nIn;
this->free_ptr = free_ptrIn;
};
void operator()(T* ptr){
free_ptr(ptr, nn);
};
};
template <typename T>
std::shared_ptr<T> ptr_to_sptr (T* ptr, size_t nn, void (*free_ptr)(T*, size_t)) {
Ptr_deleter dltr(nn, free_ptr);
std::shared_ptr<T> sp (ptr, dltr);
return sp;
};
"""
shared_ptr[double] d_ptr_to_sptr "ptr_to_sptr"(double* ptr, size_t nn, void (*free_ptr)(double*, size_t) nogil) nogil
cdef void free_d_ptr(double* ptr, size_t nn) nogil:
free(ptr)
cdef shared_ptr[double] sp_d_empty(size_t nn) nogil:
return d_ptr_to_sptr(<double*> nullCheckMalloc(nn*sizeof(double)), nn, &free_d_ptr)
My understanding is that the "right" way to handle malloced arrays is to use a custom deleter like I did. I personally prefer sticking with somewhat-raw C pointers (double* instead of double[] or something), since it's more natural in Cython and my projects.
I think it's reasonably easy to see how to change free_ptr for more complicated data types. For simple data types it could be done in less lines and less convoluted, but I wanted to have the same base.
I like my solution in the regard that I can just "wrap" existing Cython/C code raw pointers in a shared_ptr.
When working with C++ (especially newer standards like C++20) I think verbatim code is pretty often necessary. But I've intentionally defined free_d_ptr in Cython, so it's easy to use existing Cython code to handle the actual work done to free/clear/whatever the array.
I didn't get C++11 std::arrays to work, and it's apparently not "properly" possible in Cython in general (see Interfacing C++11 array with Cython).
I didn't get double[] or similar to work either (is possible in C++20), but with verbatim C++ code I think this should be doable in Cython. I prefer more C-like pointers/arrays anyway as I said.
when I declare a cdef function that returns a double I write cdef double method_name(...). If it does not return something and I just omit it to cdef method_name(...) then cython --annotate marks it yellow. How to declare that the method/function does not return anything?
cdef void method_name(...) crashes with a segmentation fault
cdef None method_name(...) -> 'None' is not a type identifier
--annotate marks it as yellow as cython assumes the return type to be a python object if you omit the return type annotation (Cython Language Basics).
Specifying void as return type works for me. It's also used in quite some of the official examples, Just make sure not to return anything.
For me (cython 0.21.1) defining the c function with void works:
# mymod.pyx file
cdef void mycfunc(int* a):
a[0] = 2
def myfunc():
cdef int a = 1
mycfunc(&a)
print(a)
The c function is not yellow in the annotated html file and
python -c 'from mymod import myfunc; myfunc()'
prints 2 as expected.
A bug in Cython version 0.22. Update to 0.23.4 solved it.
I am trying to create a wrapper in Cython for a library which uses Eigen::Matrix3d matrices. How can I set an individual element/coefficient of the Matrix3d object?
I know, I can get the value with the coeff(row, col) method but could not find any function set_coeff(row, col, value) - or however that might be called - to set the value.
After declaring the Matrix3d with
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
I want to set the values, but none of the following constructs work in Cython:
t << 1,2,3,4,5,6,7,8,9
t(0,0) = 1
t[0][0] = 1
and I cannot use a constructor with the values, because to my knowledge there does not exist any.
Here are the files I have come up so far:
decl_eigen.pxd:
cdef extern from "Eigen/Dense" namespace "Eigen":
cdef cppclass Vector3d:
Matrix3d() except +
double coeff(int row, int col)
decl_foo.pxd:
cimport decl_eigen
cdef extern from "../foo.hpp" namespace "MyFoo":
cdef cppclass Bar:
Bar() except +
void transform(decl_eigen.Matrix3d &transformation)
foo.pyx:
import decl_eigen
cimport decl_foo
cdef class Bar:
cdef decl_foo.Bar *thisptr
def __cinit__(self):
self.thisptr = new decl_foo.Bar()
def __dealloc__(self):
del self.thisptr
def transform(self, transformation):
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
for i in range(3):
for j in range(3):
k = i*3 + j
# Set the coefficient of t(i,j) to transformation[k], but how????
self.thisptr.transform(t)
Thanks.
It's not as straightforward as it should be, but you can make it work.
Element access in Eigen looks to mostly be done through operator():
// (copied from http://eigen.tuxfamily.org/dox/GettingStarted.html)
MatrixXd m(2,2);
m(0,0) = 3;
m(1,0) = 2.5;
m(0,1) = -1;
m(1,1) = m(1,0) + m(0,1);
Therefore, we need to define operator() so you can access it in Cython. I've assumed it returns a double& - I can't actually find the definition in Eigen since it's buried deep in a template class hierarchy (It's not terribly important what it actually returns - it acts like it returns a double&, which should be good enough).
Unfortunately, operator() seems slightly broken in Cython (see Cython C++ wrapper operator() overloading error) so we have to alias it as something else. I've used element.
cdef extern from "eigen3/Eigen/Dense" namespace "Eigen":
# I'm also unsure if you want a Matrix3d or a Vector3d
# so I assumed matrix
cdef cppclass Matrix3d:
Matrix3d() except +
double& element "operator()"(int row,int col)
In principle we'd just like to be able to do m.element(0,0) = 5. However, Cython doesn't like this. Therefore, I've had to create a function which does this through a slightly complicated assignment to pointer type mechanism.
cdef void set_matrix_element(Matrix3d& m, int row, int col, double elm):
cdef double* d = &(m.element(row,col))
d[0] = elm
Therefore, to set a matrix element, we just call this function. Here's the function I made to test it on:
def get_numbers():
cdef Matrix3d m = Matrix3d()
cdef int i
for i in range(3):
set_matrix_element(m,i,i,i)
return m.element(0,0),m.element(1,1),m.element(2,2),m.element(1,2)
# returns 0,1,2, and something else (I get 0, but in principle
# I think it's undefined since it's never been specifically set)