How to access the typed-memory view element of a class declared in cython? - cython

I am a beginner and I am sure this question is too simple. I am trying to test memory views in cython to get to know them much better.In my code I pass each memory view element (like [1,2]) as the cy class element move.
cdef class cy:
cdef public long[:] move
def __init__(self, move):
self.move = move
lst = []
for i in range(100):
lst.append([i, i+1])
cdef long[:, :] memview = np.asarray(lst)
b0 = cy(memview[0])
print(b0.move)
When I print the results. I get this:
<MemoryView of 'ndarray' object> # I expect for sth like [12, 13]
I need cy class prints out a list. How can I fix it?
there is another problem which occurs to me when I use this code:
cdef class parent:
cdef public:
list children
list moves
def __init__(self):
self.children = []
def add_children(self, moves):
cdef int i = 0
cdef int N = len(moves)
for i in range(N):
self.children.append(cy(moves[i]))
cdef int[:, :] moves = np.asarray(lst, dtype=np.int32)
obj = parent()
for move in moves:
obj.add_children(move)
After running this code I always get this error:
TypeError: a bytes-like object is required, not 'int'.
What causes this error and how can I fix this one?

Your first issue is just that a memoryview doesn't have a useful __str__ function for print to use. You can either convert it to an object that does print nicely
print(list(b0.moves))
print(np.asarray(b0.moves))
Or you can iterate through it yourself:
for i in range(b0.moves.shape[0]):
print(b0.moves[i], end=' ') # need to have Cython set to use Python 3 syntax for this line
print()
Your second problem is harder to solve since you don't tell us what line the error comes from. I think it's the constructor of cy which expects a memoryview but you pass an integer to. (I get a slightly different error message though).

Related

Can Python object be referenced under cdef function?

In the code below, self is a python object, as it is declared from def instead of cdef. Can a python object be referenced under a cdef function, just like how it is used under the c_function from my example below?
I am confused because cdef makes it sound like cdef is a C function, so I am not sure if it is able to take on a python object.
class A(self, int w):
def __init__():
self.a = w
cdef c_function (self, int b):
return self.a + b
Thank you,
I am the OP.
From "Cython: A Guide for Python Programmers":
Source: https://books.google.com.hk/books?id=H3JmBgAAQBAJ&pg=PA49&lpg=PA49&dq=python+object+cdef+function&source=bl&ots=QI9If_wiyR&sig=ACfU3U1CmEPBaEVmW0UN1_9m9G8B9a6oFQ&hl=en&sa=X&ved=2ahUKEwiBgs3Lw5vqAhXSZt4KHTs0DK0Q6AEwBHoECAoQAQ#v=onepage&q=python%20object%20cdef%20function&f=false
"[...] Nothing prevents us from declaring and using Python objects and dynamic variables in cdef functions, or accepting them as arguments"
So I take this as the book saying "yes" to my original question.

Strange behaviour when creating python attributes in cython cdef class

We have given Cython code:
cdef extern from "C_File_A.h":
cdef struct C_Obj_A:
pass
cdef extern from "C_File_B.h":
cdef struct C_Obj_B:
pass
cdef class pC_Obj_A:
cdef const C_Obj_A * _c_self
cdef class pC_Obj_B:
cdef const C_Obj_B * _c_self
cdef class pC_Obj_C:
cdef const C_Obj_A * _c_a
cdef const C_Obj_B * _c_b
cdef class Obj_A_Wrap(pC_Obj_A):
def __init__(self, pC_Obj_C obj_c):
self._c_self = obj_c._c_a
cdef class Obj_B_Wrap(pC_Obj_B):
def __init__(self, pC_Obj_C obj_c):
self._c_self = obj_c._c_b
cdef class Stack:
cdef public pC_Obj_A obj_a
cdef public pC_Obj_B obj_b
def __init__(self, pC_Obj_C obj_c):
# Working
self.obj_a = Obj_A_Wrap(obj_c)
self.obj_b = Obj_B_Wrap(obj_c)
# Working
self.obj_a._c_self = obj_c._c_a
self.obj_b = Obj_B_Wrap(obj_c)
# Working
self.obj_a = Obj_A_Wrap(obj_c)
self.obj_b._c_self = obj_c._c_b
# Not working
self.obj_a._c_self = obj_c._c_a
self.obj_b._c_self = obj_c._c_b
I need a python object Stack with attrubutes accessible from Python, so I have added to Stack class cdef public pC_Obj_A obj_a and cdef public pC_Obj_B obj_b.These objects are wrappers to the C struct pointers.
When I initialize these objects with intermediary wrappers i.e. Obj_A_Wrap everything is fine.
When I initialize one of these objects directly i.e. self.obj_a._c_self = obj_c._c_a also everything is fine.
When both obj_a and obj_b are initialized directly (# Not Working part of code) I have got strange behaviour of my C library that inlcude C_File_A and C_File_B and respectively the C structs definitions. The behaviour is similar to memory corruption, or overwriting some parts of the memory that should not be.
I have no idea why the direct initialization causes this strange behaviour. Maybe you know?
I have found the solution of my problem. When I was trying to solve this problem I have printed only _c_self attribute of the given object to check that the pointer was properly assigned and it was but when I printed entire object it turned out that python object is None instead of proper object declared as attribute.
print(self.obj_a, self.obj_b) # 66f000c0 66f000c0
print(f'{<int>self.obj_a._c_self:x} {<int>self.obj_b._c_self:x}') # None None
The solution is to add Create function to cdef class:
cdef class pC_Obj_A:
cdef const C_Obj_A * _c_self
#staticmethod
cdef Create(C_Obj_A * ptr):
cdef pC_Obj_A result = pC_Obj_A()
result._c_self = ptr
return result
And use it like this:
cdef class Stack:
cdef public pC_Obj_A obj_a
cdef public pC_Obj_B obj_b
def __init__(self, pC_Obj_C obj_c):
self.obj_a = pC_Obj_A.Create(obj_c._c_a)
self.obj_b = pC_Obj_B.Create(obj_c._c_b)
Then printout is:
print(self.obj_a, self.obj_b) # <pC_Obj_A object at 0x029FF610> <pC_Obj_B object at 0x029FF620>
print(f'{<int>self.obj_a._c_self:x} {<int>self.obj_b._c_self:x}') # 2134b9c 2134c08
And everything works great!

issue using deepcopy function for cython classes

I've been playing with Cython recently for the speed ups, but when I was trying to use copy.deepcopy() some error occurred.Here is the code:
from copy import deepcopy
cdef class cy_child:
cdef public:
int move[2]
int Q
int N
def __init__(self, move):
self.move = move
self.Q = 0
self.N = 0
a = cy_child((1,2))
b = deepcopy(a)
This is the error:
can't pickle _cython_magic_001970156a2636e3189b2b84ebe80443.cy_child objects
How can I solve the problem for this code?
As hpaulj says in the comments, deepcopy looks to use pickle by default to do its work. Cython cdef classes didn't used to be pickleable. In recent versions of Cython they are where possible (see also http://blog.behnel.de/posts/whats-new-in-cython-026.html) but pickling the array looks to be a problem (and even without the array I didn't get it to work).
The solution is to implement the relevant functions yourself. I've done __deepcopy__ since it's simple but alternatively you could implement the pickle protocol
def __deepcopy__(self,memo_dictionary):
res = cy_child(self.move)
res.Q = self.Q
res.N = self.N
return res
I suspect that you won't need to do that in the future as Cython improves their pickle implementation.
A note on memo_dictionary: Suppose you have
a=[None]
b=[A]
a[0]=B
# i.e. A contains a link to B and B contains a link to A
c = deepcopy(a)
memo_dictionary is used by deepcopy to keep a note of what it's already copied so that it doesn't loop forever. You don't need to do much with it yourself. However, if your cdef class contains a Python object (including another cdef class) you should copy it like this:
cdef class C:
cdef object o
def __deepcopy__(self,memo_dictionary):
# ...
res.o = deepcopy(self.o,memo_dictionary)
# ...
(i.e. make sure it gets passed on to further calls of deepcopy)

Set coefficient/element of Eigen::Matrix3d in Cython

I am trying to create a wrapper in Cython for a library which uses Eigen::Matrix3d matrices. How can I set an individual element/coefficient of the Matrix3d object?
I know, I can get the value with the coeff(row, col) method but could not find any function set_coeff(row, col, value) - or however that might be called - to set the value.
After declaring the Matrix3d with
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
I want to set the values, but none of the following constructs work in Cython:
t << 1,2,3,4,5,6,7,8,9
t(0,0) = 1
t[0][0] = 1
and I cannot use a constructor with the values, because to my knowledge there does not exist any.
Here are the files I have come up so far:
decl_eigen.pxd:
cdef extern from "Eigen/Dense" namespace "Eigen":
cdef cppclass Vector3d:
Matrix3d() except +
double coeff(int row, int col)
decl_foo.pxd:
cimport decl_eigen
cdef extern from "../foo.hpp" namespace "MyFoo":
cdef cppclass Bar:
Bar() except +
void transform(decl_eigen.Matrix3d &transformation)
foo.pyx:
import decl_eigen
cimport decl_foo
cdef class Bar:
cdef decl_foo.Bar *thisptr
def __cinit__(self):
self.thisptr = new decl_foo.Bar()
def __dealloc__(self):
del self.thisptr
def transform(self, transformation):
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
for i in range(3):
for j in range(3):
k = i*3 + j
# Set the coefficient of t(i,j) to transformation[k], but how????
self.thisptr.transform(t)
Thanks.
It's not as straightforward as it should be, but you can make it work.
Element access in Eigen looks to mostly be done through operator():
// (copied from http://eigen.tuxfamily.org/dox/GettingStarted.html)
MatrixXd m(2,2);
m(0,0) = 3;
m(1,0) = 2.5;
m(0,1) = -1;
m(1,1) = m(1,0) + m(0,1);
Therefore, we need to define operator() so you can access it in Cython. I've assumed it returns a double& - I can't actually find the definition in Eigen since it's buried deep in a template class hierarchy (It's not terribly important what it actually returns - it acts like it returns a double&, which should be good enough).
Unfortunately, operator() seems slightly broken in Cython (see Cython C++ wrapper operator() overloading error) so we have to alias it as something else. I've used element.
cdef extern from "eigen3/Eigen/Dense" namespace "Eigen":
# I'm also unsure if you want a Matrix3d or a Vector3d
# so I assumed matrix
cdef cppclass Matrix3d:
Matrix3d() except +
double& element "operator()"(int row,int col)
In principle we'd just like to be able to do m.element(0,0) = 5. However, Cython doesn't like this. Therefore, I've had to create a function which does this through a slightly complicated assignment to pointer type mechanism.
cdef void set_matrix_element(Matrix3d& m, int row, int col, double elm):
cdef double* d = &(m.element(row,col))
d[0] = elm
Therefore, to set a matrix element, we just call this function. Here's the function I made to test it on:
def get_numbers():
cdef Matrix3d m = Matrix3d()
cdef int i
for i in range(3):
set_matrix_element(m,i,i,i)
return m.element(0,0),m.element(1,1),m.element(2,2),m.element(1,2)
# returns 0,1,2, and something else (I get 0, but in principle
# I think it's undefined since it's never been specifically set)

Cython - copy constructors

I've got a C library that I'm trying to wrap in Cython. One of the classes I'm creating contains a pointer to a C structure. I'd like to write a copy constructor that would create a second Python object pointing to the same C structure, but I'm having trouble, as the pointer cannot be converted into a python object.
Here's a sketch of what I'd like to have:
cdef class StructName:
cdef c_libname.StructName* __structname
def __cinit__(self, other = None):
if not other:
self.__structname = c_libname.constructStructName()
elif type(other) is StructName:
self.__structname = other.__structname
The real problem is that last line - it seems Cython can't access cdef fields from within a python method. I've tried writing an accessor method, with the same result. How can I create a copy constructor in this situation?
When playing with cdef classes, attribute access are compiled to C struct member access. As a consequence, to access to a cdef member of an object A you have to be sure of the type of A. In __cinit__ you didn't tell Cython that other is an instance of StructName. Therefore Cython refuses to compile other.__structname. To fix the problem, just write
def __cinit__(self, StructName other = None):
Note: None is equivalent to NULL and therefore is accepted as a StructName.
If you want more polymorphism then you have to rely on type casts:
def __cinit__(self, other = None):
cdef StructName ostr
if not other:
self.__structname = c_libname.constructStructName()
elif type(other) is StructName:
ostr = <StructName> other
self.__structname = ostr.__structname