I have a C-header file (let's call it myheader.h) that contains some character string definitions such as:
#define MYSTRING "mystring-constant"
In Cython, I create a cmy.pxd file that contains:
cdef extern from "myheader.h":
cdef const char* MYSTRING "MYSTRING"
and a corresponding my.pyx file that contains some class definitions, all headed by:
from cmy cimport *
I then try to reference that string in a Python script:
from my import *
def main():
print("CONSTANT ", MYSTRING)
if __name__ == '__main__':
main()
Problem is that I keep getting an error:
NameError: name 'MYSTRING' is not defined
I've searched the documentation and can't identify the problem. Any suggestions would be welcomed - I confess it is likely something truly silly.
You cannot access cdef-variables from Python. So you have to create a Python object which would correspond to your define, something like this (it uses Cython>=0.28-feature verbatim-C-code, so you need a recent Cython version to run the snippet):
%%cython
cdef extern from *:
"""
#define MYSTRING "mystring-constant"
"""
# avoid name clash with Python-variable
# in cdef-code the value can be accessed as MYSTRING_DEFINE
cdef const char* MYSTRING_DEFINE "MYSTRING"
#python variable, can be accessed from Python
#the data is copied from MYSTRING_DEFINE
MYSTRING = MYSTRING_DEFINE
and now MYSTRING is a bytes-object:
>>> print(MYSTRING)
b'mystring-constant'
Related
I am trying to use a c DLL in cython and during the compilation phase I get a warning from the C compiler :
warning C4133: "=" : incompatible types - from 'foobar *' to 'foobar *'.
My pxd looks like this :
#!/usr/bin/env python3
#cython: language_level=3
cdef extern from "typedef.h"
struct foobar:
long *index
double *my_array
int value
cdef extern from "functions.h"
foobar *get_foobar(char *name);
And my pyx like that :
cimport pxd_file_name
cdef class Handler:
cdef pxd_file_name.foobar *__foobar
def load_foobar(self, char *name):
self.__foobar = pxd_file_name.get_foobar(name) <==
def another_method(self):
pass
I got the warning because of the line marked by an arrow and I don't understand why.
Is there a way to fix this ?
I manage to found my mistake.
Because in my .h file, my struct was declared using typedef, I had to write ctypedef struct foobar instead of struct foobar in my pxd file
I have a cython extension type that I want to make more general. One of the attributes of this extension type is a double and I want it to be a memoryview (double[::1]) when needed.
Here is a simple example :
import numpy as np
cimport numpy as np
cimport cython
cdef class Test:
cdef bint numeric
cdef double du
def __init__(self, bint numeric):
self.numeric = numeric
if self.numeric:
self.du = 1
else:
self.du = np.ones(10)
def disp(self)
print(self.du)
Test(True).disp() # returns 1
Test(False).disp() # gives of course an error
I tried to subclass Test changing du type to double[::1] and implementing a new __init__ but it seems that we can't override class attributes of extension types. Even if it worked, it wouldn't be satisfactory because I don't really want to have one extension type for each case.
The best would be that my extension type directly handle both cases (scalar du and memoryview du).
Is there a way to do this with Cython ?
Unfortunately, you cannot use fused_type as attributes type. You can have two options here:
You could try to use the memory adress of the variable you want to call, and cast it when needed (everything is explained here.) Unfortunately, I did not succeed at making it work with typed memory views.
Or you can use your defined attribute numeric to call the appropriate method:
import numpy as np
cimport numpy as np
cimport cython
cdef class Test:
cdef bint numeric
cdef double du_numeric
cdef double[:] du_mem_view
def __init__(self, bint numeric):
self.numeric = numeric
if self.numeric:
self.du_numeric = 1
else:
self.du_mem_view = np.ones(10)
def disp(self):
if self.numeric:
print(self.du_numeric)
else:
print(self.du_numeric_mem_view)
Test(True).disp() # returns 1
Test(False).disp() # Does not give an error anymore !
Methods go unrecognized when I iterate over objects of the same type (or of derived class) and call the same method within a loop.
This happens when I iterate over objects stored in a data structure (in this case, collections.OrderedDict) within a class I've defined. I've made the methods available to Cython in a pxd file.
This happens whether or not MyClass is a base class or derived class.
I'm using unittest to test my code.
I'm wondering if Cython doesn't support this or if some information about an object is missing when it's a Cython extension.
myclass.pxd
cdef class MyClass():
cdef public object _dict
cdef void add_obj(self, name, obj)
cdef void m(self)
cdef void _m(self)
myclass.pyx
from collections import OrderedDict
cdef class MyClass():
def __cinit__(self):
self._dict = OrderedDict()
cdef void add_obj(self, name, obj):
self._dict[name] = obj
cdef void m(self):
# user defines this in derived class
pass
cdef void _m(self):
cdef int i = 0
print('running user defined method')
self.m()
print(self._dict.keys())
print('adding objects')
for key, obj in self._dict.items():
# ERROR
print('obj')
print(obj)
obj.m()
i += 1
print('added an object')
print(i)
test.pyx
from mypkg.core.myclass cimport MyClass
import unittest
cdef class ChildClass(MyClass):
cdef void m(self):
self.add_obj('a', MyClass())
self.add_obj('b', MyClass())
self.add_obj('c', MyClass())
cdef class ParentClass(MyClass):
cdef void m(self):
self.add_obj('a', ChildClass())
self.add_obj('b', ChildClass())
self.add_obj('c', ChildClass())
cdef ParentClass H = ParentClass()
class SetupTree(unittest.TestCase):
def setUp(self):
H._m()
def test_tree(self):
print(H._dict)
print(len(H._dict))
Output
running user defined method
odict_keys(['a', 'b', 'c'])
adding objects
obj
<mypkg.test.test_system2.ChildClass object at 0x7f86ff915b38>
AttributeError: 'mypkg.test.test_system2.ChildClass' object has no attribute 'm'
Exception ignored in: 'mypkg.core.group.MyClass._m'
AttributeError: 'mypkg.test.test_system2.ChildClass' object has no attribute 'm'
OrderedDict([('a', <mypkg.test.test_system2.ChildClass object at 0x7f86ff915b38>), ('b', <mypkg.test.test_system2.ChildClass object at 0x7f86ff915d30>), ('c', <mypkg.test.test_system2.ChildClass object at 0x7f86ff915c88>)])
3
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
cdef methods can only be called if they can be found at compile time - they can't be looked up as Python attributes at runtime. You've done nothing in _m to tell Cython that obj is a MyClass, so it does not know of the existence of the cdef method.
You have two options:
Change the cdef m(...) to def (or cpdef) so it can be found by Python at runtime.
Tell Cython that obj is an instance of MyClass:
cdef MyClass obj # you'll receive an TypeError exception if you ever
# attempt to assign something that isn't a MyClass to obj
for key, obj in self._dict.items():
obj.m() # now works!
I'm chasing my tail with what I suspect is a simple problem, but I can't seem to find any explanation for the observed behavior. Assume I have a constant in a C header file defined by:
#define FOOBAR 128
typedef uint32_t mytype_t;
I convert this in Cython by putting the following in the .pxd file:
cdef int _FOOBAR "FOOBAR"
ctypedef uint32_t mytype_t
In my .pyx file, I have a declaration:
FOOBAR = _FOOBAR
followed later in a class definition:
cdef class MyClass:
cdef mytype_t myvar
def __init__(self):
try:
self.myvar = FOOBAR
print("GOOD")
except:
print("BAD")
I then try to execute this with a simple program:
try:
foo = MyClass()
except:
print("FAILED TO CREATE CLASS")
Sadly, this errors out, but I don't get an error message - I just get the exception print output:
BAD
Any suggestions on root cause would be greatly appreciated.
I believe I have finally tracked it down. The root cause issue is that FOOBAR in my code was actually set to UINT32MAX. Apparently, Cython/Python interprets that as a -1 and Python then rejects setting a uint32_t variable equal to it. The solution is to define FOOBAR to be 0xffffffff - apparently Python thinks that is a non-negative value and accepts it.
This seems like a question that should have an obvious answer, but for some reason I can't find any examples online.
I am wrapping a vector of C++ objects in a Python class using Cython. I also have a Cython wrapper for the C++ class already coded. I can get several methods such as __len__(), __getitem__(), and resize() to work properly, but the __setitem__() method is giving me problems.
For simplicity, I coded a small example using a vector of ints. I figure if I can get this code to work, then I can build on that to get the solution for my C++ class as well.
MyPyModule.pyx
# distutils: language = c++
from libcpp.vector cimport vector
from cython.operator cimport dereference as deref
cdef class MyArray:
cdef vector[int]* thisptr
def __cinit__(self):
self.thisptr = new vector[int]()
def __dealloc__(self):
del self.thisptr
def __len__(self):
return self.thisptr.size()
def __getitem__(self, size_t key):
return self.thisptr.at(key)
def resize(self, size_t newsize):
self.thisptr.resize(newsize)
def __setitem__(self, size_t key, int value):
# Attempt 1:
# self.thisptr.at(key) = value
# Attempt 2:
# cdef int* itemptr = &(self.thisptr.at(key))
# itemptr[0] = value
# Attempt 3:
# (self.thisptr)[key] = value
# Attempt 4:
self[key] = value
When I tried to cythonize using Attempt 1, I got the error Cannot assign to or delete this. When I tried Attempt 2, the .cpp file was created, but the compiler complained that:
error: cannot convert ‘__Pyx_FakeReference<int>*’ to ‘int*’ in assignment
__pyx_v_itemptr = (&__pyx_t_1);
On Attempt 3, Cython would not build the file because Cannot assign type 'int' to 'vector[int]'. (When I tried this style with the C++ object instead of int, it complained because I had a reference as a left-value.) Attempt 4 compiles, but when I try to use it, I get a segfault.
Cython docs say that returning a reference as a left-value is not supported, which is fine -- but how do I get around it so that I can assign a new value to one of my vector elements?
There are two ways to access the vector through a pointer,
def __setitem__(self, size_t key, int value):
deref(self.thisptr)[key] = value
# or
# self.thisptr[0][key] = value
Cython translates those two cases as follows:
Python: deref(self.thisptr)[key] = value
C++: ((*__pyx_v_self->thisptr)[__pyx_v_key]) = __pyx_v_value;
Python: self.thisptr[0][key] = value
C++: ((__pyx_v_self->thisptr[0])[__pyx_v_key]) = __pyx_v_value;
which are equivalent i.e. access the same vector object.
Instead of trying to handle a pointer from Cython code, you can let Cython itself do it for you:
cdef class MyArray:
cdef vector[int] thisptr
def __len__(self):
return self.thisptr.size()
def __getitem__(self, size_t key):
return self.thisptr[key]
def __setitem__(self, size_t key, int value):
self.thisptr[key] = value
def resize(self, size_t newsize):
self.thisptr.resize(newsize)
Is there any problem with this approach?
I have already accepted J.J. Hakala's answer (many thanks!). I tweaked that method to include an out-of-bounds check, since it uses the [] operator instead of the at() method:
cdef class MyArray:
(....)
def __setitem__(self, size_t key, int value):
if key < self.thisptr.size():
deref(self.thisptr)[key] = value
else:
raise IndexError("Index is out of range.")