How can I declare constants (of various types - not just enum values) and share them between multiple .pyx files?
Within a .pyx file, I can declare and use constants:
DEF FavouriteFood = "spam"
DEF ArraySize = 42
print(FavouriteFood)
print(ArraySize)
And using a .pxd file, I can share C functions or share types.
But how can I share constants? The docs indicate a very limited way of doing this with anonymous enums, but values in those can only be ints. How can I import, for example, a constant string or float into a .pyx file?
You could use a very short inline function (in the pxd file) that just returns the constant
cdef inline const char* GetFavouriteFood():
return "spam"
cdef inline float GetHowMuch():
return 4.5
The other option would be to define the constants in C in a header file then (in your pxd) do
cdef extern from "myconstants.h":
const char* FavouriteFood
float HowMuch
The C compiler (rather than Cython) enforces the constness so errors will appear at that stage if you try to change them. This does involve create an extra file so personally I prefer the inline function approach.
Edit (2018):
You can now include C code directly in Cython which makes the second approach easier:
cdef extern from *:
"""const char* FavouriteFood = "spam";
const float HowMuch = 4.5;"""
const char* FavouriteFood
float HowMuch
Related
I want to use a C++ shared_ptr as a replacement for raw C pointers. As a simple example the following code seems to work as intended:
from libcpp.memory cimport shared_ptr, allocator
cdef shared_ptr[double] spd
cdef allocator[double] allo
spd.reset(allo.allocate(13))
The size is chosen as 13 here, but in general is not know at compile time.
I'm not sure if this is correct, but I haven't had any errors (no memory leaks and segfaults yet). I'm curious if there is a more optimal solution with make_shared.
But I can't use C++11 arrays because Cython doesn't allow literals as templates, e.g. something like
cdef shared_ptr[array[double]] spd = make_shared[array[double,13]]()
and "normal" arrays which are supposed to work with C++20 compiler (e.g. gcc 10) are causing problems in one way or another:
# Cython error "Expected an identifier or literal"
cdef shared_ptr[double[]] spd = make_shared[double[]](3)
# Can't get ptr to work
ctypedef double[] darr
cdef shared_ptr[darr] spd = make_shared[darr](13)
cdef double* ptr = spd.get() # or spd.get()[0] or <double*> spd.get()[0] or ...
Is the allocator solution the correct and best one or is there another way how to do it?
Here is what I'm going with
cdef extern from *:
"""
template <typename T>
struct Ptr_deleter{
size_t nn;
void (*free_ptr)(T*, size_t);
Ptr_deleter(size_t nIn, void (*free_ptrIn)(T*, size_t)){
this->nn = nIn;
this->free_ptr = free_ptrIn;
};
void operator()(T* ptr){
free_ptr(ptr, nn);
};
};
template <typename T>
std::shared_ptr<T> ptr_to_sptr (T* ptr, size_t nn, void (*free_ptr)(T*, size_t)) {
Ptr_deleter dltr(nn, free_ptr);
std::shared_ptr<T> sp (ptr, dltr);
return sp;
};
"""
shared_ptr[double] d_ptr_to_sptr "ptr_to_sptr"(double* ptr, size_t nn, void (*free_ptr)(double*, size_t) nogil) nogil
cdef void free_d_ptr(double* ptr, size_t nn) nogil:
free(ptr)
cdef shared_ptr[double] sp_d_empty(size_t nn) nogil:
return d_ptr_to_sptr(<double*> nullCheckMalloc(nn*sizeof(double)), nn, &free_d_ptr)
My understanding is that the "right" way to handle malloced arrays is to use a custom deleter like I did. I personally prefer sticking with somewhat-raw C pointers (double* instead of double[] or something), since it's more natural in Cython and my projects.
I think it's reasonably easy to see how to change free_ptr for more complicated data types. For simple data types it could be done in less lines and less convoluted, but I wanted to have the same base.
I like my solution in the regard that I can just "wrap" existing Cython/C code raw pointers in a shared_ptr.
When working with C++ (especially newer standards like C++20) I think verbatim code is pretty often necessary. But I've intentionally defined free_d_ptr in Cython, so it's easy to use existing Cython code to handle the actual work done to free/clear/whatever the array.
I didn't get C++11 std::arrays to work, and it's apparently not "properly" possible in Cython in general (see Interfacing C++11 array with Cython).
I didn't get double[] or similar to work either (is possible in C++20), but with verbatim C++ code I think this should be doable in Cython. I prefer more C-like pointers/arrays anyway as I said.
How can I tell Cython that my C++ class has overloaded operator=? I tried:
cdef extern from "my_source.H":
cdef cppclass MyStatus:
void operator=(const char* status)
cdef public void setStatus(MyStatus& status):
status = "FOO"
but Cython either complains "Assignment to reference status" or (if I make status a non-reference) constructs a python object out of the string "FOO" and then tries to assign the python object to status.
The problem in your code is, that for Cython "FOO" is a Python-object. For expressions like
char *s = "FOO"
Cython is clever enough to understand, what you want and automatically interprets "FOO" as char *.
However, Cython doesn't really "understand"/interpret the signatures of wrapped c++-functions (for that it must be a c++-compiler) and thus cannot know, that you want "FOO" be a char *.
Thus you have to help Cython, for example:
status = <const char *>"FOO"
You also have to work around the problem with reference, for example via:
cdef public void setStatus(MyStatus *status):
status[0] = <const char *>"FOO"
or if you want to have keep the signature of the function intact:
cdef public void setStatus(MyStatus& status):
cdef MyStatus * as_ptr = &status
as_ptr[0] = <const char *>"FOO"
I'm not completely sure the problem with the assigment to reference isn't a bug.
Another observation: the assigment operators aren't part of the "official" wrap of the standard containers, see here or here.
I have successfully wrapped some of the api functions of a C library with Cython. I would now like to expose some of the global C arrays used within the library but I am having a tough time figuring out how to do it.
The array pointer is initialized and allocated inside epanet.c:
void initpointers()
{
H = NULL;
}
int allocdata()
{
n = MaxNodes + 1;
H = (double *) calloc(n, sizeof(double));
}
It is made global in a header file (vars.h):
extern double *H;
I can access the values in the array within the C code of course by doing something like:
for (i=1; i<=Nnodes; i++)
{
H[i] = some_value;
}
I would like to write a function in my pyepanet.pyx file like:
def printH(int Nnodes, double *H):
for i in range(1, Nnodes+1):
print H[i]
but, I am missing the necessary connection between C and cython/python. I have searched quite a bit for an answer but with no luck. Please help.
I finally found something that works. It seems that I can declare the pointer without referring to the header file. When I was trying to refer to the header file before, I think I was getting errors due to redefining items. Here is the code from my .pyx file that works now:
cdef:
extern double *H
extern int Nnodes
def printH():
cdef int i
for i in range(1,Nnodes+1):
print round(H[i], 2)
I am trying to create a wrapper in Cython for a library which uses Eigen::Matrix3d matrices. How can I set an individual element/coefficient of the Matrix3d object?
I know, I can get the value with the coeff(row, col) method but could not find any function set_coeff(row, col, value) - or however that might be called - to set the value.
After declaring the Matrix3d with
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
I want to set the values, but none of the following constructs work in Cython:
t << 1,2,3,4,5,6,7,8,9
t(0,0) = 1
t[0][0] = 1
and I cannot use a constructor with the values, because to my knowledge there does not exist any.
Here are the files I have come up so far:
decl_eigen.pxd:
cdef extern from "Eigen/Dense" namespace "Eigen":
cdef cppclass Vector3d:
Matrix3d() except +
double coeff(int row, int col)
decl_foo.pxd:
cimport decl_eigen
cdef extern from "../foo.hpp" namespace "MyFoo":
cdef cppclass Bar:
Bar() except +
void transform(decl_eigen.Matrix3d &transformation)
foo.pyx:
import decl_eigen
cimport decl_foo
cdef class Bar:
cdef decl_foo.Bar *thisptr
def __cinit__(self):
self.thisptr = new decl_foo.Bar()
def __dealloc__(self):
del self.thisptr
def transform(self, transformation):
cdef decl_eigen.Matrix3d t = decl_eigen.Matrix3d()
for i in range(3):
for j in range(3):
k = i*3 + j
# Set the coefficient of t(i,j) to transformation[k], but how????
self.thisptr.transform(t)
Thanks.
It's not as straightforward as it should be, but you can make it work.
Element access in Eigen looks to mostly be done through operator():
// (copied from http://eigen.tuxfamily.org/dox/GettingStarted.html)
MatrixXd m(2,2);
m(0,0) = 3;
m(1,0) = 2.5;
m(0,1) = -1;
m(1,1) = m(1,0) + m(0,1);
Therefore, we need to define operator() so you can access it in Cython. I've assumed it returns a double& - I can't actually find the definition in Eigen since it's buried deep in a template class hierarchy (It's not terribly important what it actually returns - it acts like it returns a double&, which should be good enough).
Unfortunately, operator() seems slightly broken in Cython (see Cython C++ wrapper operator() overloading error) so we have to alias it as something else. I've used element.
cdef extern from "eigen3/Eigen/Dense" namespace "Eigen":
# I'm also unsure if you want a Matrix3d or a Vector3d
# so I assumed matrix
cdef cppclass Matrix3d:
Matrix3d() except +
double& element "operator()"(int row,int col)
In principle we'd just like to be able to do m.element(0,0) = 5. However, Cython doesn't like this. Therefore, I've had to create a function which does this through a slightly complicated assignment to pointer type mechanism.
cdef void set_matrix_element(Matrix3d& m, int row, int col, double elm):
cdef double* d = &(m.element(row,col))
d[0] = elm
Therefore, to set a matrix element, we just call this function. Here's the function I made to test it on:
def get_numbers():
cdef Matrix3d m = Matrix3d()
cdef int i
for i in range(3):
set_matrix_element(m,i,i,i)
return m.element(0,0),m.element(1,1),m.element(2,2),m.element(1,2)
# returns 0,1,2, and something else (I get 0, but in principle
# I think it's undefined since it's never been specifically set)
So, if you've got a header file.
%%file test.h
struct mystruct{
int i;
int j;
};
And then you wrap it in Cython:
cdef extern from "test.h" nogil:
struct mystruct:
int i
int j
And some function that returns back out to Py:
def spit_out_dict():
return mystruct(5,10)
Cython correctly automatically generates a dict wrapper. However, when I wrap the original C header in a namespace, I haven't been able to get get Cython to still generate the dict wrapper correctly, something along these lines:
%%file test2.h
namespace outerspace{
struct mystruct{
int i;
int j;
};
}
And Cython/Python:
cdef extern from "test2.h" namespace "outerspace" nogil:
struct mynewstruct:
int i
int j
def spit_out_dict():
return mynewstruct(5,10)
This won't compile -- lots of namespace complaint errors -- anyone experienced this before?
Your problem is that Cython seems to only expect namespaces to be used with cppclass. For structs, it generates some functions but just copies the full namespaced name in, causing errors:
static PyObject* __pyx_convert__to_py_outerspace::mystruct(struct outerspace::mystruct s);
^
py_bit.cpp: In function ‘PyObject* __pyx_pf_6py_bit_spit_out_dict(PyObject*)’:
py_bit.cpp:721:15: error: ‘__pyx_convert__to_py_outerspace’ has not been declared
where it's trying to create a function called __pyx_convert__to_py_<classname>. (I think this might be worth submitting a bug report for.)
The trick in such circumstances is usually to lie to Cython. I create three files:
// test2.hpp
namespace outerspace{
struct mystruct{
int i;
int j;
};
}
,
// test2_cy.hpp - a wrapper file purely for Cython's benefit
#include "test2.hpp"
using outerpsace::mystruct;
and the cython file
cdef extern from "test2_cy.hpp": # (I didn't test with "nogil", but it's probably fine...)
struct mynewstruct:
int i
int j
def spit_out_dict():
# for some reason using "return mystruct(5,10)" doesn't work, but this does...
cdef mystruct a = mystruct(5,10)
return a
This is a bug in Cython, fixed at https://github.com/cython/cython/commit/fa946e8435a4dcc3497fc7b0f4e87256d40844ba