I am trying to use swig to generate interface on some existing C++ methods. I am referring to the following link for practice to generate on c++ template methods
https://valelab4.ucsf.edu/svn/3rdpartypublic/swig/Examples/python/template/
File : example.h
// Some template definitions
template<class T> T max(T a, T b) { return a>b ? a : b; }
template<class T> class vector {
T *v;
int sz;
public:
vector(int _sz) {
v = new T[_sz];
sz = _sz;
}
T &get(int index) {
return v[index];
}
void set(int index, T &val) {
v[index] = val;
}
#ifdef SWIG
%extend {
T getitem(int index) {
return $self->get(index);
}
void setitem(int index, T val) {
$self->set(index,val);
}
}
#endif
};
File : example.i
%module example
%{
#include "example.h"
%}
/* Let's just grab the original header file here */
%include "example.h"
/* Now instantiate some specific template declarations */
%template(maxint) max<int>;
%template(maxdouble) max<double>;
%template(vecint) vector<int>;
%template(vecdouble) vector<double>;
setup.py
from distutils.core import setup, Extension
#name of module
name = "example"
#version of module
version = "1.0"
# specify the name of the extension and source files
# required to compile this
ext_modules = Extension(name='_example',sources=["example.i","example.h"])
setup(name=name,
version=version,
ext_modules=[ext_modules])
step 1: compile module
python setup.py build_ext
step 2: install in the current directory
python setup.py install --install-platlib=.
I am seeing the following errors as the output
python setup.py build_ext
running build_ext
building '_example' extension
swigging example.i to example_wrap.c
swig -python -o example_wrap.c example.i
example.h:5: Warning 301: class keyword used, but not in C++ mode.
example.h:5: Error: Syntax error - possibly a missing semicolon.
error: command 'swig' failed with exit status 1
Use the following in setup.py to pass the -c++ option to SWIG. Also remove example.h from sources or the compiler tries to compile the header. It's already included in the generated file.
ext_modules = Extension(name='_example', sources=["example.i"], swig_opts=['-c++'])
Related
When calling a swig generated function returning std::tuple, i get a swig object of that std::tuple.
Is there a way to use type-maps or something else to extract the values? I have tried changing the code to std::vector for a small portion of the code, and that works. (using %include <std_vector.i> and templates) But i don't want to make too many changes in the C++ part.
Edit: here is a minimal reproducible example:
foo.h
#pragma once
#include <tuple>
class foo
{
private:
double secret1;
double secret2;
public:
foo();
~foo();
std::tuple<double, double> return_thing(void);
};
foo.cpp
#include "foo.h"
#include <tuple>
foo::foo()
{
secret1 = 1;
secret2 = 2;
}
foo::~foo()
{
}
std::tuple<double, double> foo::return_thing(void) {
return {secret1, secret2};
}
foo.i
%module foo
%{
#include"foo.h"
%}
%include "foo.h"
When compiled on my linux using
-:$ swig -python -c++ -o foo_wrap.cpp foo.i
-:$ g++ -c foo.cpp foo_wrap.cpp '-I/usr/include/python3.8' '-fPIC' '-std=c++17' '-I/home/simon/Desktop/test_stack_overflow_tuple'
-:$ g++ -shared foo.o foo_wrap.o -o _foo.so
I can import it in python as shown:
test_module.ipynb
import foo as f
Foo = f.foo()
return_object = Foo.return_thing()
type(return_object)
print(return_object)
Outputs is
SwigPyObject
<Swig Object of type 'std::tuple< double,double > *' at 0x7fb5845d8420>
Hopefully this is more helpful, thank you for responding
To clarify i want to be able to use the values in python something like this:
main.cpp
#include "foo.h"
#include <iostream>
//------------------------------------------------------------------------------'
using namespace std;
int main()
{
foo Foo = foo();
auto [s1, s2] = Foo.return_thing();
cout << s1 << " " << s2 << endl;
}
//------------------------------------------------------------------------------
Github repo if anybody is interested
https://github.com/simon-cmyk/test_stack_overflow_tuple
Our goal is to make something like the following SWIG interface work intuitively:
%module test
%include "std_tuple.i"
%std_tuple(TupleDD, double, double);
%inline %{
std::tuple<double, double> func() {
return std::make_tuple(0.0, 1.0);
}
%}
We want to use this within Python in the following way:
import test
r=test.func()
print(r)
print(dir(r))
r[1]=1234
for x in r:
print(x)
i.e. indexing and iteration should just work.
By re-using some of the pre-processor tricks I used to wrap std::function (which were themselves originally from another answer here on SO) we can define a neat macro that "just wraps" std::tuple for us. Although this answer is Python specific it should in practice be fairly simple to adapt for most other languages too. I'll post my std_tuple.i file, first and then annotate/explain it after:
// [1]
%{
#include <tuple>
#include <utility>
%}
// [2]
#define make_getter(pos, type) const type& get##pos() const { return std::get<pos>(*$self); }
#define make_setter(pos, type) void set##pos(const type& val) { std::get<pos>(*$self) = val; }
#define make_ctorargN(pos, type) , type v##pos
#define make_ctorarg(first, ...) const first& v0 FOR_EACH(make_ctorargN, __VA_ARGS__)
// [3]
#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1) action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1) action(1,a2) action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1) action(1,a2) action(2,a3) action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1) action(1,a2) action(2,a3) action(3,a4) action(4,a5)
#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef
// [4]
%define %std_tuple(Name, ...)
%rename(Name) std::tuple<__VA_ARGS__>;
namespace std {
struct tuple<__VA_ARGS__> {
// [5]
tuple(make_ctorarg(__VA_ARGS__));
%extend {
// [6]
FOR_EACH(make_getter, __VA_ARGS__)
FOR_EACH(make_setter, __VA_ARGS__)
size_t __len__() const { return std::tuple_size<std::decay_t<decltype(*$self)>>{}; }
%pythoncode %{
# [7]
def __getitem__(self, n):
if n >= len(self): raise IndexError()
return getattr(self, 'get%d' % n)()
def __setitem__(self, n, val):
if n >= len(self): raise IndexError()
getattr(self, 'set%d' % n)(val)
%}
}
};
}
%enddef
This is just the extra includes we need for our macro to work
These apply to each of the type arguments we supply to our %std_tuple macro invocation, we need to be careful with commas here to keep the syntax correct.
This is the mechanics of our FOR_EACH macro, which invokes each action per argument in our variadic macro argument list
Finally the definition of %std_tuple can begin. Essentially this is manually doing the work of %template for each specialisation of std::tuple we care to name inside of the std namespace.
We use our macro for each magic to declare a constructor with arguments for each element of the correct type. The actual implementation here is the default one from the C++ library which is exactly what we need/want though.
We use our FOR_EACH macro twice to make a member function get0, get1, getN of the correct type of each tuple element and the correct number of them for the template argument size. Likewise for setN. Doing it this way allows the usual SWIG typemaps for double, etc. or whatever types your tuple contains to be applied automatically and correctly for each call to std::get<N>. These are really just an implementation detail, not intended to be part of the public interface, but exposing them makes no real odds.
Finally we need an implementation of __getitem__ and a corresponding __setitem__. These simply look up and call the right getN/setN function on the class and call that instead. We take care to raise IndexError instead of the default exception if an invalid index is used as this will stop iteration correctly when we try to iterate of the tuple.
This is then sufficient that we can run our target code and get the following output:
$ swig3.0 -python -c++ -Wall test.i && g++ -shared -o _test.so test_wrap.cxx -I/usr/include/python3.7 -m32 && python3.7 run.py
<test.TupleDD; proxy of <Swig Object of type 'std::tuple< double,double > *' at 0xf766a260> >
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__swig_destroy__', '__swig_getmethods__', '__swig_setmethods__', '__weakref__', 'get0', 'get1', 'set0', 'set1', 'this']
0.0
1234.0
Generally this should work as you'd hope in most input/output situations in Python.
There are a few improvements we could look to make:
Implement repr
Implement slicing so that tuple[n:m] type indexing works
Handle unpacking like Python tuples.
Maybe do some more automatic conversions for compatible types?
Avoid calling __len__ for every get/setitem call, either by caching the value in the class itself, or postponing it until the method lookup fails?
I'm trying to extern a c++ function to cython. Here is my code (all files are in the same directory)
function.cpp
int cfunc(int x){
return x;
}
wrapper.pyx
cdef extern from "function.cpp":
cpdef int cfunc(int)
def pyfunc(int x):
return cfunc(x)
setup.py
from distutils.core import setup, Extension
from Cython.Build import cythonize
source = ['function.cpp', 'wrapper.pyx']
ext = [Extension('lib', source, language='c++')]
setup(ext_modules=cythonize(ext))
When I run python setup.py build_ext --inplace it gives the following error
/home/hyunix/anaconda3/envs/c-playground/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: build/temp.linux-x86_64-3.7/function.o: in function `cfunc(int)':
function.cpp:(.text._Z5cfunci+0x0): multiple definition of `cfunc(int)'; build/temp.linux-x86_64-3.7/wrapper.o:wrapper.cpp:(.text._Z5cfunci+0x0): first defined here
collect2: error: ld returned 1 exit status
error: command '/home/hyunix/anaconda3/envs/c-playground/bin/x86_64-conda_cos6-linux-gnu-c++' failed with exit status 1
However if I remove language='c++' from setup.py it works fine. Why does this happen?
I'm using:
Python 3.7.9
Cython 0.29.21
Ubuntu 20.04
Well, when you use cpdef int cfunc(int), you're explicitly creating a new C function, and a new python function. If you want to refer to cfunc() as it's externally defined in function.cpp, your signature should be
cdef extern from "function.cpp":
int cfunc(int)
So, when you compile with the language='c++' flag, Cython is giving you an appropriate error. However, when you remove the language flag, Cython needs to reason based on compiler directives whether you're asking for a .c or a .cpp, and it defaults to .c. You should notice that your wrapper is being compiled to .c instead of .cpp when the language argument is removed. In this C compilation, Cython does not recognize the signature in the .cpp, but it does recognize the cpdef. So, no error, but you're getting an empty cfunc function, as opposed to the one defined in cpp.
Scanning dependencies of target caffeproto
[ 0%] Building CXX object src/caffe/CMakeFiles/caffeproto.dir/__/__/include/caffe/proto/caffe.pb.cc.o
In file included from /Users/cm/caffe/build/include/caffe/proto/caffe.pb.cc:4:
In file included from /Users/cm/caffe/build/include/caffe/proto/caffe.pb.h:25:
In file included from /usr/local/include/google/protobuf/generated_message_table_driven.h:34:
In file included from /usr/local/include/google/protobuf/map.h:49:
In file included from /usr/local/include/google/protobuf/map_type_handler.h:35:
In file included from /usr/local/include/google/protobuf/wire_format_lite_inl.h:43:
/usr/local/include/google/protobuf/message_lite.h:117:3: error: unknown type
name 'constexpr'
constexpr const T& get() const { return reinterpret_cast<const T&>(union_); }
^
/usr/local/include/google/protobuf/message_lite.h:117:13: error: expected member
name or ';' after declaration specifiers
constexpr const T& get() const { return reinterpret_cast<const T&>(union_); }
~~~~~~~~~ ^
I had the same issue. Here is what to do:
- if you are using cmake try to add -std=c++11 to C flags:
if(UNIX OR APPLE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wall -std=c++11")
endif()
If you are using Make file, try to add the same flag to makefile
Hope this helps.
David
I'm trying to use the Cuhre routine provided in the Cuba library. I previously encountered some errors linking a static library to Cython, so I tried to create a shared library with the Cuhre attribute. To do this, I have three files: cuhre.c, cuhre.h, and libcuhre.so (created by compiling cuhre.c).
cuhre.c has a routine tryCuhre that essentially calls on the Cuhre routing provided in the Cuba library. For simplicity, it is just for 2D integration:
double tryCuhre(integrand_t t, void * ud)
{
int comp, nregions, neval, fail;
cubareal integral[NCOMP], error[NCOMP], prob[NCOMP];
Cuhre(2, 1, t, ud, 1,
EPSREL, EPSABS, VERBOSE | LAST,
MINEVAL, MAXEVAL, 13,
STATEFILE, SPIN,
&nregions, &neval, &fail, integral, error, prob);
return (double)integral[0];
}
The variables in all caps (e.g. MINEVAL and SPIN) are all predefined at compile time and constant.
This is my cuhre.h file, which is included by cuhre.c:
#ifndef CUHRE_H_
#define CUHRE_H_
#ifdef __cplusplus
extern "C" {
#endif
typedef double cubareal;
typedef int (*integrand_t)(const int *ndim, const cubareal x[], const int
*ncomp, cubareal f[], void *userdata);
double tryCuhre(integrand_t t, void * ud);
#ifdef __cplusplus
}
#endif
#endif
And after running the set of commands
gcc -Wall -fPIC -c cuhre.c
gcc -shared -o libcuhre.so cuhre.o
I am able to create the shared library libcuhre.so. So far so good. It should be noted that up to this point, the routine works just as I want, i.e. making the executable cuhre from cuhre.c performs correctly.
I am trying to use the tryCuhre routine in a cython file now (execute.pyx). At the top, I have the declarations:
cdef extern from "math.h":
double sin(double x)
double cos(double x)
double sqrt(double x)
double atan(double x)
double exp(double x)
double log(double x)
cdef extern from "cuhre.h":
ctypedef double cubareal
ctypedef int (*integrand_t)(const int *ndim, const cubareal x[], const int *ncomp, cubareal f[], void *userdata)
double tryCuhre(integrand_t t, void * ud)
Finally, to compile, I am using the command
python setup.py build_ext --inplace
on setup.py, which is as follows:
from distutils.core import setup, Extension
from Cython.Distutils import build_ext
from Cython.Build import cythonize
from distutils.extension import Extension
sourcefiles = ['execute.pyx']
ext_modules = [Extension("execute", sourcefiles, library_dirs =
['~/Documents/project/libcuhre.so'],)]
setup(
name = 'execute',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
which compiles the file. However, whenever I try the statement
import execute
in the python shell, it raises the error:
/Documents/project/execute.so: undefined symbol: tryCuhre
I have looked all around for ways to link my artificially created libcuhre.so library, but so far none have worked. How can this issue be fixed? Furthermore, how come my program is able to find all of the methods from the math library (sin, cos, exp, etc.) but not any from my libcuhre.so? (It should also be noted that all of these files are in the same directory, ~/Documents/project.)
Thank you so much for any help!
Dependent source code and libraries need to be included in the Extension, either as libraries to link against, or source files to compile with.
library_dirs only adds to the directories the linker will search for a library but does not link anything so is not sufficient.
In this case, since the C code is self-built and a single .c, it is easiest to compile it together as a source. This also means cuhre.c will be compiled by setuptools itself, automating its compilation on the target distribution.
from setuptools import setup, Extension
from Cython.Distutils import build_ext
from Cython.Build import cythonize
sourcefiles = ['execute.pyx', 'cuhre.c']
ext_modules = [Extension("execute", sourcefiles,
include_dirs=['.'],
depends='cuhre.h',
)]
setup(
name='execute',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
Code also changed to use setuptools, distutils is deprecated and now part of setuptools.
I am working on app which uses tcl package implemented in C++ and linked as static library (app is developed long time ago). It does following:
// Library code
extern "C" int testlib_SafeInit _ANSI_ARGS_((Tcl_Interp *interp))
{
return Tcl_PkgProvide(interp, "testlib", "1.6");
}
extern "C" int testlib_Init _ANSI_ARGS_((Tcl_Interp *interp))
{
return testlib_SafeInit(interp);
}
// Application code
extern "C" int testlib_SafeInit _ANSI_ARGS_((Tcl_Interp *interp));
extern "C" int testlib_Init _ANSI_ARGS_((Tcl_Interp *interp));
int main()
{
Tcl_Interp* interp = Tcl_CreateInterp();
Tcl_Init(interp);
Tcl_PkgProvide(interp, "testlib", "1.6");
Tcl_StaticPackage(interp, "testlib", testlib_Init, testlib_SafeInit);
Tcl_Eval(interp, "package require testlib");
std::cout << "Res = " << Tcl_GetStringResult(interp);
return 0;
}
When I am removing line Tcl_PkgProvide(interp, "testlib", "1.6"); from main, package becomes invisible. Also I have noticed that testlib_Init and testlib_SafeInit are not called. I am expecting that they must be called from package require testlib. As I understand from docs each package must have pkgIndex.tcl in auto_path or tcl_pkgPath which must contain line
(package ifneeded testlib 1.6 {load {} testlib}), but here both variables does not contain such index file.
Is this a correct way of providing packages? Is there a documentation related with providing packages using static libraries?
Well, the simplest technique for statically providing a package is to just install it directly. The package init code should be the one calling Tcl_PkgProvide — you don't do so from main() usually — and you probably don't need Tcl_StaticPackage at all unless you're wanting to install the code into sub-interpreters.
int main(int argc, char*argv[])
{
Tcl_FindExecutable(argv[0]);
Tcl_Interp* interp = Tcl_CreateInterp();
Tcl_Init(interp);
testlib_Init(interp);
// OK, setup is now done
Tcl_Eval(interp, "package require testlib");
std::cout << "Res = " << Tcl_GetStringResult(interp) << "\n";
return 0;
}
However, we can move to using Tcl_StaticPackage. That allows code to say “instead of loading a DLL with this sort of name, I already know that code: here are its entry points”. If you are doing that, you need to also install a package ifneeded script; those are done through the script API only.
int main(int argc, char*argv[])
{
Tcl_FindExecutable(argv[0]);
Tcl_Interp* interp = Tcl_CreateInterp();
Tcl_Init(interp);
Tcl_StaticPackage(interp, "testlib", testlib_Init, testlib_SafeInit);
Tcl_Eval(interp, "package ifneeded testlib 1.6 {load {} testlib}");
// OK, setup is now done
Tcl_Eval(interp, "package require testlib");
std::cout << "Res = " << Tcl_GetStringResult(interp) << "\n";
return 0;
}
The testlib in the load call needs to match the testlib in the Tcl_StaticPackage call. The testlib in the package require, package ifneeded and Tcl_PkgProvide also need to all match (as do the occurrences of 1.6, the version number).
Other minor issues
Also, you don't need to use the _ANSI_ARGS_ wrapper macro. That's utterly obsolete, for really ancient and crappy compilers that we don't support any more. Just replace _ANSI_ARGS_((Tcl_Interp *interp)) with (Tcl_Interp *interp). And remember to call Tcl_FindExecutable first to initialise the static parts of the Tcl library. If you don't have argv[0] available to pass into it, use NULL instead; it affects a couple of more obscure introspection systems on some platforms, but you probably don't care about them. However, initialising the library overall is very useful: for example, it lets you make sure that the filesystem's filename encoding scheme is correctly understood! That can be a little important to code…