When calling a swig generated function returning std::tuple, i get a swig object of that std::tuple.
Is there a way to use type-maps or something else to extract the values? I have tried changing the code to std::vector for a small portion of the code, and that works. (using %include <std_vector.i> and templates) But i don't want to make too many changes in the C++ part.
Edit: here is a minimal reproducible example:
foo.h
#pragma once
#include <tuple>
class foo
{
private:
double secret1;
double secret2;
public:
foo();
~foo();
std::tuple<double, double> return_thing(void);
};
foo.cpp
#include "foo.h"
#include <tuple>
foo::foo()
{
secret1 = 1;
secret2 = 2;
}
foo::~foo()
{
}
std::tuple<double, double> foo::return_thing(void) {
return {secret1, secret2};
}
foo.i
%module foo
%{
#include"foo.h"
%}
%include "foo.h"
When compiled on my linux using
-:$ swig -python -c++ -o foo_wrap.cpp foo.i
-:$ g++ -c foo.cpp foo_wrap.cpp '-I/usr/include/python3.8' '-fPIC' '-std=c++17' '-I/home/simon/Desktop/test_stack_overflow_tuple'
-:$ g++ -shared foo.o foo_wrap.o -o _foo.so
I can import it in python as shown:
test_module.ipynb
import foo as f
Foo = f.foo()
return_object = Foo.return_thing()
type(return_object)
print(return_object)
Outputs is
SwigPyObject
<Swig Object of type 'std::tuple< double,double > *' at 0x7fb5845d8420>
Hopefully this is more helpful, thank you for responding
To clarify i want to be able to use the values in python something like this:
main.cpp
#include "foo.h"
#include <iostream>
//------------------------------------------------------------------------------'
using namespace std;
int main()
{
foo Foo = foo();
auto [s1, s2] = Foo.return_thing();
cout << s1 << " " << s2 << endl;
}
//------------------------------------------------------------------------------
Github repo if anybody is interested
https://github.com/simon-cmyk/test_stack_overflow_tuple
Our goal is to make something like the following SWIG interface work intuitively:
%module test
%include "std_tuple.i"
%std_tuple(TupleDD, double, double);
%inline %{
std::tuple<double, double> func() {
return std::make_tuple(0.0, 1.0);
}
%}
We want to use this within Python in the following way:
import test
r=test.func()
print(r)
print(dir(r))
r[1]=1234
for x in r:
print(x)
i.e. indexing and iteration should just work.
By re-using some of the pre-processor tricks I used to wrap std::function (which were themselves originally from another answer here on SO) we can define a neat macro that "just wraps" std::tuple for us. Although this answer is Python specific it should in practice be fairly simple to adapt for most other languages too. I'll post my std_tuple.i file, first and then annotate/explain it after:
// [1]
%{
#include <tuple>
#include <utility>
%}
// [2]
#define make_getter(pos, type) const type& get##pos() const { return std::get<pos>(*$self); }
#define make_setter(pos, type) void set##pos(const type& val) { std::get<pos>(*$self) = val; }
#define make_ctorargN(pos, type) , type v##pos
#define make_ctorarg(first, ...) const first& v0 FOR_EACH(make_ctorargN, __VA_ARGS__)
// [3]
#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1) action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1) action(1,a2) action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1) action(1,a2) action(2,a3) action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1) action(1,a2) action(2,a3) action(3,a4) action(4,a5)
#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef
// [4]
%define %std_tuple(Name, ...)
%rename(Name) std::tuple<__VA_ARGS__>;
namespace std {
struct tuple<__VA_ARGS__> {
// [5]
tuple(make_ctorarg(__VA_ARGS__));
%extend {
// [6]
FOR_EACH(make_getter, __VA_ARGS__)
FOR_EACH(make_setter, __VA_ARGS__)
size_t __len__() const { return std::tuple_size<std::decay_t<decltype(*$self)>>{}; }
%pythoncode %{
# [7]
def __getitem__(self, n):
if n >= len(self): raise IndexError()
return getattr(self, 'get%d' % n)()
def __setitem__(self, n, val):
if n >= len(self): raise IndexError()
getattr(self, 'set%d' % n)(val)
%}
}
};
}
%enddef
This is just the extra includes we need for our macro to work
These apply to each of the type arguments we supply to our %std_tuple macro invocation, we need to be careful with commas here to keep the syntax correct.
This is the mechanics of our FOR_EACH macro, which invokes each action per argument in our variadic macro argument list
Finally the definition of %std_tuple can begin. Essentially this is manually doing the work of %template for each specialisation of std::tuple we care to name inside of the std namespace.
We use our macro for each magic to declare a constructor with arguments for each element of the correct type. The actual implementation here is the default one from the C++ library which is exactly what we need/want though.
We use our FOR_EACH macro twice to make a member function get0, get1, getN of the correct type of each tuple element and the correct number of them for the template argument size. Likewise for setN. Doing it this way allows the usual SWIG typemaps for double, etc. or whatever types your tuple contains to be applied automatically and correctly for each call to std::get<N>. These are really just an implementation detail, not intended to be part of the public interface, but exposing them makes no real odds.
Finally we need an implementation of __getitem__ and a corresponding __setitem__. These simply look up and call the right getN/setN function on the class and call that instead. We take care to raise IndexError instead of the default exception if an invalid index is used as this will stop iteration correctly when we try to iterate of the tuple.
This is then sufficient that we can run our target code and get the following output:
$ swig3.0 -python -c++ -Wall test.i && g++ -shared -o _test.so test_wrap.cxx -I/usr/include/python3.7 -m32 && python3.7 run.py
<test.TupleDD; proxy of <Swig Object of type 'std::tuple< double,double > *' at 0xf766a260> >
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__swig_destroy__', '__swig_getmethods__', '__swig_setmethods__', '__weakref__', 'get0', 'get1', 'set0', 'set1', 'this']
0.0
1234.0
Generally this should work as you'd hope in most input/output situations in Python.
There are a few improvements we could look to make:
Implement repr
Implement slicing so that tuple[n:m] type indexing works
Handle unpacking like Python tuples.
Maybe do some more automatic conversions for compatible types?
Avoid calling __len__ for every get/setitem call, either by caching the value in the class itself, or postponing it until the method lookup fails?
The default typemap provided by swig expects a python list as input for constructing a set instead of native Python set (see here). Indeed,
%include "std_set.i"
%template(IntSet) std::set<int>;
will trigger the following behavior in python:
>>> import MyClass
>>> MyClass.IntSet([1,2,3,4]) # works
>>> MyClass.IntSet({1,2,3,4}) # does not work
I tried to create a custom typemap to do so but I failed so far. Here my current swig file to do so:
%module MyClass
%{
#include "MyClass.h"
%}
%include <typemaps.i>
%typemap(in) setin {$1 = PySequence_List($input);}
%typemap(out) setout {$result = PySet_New($1);}
%include "std_set.i"
%template(IntSet) std::set<int>;
%include "MyClass.h"
Would you know how to make it ?
I have a C-header file (let's call it myheader.h) that contains some character string definitions such as:
#define MYSTRING "mystring-constant"
In Cython, I create a cmy.pxd file that contains:
cdef extern from "myheader.h":
cdef const char* MYSTRING "MYSTRING"
and a corresponding my.pyx file that contains some class definitions, all headed by:
from cmy cimport *
I then try to reference that string in a Python script:
from my import *
def main():
print("CONSTANT ", MYSTRING)
if __name__ == '__main__':
main()
Problem is that I keep getting an error:
NameError: name 'MYSTRING' is not defined
I've searched the documentation and can't identify the problem. Any suggestions would be welcomed - I confess it is likely something truly silly.
You cannot access cdef-variables from Python. So you have to create a Python object which would correspond to your define, something like this (it uses Cython>=0.28-feature verbatim-C-code, so you need a recent Cython version to run the snippet):
%%cython
cdef extern from *:
"""
#define MYSTRING "mystring-constant"
"""
# avoid name clash with Python-variable
# in cdef-code the value can be accessed as MYSTRING_DEFINE
cdef const char* MYSTRING_DEFINE "MYSTRING"
#python variable, can be accessed from Python
#the data is copied from MYSTRING_DEFINE
MYSTRING = MYSTRING_DEFINE
and now MYSTRING is a bytes-object:
>>> print(MYSTRING)
b'mystring-constant'
This is my first encounter with SWIG, and I'm not finding anything useful by searching, which probably means I'm not even looking in the right direction. If someone could get me vectored, that would be great.
I have a C function that takes a double* as an argument and uses it as an output:
int myFunc(double* p, int len){
int i;
for(i=0; i<len; i++){
p[i] = (double)i;
}
return 0;
}
To wrap it, I didn't do anything tricky or custom, I just let SWIG do whatever it does by default:
%module MyModule
%{
/* Includes the header in the wrapper code */
#include "MyModule.h"
%}
/* Parse the header file to generate wrappers */
%include "MyModule.h"
This all worked in that it compiled and built a module I can load. However, one might reasonably expect to be able to call this from Python like this, and I can't:
numbers = [0.0]*5
myFunc(numbers, 5)
SWIG complains that the first argument is the wrong type. I've been reading about a thing called typemaps.i that lets you specify type conversions manually. Is this necessary for something as standard as this example? I'd be grateful if someone could tell me what I should read or Google for.
Using builtin types as input parameters with SWIG is handled automatically, but SWIG needs some help with output parameters. Below is a simple way to do it.
Note that %inline is a way to add a function and a SWIG wrapper for it in a .i file instead of making a header file and using #include and %include. It's good for small examples.
Reference: carrays.i
%module x
%include <carrays.i>
%array_functions(double,doubles)
%inline %{
int myFunc(double* p, int len)
{
int i;
for(i = 0; i < len; i++)
p[i] = (double)i;
return 0;
}
%}
Output:
>>> import x
>>> pd=x.new_doubles(10)
>>> pd
<Swig Object of type 'double *' at 0x0000000002235CC0>
>>> x.myFunc(pd,10)
0
>>> x.doubles_getitem(pd,0)
0.0
>>> x.doubles_getitem(pd,1)
1.0
>>> x.doubles_getitem(pd,9)
9.0
>>> x.delete_doubles(pd)
With a little more work, we can get a more Pythonic interface using typemaps. Below a set of in/argout/argfree typemaps generate type C parameters from a single Python input object.
%typemap(in) defines how to convert the Python input to the input parameters. It checks for an integer object and generates the 2nd parameter (len), then uses malloc to generate the 1st parameter.
%typemap(argout) defines how to convert the output parameters to a Python object, and appends it to the existing return value represented by $result.
%typemap(free) frees the allocated parameter from the in typemap.
References: Common typemap methods, Multi-argument typemaps
%module x
%include <exception.i>
%typemap(in) (double* p, int len) %{
if(!PyLong_Check($input))
SWIG_exception(SWIG_TypeError, "expected integer");
$2 = PyLong_AsUnsignedLong($input);
$1 = malloc($2 * sizeof(double));
%}
%typemap(freearg) (double* p, int len) %{
free($1);
%}
%typemap(argout) (double* p, int len) {
PyObject* list = PyList_New($2);
int i;
for(i = 0; i < $2; ++i)
PyList_SET_ITEM(list, i, PyFloat_FromDouble($1[i]));
$result = SWIG_Python_AppendOutput($result, list);
}
%inline %{
int myFunc(double * p, int len)
{
int i;
for(i = 0; i < len; i++)
p[i] = (double)i;
return 0;
}
%}
Output:
>>> import x
>>> x.myFunc(5)
[0, [0.0, 1.0, 2.0, 3.0, 4.0]]
>>> return_value, double_list = x.myFunc(3)
>>> return_value
0
>>> double_list
[0.0, 1.0, 2.0]
If I have these two C++ files:
foo.cpp:
#include "foo.h"
void foo(Foo* p) {};
foo.h:
class Foo {};
void foo(Foo*);
I can write this SWIG interface
%{
#include "foo.h"
%}
%include <cpointer.i>
%pointer_class(Foo, Foop)
%include "foo.h"
compile, and call
>>>p = Foop()
>>>foo(p)
How do I write the interface if Foo is typedef instead
typedef int Foo;
so that I can do the same Python calls as above?
I don't think %pointer_class is meant to be used with a typedef. SWIG considers a typedef the same as the original type and doesn't generate a wrapper for the typedef-ed name. For example, SWIG only generates Bar for Python:
%module x
%inline %{
class Bar {};
typedef Bar Foo;
void foo(Foo* p) {};
%}
Output:
>>> import x
>>> x.Foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Foo'
>>> p=x.Bar()
>>> x.foo(p)
>>>
So I believe your case with typedef int Foo; and %pointer_class(Foo, Foop) doesn't work because the Foo wrapper doesn't exist. I found that the below "trick" works for a class and an int if you use a typedef in both cases and use the same value for both parameters of %pointer_class:
%module x
%inline %{
class Bar {};
typedef Bar Foo;
void foo(Foo* p) {};
%}
%include <cpointer.i>
%pointer_class(Foo, Foo) // doesn't work if 2nd param doesn't match 1st.
And:
%module x
%inline %{
typedef int Foo;
void foo(Foo* p) {};
%}
%include <cpointer.i>
%pointer_class(Foo, Foo) // doesn't work if 2nd param doesn't match 1st.
But if you really want an opaque type, don't let SWIG see the definition at all by only forward declaring the class and hiding the definition:
%module x
%inline %{
class Foo;
void foo(Foo* p);
%}
%include <cpointer.i>
%pointer_class(Foo, Foop)