When calling a swig generated function returning std::tuple, i get a swig object of that std::tuple.
Is there a way to use type-maps or something else to extract the values? I have tried changing the code to std::vector for a small portion of the code, and that works. (using %include <std_vector.i> and templates) But i don't want to make too many changes in the C++ part.
Edit: here is a minimal reproducible example:
foo.h
#pragma once
#include <tuple>
class foo
{
private:
double secret1;
double secret2;
public:
foo();
~foo();
std::tuple<double, double> return_thing(void);
};
foo.cpp
#include "foo.h"
#include <tuple>
foo::foo()
{
secret1 = 1;
secret2 = 2;
}
foo::~foo()
{
}
std::tuple<double, double> foo::return_thing(void) {
return {secret1, secret2};
}
foo.i
%module foo
%{
#include"foo.h"
%}
%include "foo.h"
When compiled on my linux using
-:$ swig -python -c++ -o foo_wrap.cpp foo.i
-:$ g++ -c foo.cpp foo_wrap.cpp '-I/usr/include/python3.8' '-fPIC' '-std=c++17' '-I/home/simon/Desktop/test_stack_overflow_tuple'
-:$ g++ -shared foo.o foo_wrap.o -o _foo.so
I can import it in python as shown:
test_module.ipynb
import foo as f
Foo = f.foo()
return_object = Foo.return_thing()
type(return_object)
print(return_object)
Outputs is
SwigPyObject
<Swig Object of type 'std::tuple< double,double > *' at 0x7fb5845d8420>
Hopefully this is more helpful, thank you for responding
To clarify i want to be able to use the values in python something like this:
main.cpp
#include "foo.h"
#include <iostream>
//------------------------------------------------------------------------------'
using namespace std;
int main()
{
foo Foo = foo();
auto [s1, s2] = Foo.return_thing();
cout << s1 << " " << s2 << endl;
}
//------------------------------------------------------------------------------
Github repo if anybody is interested
https://github.com/simon-cmyk/test_stack_overflow_tuple
Our goal is to make something like the following SWIG interface work intuitively:
%module test
%include "std_tuple.i"
%std_tuple(TupleDD, double, double);
%inline %{
std::tuple<double, double> func() {
return std::make_tuple(0.0, 1.0);
}
%}
We want to use this within Python in the following way:
import test
r=test.func()
print(r)
print(dir(r))
r[1]=1234
for x in r:
print(x)
i.e. indexing and iteration should just work.
By re-using some of the pre-processor tricks I used to wrap std::function (which were themselves originally from another answer here on SO) we can define a neat macro that "just wraps" std::tuple for us. Although this answer is Python specific it should in practice be fairly simple to adapt for most other languages too. I'll post my std_tuple.i file, first and then annotate/explain it after:
// [1]
%{
#include <tuple>
#include <utility>
%}
// [2]
#define make_getter(pos, type) const type& get##pos() const { return std::get<pos>(*$self); }
#define make_setter(pos, type) void set##pos(const type& val) { std::get<pos>(*$self) = val; }
#define make_ctorargN(pos, type) , type v##pos
#define make_ctorarg(first, ...) const first& v0 FOR_EACH(make_ctorargN, __VA_ARGS__)
// [3]
#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1) action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1) action(1,a2) action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1) action(1,a2) action(2,a3) action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1) action(1,a2) action(2,a3) action(3,a4) action(4,a5)
#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef
// [4]
%define %std_tuple(Name, ...)
%rename(Name) std::tuple<__VA_ARGS__>;
namespace std {
struct tuple<__VA_ARGS__> {
// [5]
tuple(make_ctorarg(__VA_ARGS__));
%extend {
// [6]
FOR_EACH(make_getter, __VA_ARGS__)
FOR_EACH(make_setter, __VA_ARGS__)
size_t __len__() const { return std::tuple_size<std::decay_t<decltype(*$self)>>{}; }
%pythoncode %{
# [7]
def __getitem__(self, n):
if n >= len(self): raise IndexError()
return getattr(self, 'get%d' % n)()
def __setitem__(self, n, val):
if n >= len(self): raise IndexError()
getattr(self, 'set%d' % n)(val)
%}
}
};
}
%enddef
This is just the extra includes we need for our macro to work
These apply to each of the type arguments we supply to our %std_tuple macro invocation, we need to be careful with commas here to keep the syntax correct.
This is the mechanics of our FOR_EACH macro, which invokes each action per argument in our variadic macro argument list
Finally the definition of %std_tuple can begin. Essentially this is manually doing the work of %template for each specialisation of std::tuple we care to name inside of the std namespace.
We use our macro for each magic to declare a constructor with arguments for each element of the correct type. The actual implementation here is the default one from the C++ library which is exactly what we need/want though.
We use our FOR_EACH macro twice to make a member function get0, get1, getN of the correct type of each tuple element and the correct number of them for the template argument size. Likewise for setN. Doing it this way allows the usual SWIG typemaps for double, etc. or whatever types your tuple contains to be applied automatically and correctly for each call to std::get<N>. These are really just an implementation detail, not intended to be part of the public interface, but exposing them makes no real odds.
Finally we need an implementation of __getitem__ and a corresponding __setitem__. These simply look up and call the right getN/setN function on the class and call that instead. We take care to raise IndexError instead of the default exception if an invalid index is used as this will stop iteration correctly when we try to iterate of the tuple.
This is then sufficient that we can run our target code and get the following output:
$ swig3.0 -python -c++ -Wall test.i && g++ -shared -o _test.so test_wrap.cxx -I/usr/include/python3.7 -m32 && python3.7 run.py
<test.TupleDD; proxy of <Swig Object of type 'std::tuple< double,double > *' at 0xf766a260> >
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__swig_destroy__', '__swig_getmethods__', '__swig_setmethods__', '__weakref__', 'get0', 'get1', 'set0', 'set1', 'this']
0.0
1234.0
Generally this should work as you'd hope in most input/output situations in Python.
There are a few improvements we could look to make:
Implement repr
Implement slicing so that tuple[n:m] type indexing works
Handle unpacking like Python tuples.
Maybe do some more automatic conversions for compatible types?
Avoid calling __len__ for every get/setitem call, either by caching the value in the class itself, or postponing it until the method lookup fails?
How can I declare a Julia function that returns a function with a specific signature. For example, say I want to return a function that takes an Int and returns an Int:
function buildfunc()::?????
mult(x::Int) = x * 2
return mult
end
What should the question marks be replaced with?
One thing needs to be made clear.
Adding a type declaration on the returned parameter is just an assertion, not part of function definition. To understand what is going on look at the lowered (this is a pre-compilation stage) code of a function:
julia> f(a::Int)::Int = 2a
f (generic function with 1 method)
julia> #code_lowered f(5)
CodeInfo(
1 ─ %1 = Main.Int
│ %2 = 2 * a
│ %3 = Base.convert(%1, %2)
│ %4 = Core.typeassert(%3, %1)
└── return %4
)
In this case since the returned type is obvious this assertion will be actually removed during the compilation process (try #code_native f(5) to see yourself).
If you want for some reason to generate functions I recommend to use the #generated macro. Be warned: meta-programming is usually an overkill for solving any Julia related problem.
#generated function f2(x)
if x <: Int
quote
2x
end
else
quote
10x
end
end
end
Now we have a function f2 where the source code of f2 is going to depend on the parameter type:
julia> f2(3)
6
julia> f2(3.)
30.0
Note that this function generation is actually happening during the compile time:
julia> #code_lowered f2(2)
CodeInfo(
# REPL[34]:1 within `f2'
┌ # REPL[34]:4 within `macro expansion'
1 ─│ %1 = 2 * x
└──│ return %1
└
)
Hope that clears things out.
You can use Function type for this purpose. From Julia documentation:
Function is the abstract type of all functions
function b(c::Int64)::Int64
return c+2;
end
function a()::Function
return b;
end
Which prints:
julia> println(a()(2));
4
Julia will throw exception for Float64 input.
julia> println(a()(2.0));
ERROR: MethodError: no method matching b(::Float64) Closest candidates are: b(::Int64)
I'm chasing my tail with what I suspect is a simple problem, but I can't seem to find any explanation for the observed behavior. Assume I have a constant in a C header file defined by:
#define FOOBAR 128
typedef uint32_t mytype_t;
I convert this in Cython by putting the following in the .pxd file:
cdef int _FOOBAR "FOOBAR"
ctypedef uint32_t mytype_t
In my .pyx file, I have a declaration:
FOOBAR = _FOOBAR
followed later in a class definition:
cdef class MyClass:
cdef mytype_t myvar
def __init__(self):
try:
self.myvar = FOOBAR
print("GOOD")
except:
print("BAD")
I then try to execute this with a simple program:
try:
foo = MyClass()
except:
print("FAILED TO CREATE CLASS")
Sadly, this errors out, but I don't get an error message - I just get the exception print output:
BAD
Any suggestions on root cause would be greatly appreciated.
I believe I have finally tracked it down. The root cause issue is that FOOBAR in my code was actually set to UINT32MAX. Apparently, Cython/Python interprets that as a -1 and Python then rejects setting a uint32_t variable equal to it. The solution is to define FOOBAR to be 0xffffffff - apparently Python thinks that is a non-negative value and accepts it.
In Haskell (GHC), how can one obtain the type signature of the list of functions shown below?
[tail,init,reverse]
I unsuccessfully tried using the typeOf function of the Data.Typeable module. Specifically, I try to run the following Haskell script:
import Data.Typeable
import Test.HUnit
myTest = TestCase
( assertEqual "\n\nShould have been \"[[a] -> [a]]\""
"[[a] -> [a]]"
(show ( typeOf [tail,init,reverse] )) )
tests = TestList [ (TestLabel "myTest" myTest) ]
However, GHC responds with the following error:
C:\>ghci my_script.hs
GHCi, version 8.0.2: http://www.haskell.org/ghc/ :? for help
[1 of 1] Compiling Main ( my_script.hs, interpreted )
my_script.hs:7:21: error:
* No instance for (Typeable a0) arising from a use of `typeOf'
* In the first argument of `show', namely
`(typeOf [tail, init, reverse])'
In the third argument of `assertEqual', namely
`(show (typeOf [tail, init, reverse]))'
In the first argument of `TestCase', namely
`(assertEqual
"\n\
\\n\
\Should have been \"[[a] -> [a]]\""
"[[a] -> [a]]"
(show (typeOf [tail, init, reverse])))'
Failed, modules loaded: none.
Prelude>
Update: The following HUnit test case isn't quite what I wanted, but I did get it to pass (based on David Young's suggestion). This test case at least forces the compiler to confirm that [tail,init,reverse] is of type [ [a] -> [a] ].
import Data.Typeable
import Test.HUnit
myTest = TestCase
( assertEqual "\n\nShould have been 3"
3
( length ( [tail,init,reverse] :: [[a]->[a]] ) ) )
tests = TestList [ (TestLabel "myTest" myTest) ]
C:\>my_script.hs
GHCi, version 8.0.2: http://www.haskell.org/ghc/ :? for help
[1 of 1] Compiling Main ( my_script.hs, interpreted )
Ok, modules loaded: Main.
*Main> runTestTT tests
Cases: 1 Tried: 1 Errors: 0 Failures: 0
You don't need a unit test to check a function's type. A unit tests runs after the code has been compiled, it's a dynamic test. However, type checking is a static test: all types are tested during the compilation of your program. Therefore, we can use GHC as a minimal static type checker and reduce your program to:
main :: IO ()
main = return ()
where
tailInitReverseAreListFunctions :: [[a] -> [a]]
tailInitReverseAreListFunctions = [tail, init, reverse]
You don't even need that test anymore the moment you actually test your functions with real data, because that application will (statically) test the function's type too.
Remember, Haskell is a statically typed language. The types are checked during compilation, before your code is run. Any type checking unit-test is therefore more or less a code-smell, because it can only pass.
I have a function in a module that looks something like this:
module MyLibrary (throwIfNegative) where
throwIfNegative :: Integral i => i -> String
throwIfNegative n | n < 0 = error "negative"
| otherwise = "no worries"
I could of course return Maybe String or some other variant, but I think it's fair to say that it's a programmer error to call this function with a negative number so using error is justified here.
Now, since I like having my test coverage at 100% I want to have a test case that checks this behavior. I have tried this
import Control.Exception
import Test.HUnit
import MyLibrary
case_negative =
handleJust errorCalls (const $ return ()) $ do
evaluate $ throwIfNegative (-1)
assertFailure "must throw when given a negative number"
where errorCalls (ErrorCall _) = Just ()
main = runTestTT $ TestCase case_negative
and it sort of works, but it fails when compiling with optimizations:
$ ghc --make -O Test.hs
$ ./Test
### Failure:
must throw when given a negative number
Cases: 1 Tried: 1 Errors: 0 Failures: 1
I'm not sure what's happening here. It seems like despite my use of evaluate, the function does not get evaluated. Also, it works again if I do any of these steps:
Remove HUnit and call the code directly
Move throwIfNegative to the same module as the test case
Remove the type signature of throwIfNegative
I assume this is because it causes the optimizations to be applied differently. Any pointers?
Optimizations, strictness, and imprecise exceptions can be a bit tricky.
The easiest way to reproduce this problem above is with a NOINLINE on throwIfNegative (the function isn't being inlined across module boundaries either):
import Control.Exception
import Test.HUnit
throwIfNegative :: Int -> String
throwIfNegative n | n < 0 = error "negative"
| otherwise = "no worries"
{-# NOINLINE throwIfNegative #-}
case_negative =
handleJust errorCalls (const $ return ()) $ do
evaluate $ throwIfNegative (-1)
assertFailure "must throw when given a negative number"
where errorCalls (ErrorCall _) = Just ()
main = runTestTT $ TestCase case_negative
Reading the core, with optimizations on, the GHC inlines evaluate properly (?):
catch#
# ()
# SomeException
(\ _ ->
case throwIfNegative (I# (-1)) of _ -> ...
and then floats out the call to throwIfError, outside of the case scrutinizer:
lvl_sJb :: String
lvl_sJb = throwIfNegative lvl_sJc
lvl_sJc = I# (-1)
throwIfNegative =
\ (n_adO :: Int) ->
case n_adO of _ { I# x_aBb ->
case <# x_aBb 0 of _ {
False -> lvl_sCw; True -> error lvl_sCy
and strangely, at this point, no other code now calls lvl_sJb, so the entire test becomes dead code, and is stripped out -- GHC has determined that it is unused!
Using seq instead of evaluate is happy enough:
case_negative =
handleJust errorCalls (const $ return ()) $ do
throwIfNegative (-1) `seq` assertFailure "must throw when given a negative number"
where errorCalls (ErrorCall _) = Just ()
or a bang pattern:
case_negative =
handleJust errorCalls (const $ return ()) $ do
let !x = throwIfNegative (-1)
assertFailure "must throw when given a negative number"
where errorCalls (ErrorCall _) = Just ()
so I think we should look at the semantics of evaluate:
-- | Forces its argument to be evaluated to weak head normal form when
-- the resultant 'IO' action is executed. It can be used to order
-- evaluation with respect to other 'IO' operations; its semantics are
-- given by
--
-- > evaluate x `seq` y ==> y
-- > evaluate x `catch` f ==> (return $! x) `catch` f
-- > evaluate x >>= f ==> (return $! x) >>= f
--
-- /Note:/ the first equation implies that #(evaluate x)# is /not/ the
-- same as #(return $! x)#. A correct definition is
--
-- > evaluate x = (return $! x) >>= return
--
evaluate :: a -> IO a
evaluate a = IO $ \s -> let !va = a in (# s, va #) -- NB. see #2273
That #2273 bug is a pretty interesting read.
I think GHC is doing something suspicious here, and recommend not using evalaute (instead, use seq directly). This needs more thinking about what GHC is doing with the strictness.
I've filed a bug report to help get a determination from GHC HQ.