We use SWIG to generate Python wrappers for some C++ code. SWIG directives start with %, and look like this in headers:
#ifdef SWIG
%apply int* INT_RET_VAL{int * is_donor};
#ENDIF
ErrLevel mmct_hbond_is_donor(MM_Index ct, int atom_idx, int* is_donor);
#ifdef SWIG
%clear int * is_donor;
#ENDIF
When clang-format processes them, the %apply turns to % apply, which is not a SWIG statement anymore! There are lots of these in our code base currently. Is there a good way to convince clang-format to skip the %apply and %clear statements? Or maybe everything in an #ifdef SWIG?
You can use // clang-format off.
int formatted_code;
// clang-format off
void unformatted_code ;
// clang-format on
void formatted_code_again;
https://clang.llvm.org/docs/ClangFormatStyleOptions.html#disabling-formatting-on-a-piece-of-code
Related
I see here that the swig interface file has identical declarations at 2 places (in and after %{ }% part).
/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
%}
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
Why is this redundancy needed?
The code between %{ and %} is directly injected in the generated SWIG wrapper code to make the declarations available to the compiler. For example you could have a utility function in this block that's only visible to the compiler but not exposed to the target language, or add #include statements needed to compile the wrapper.
Code outside that block is parsed by SWIG and exposed to the target language. If you left out one of the declarations in this section it wouldn't be exposed to the target language.
You can eliminate the redundancy in this case by using %inline, which both injects the code into the wrapper and parses it for exposure to the target language:
%module example
%inline %{
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
%}
I've been reading in the CUDA Programming Guide about template functions and is something like this working?
#include <cstdio>
/* host struct */
template <typename T>
struct Test {
T *val;
int size;
};
/* struct device */
template <typename T>
__device__ Test<T> *d_test;
/* test function */
template <typename T>
T __device__ testfunc() {
return *d_test<T>->val;
}
/* test kernel */
__global__ void kernel() {
printf("funcout = %g \n", testfunc<float>());
}
I get the correct result but a warning:
"warning: a host variable "d_test [with T=T]" cannot be directly read in a device function" ?
Has the struct in the testfunction to be instantiated with *d_test<float>->val ?
KR,
Iggi
Unfortunately, the CUDA compiler seems to generally have some issues with variable templates. If you look at the assembly, you'll see that everything works just fine. The compiler clearly does instantiate the variable template and allocates a corresponding device object.
.global .align 8 .u64 _Z6d_testIfE;
The generated code uses this object just like it's supposed to
ld.global.u64 %rd3, [_Z6d_testIfE];
I'd consider this warning a compiler bug. Note that I cannot reproduce the issue with CUDA 10 here, so this issue has most likely been fixed by now. Consider updating your compiler…
#MichaelKenzel is correct.
This is almost certainly an nvcc bug - which I have now filed (you might need an account to access that.
Also note I've been able to reproduce the issue with less code:
template <typename T>
struct foo { int val; };
template <typename T>
__device__ foo<T> *x;
template <typename T>
int __device__ f() { return x<T>->val; }
__global__ void kernel() { int y = f<float>(); }
and have a look at the result on GodBolt as well.
I've been using the REFLECTABLE macro from this answer in my C++ header file, which looks like:
#ifndef TIMER_H
#define TIMER_H
// From the linked question, but (deliberately) ignored by SWIG here,
// only relevant is as much as it defines REFLECTABLE for other preprocessors
#include "reflection.hh"
struct Timer {
REFLECTABLE (
(float) startTime,
(float) endTime,
)
};
#endif
The SWIG preprocessor doesn't follow #include and doesn't handle the definition of REFLECTABLE from the linked question anyway, so my plan is to ignore that entirely and replace it with a more appropriate SWIG specific definition of REFLECTABLE.
My goal is for the SWIG preprocessor to expand Timer as though there was nothing special going on, i.e.:
struct Timer {
float startTime;
float endTime;
};
To that end I have written a simple reflection.i that makes a recursive call to a %define SWIG macro:
#define REM(...) __VA_ARGS__
#define PAIR(x) REM x
%define EXPAND(tail, ...)
PAIR(tail);
EXPAND(__VA_ARGS__)
%enddef
#define REFLECTABLE(...) EXPAND(__VA_ARGS__)
That I can use in test.i:
%module test
%include "reflection.i"
%include "timer.h"
When we run swig -python -Wall -E test.i to inspect the result of running this through the SWIG preprocessor it almost works, but the definition at the end of recursion doesn't quite match what I'd like:
struct Timer {
/*#SWIG:reflection.i,4,EXPAND#*/
float startTime;
/*#SWIG:reflection.i,4,EXPAND#*/
float endTime;
EXPAND
};
The problem is that recursion gets terminated by the bareword EXPAND when ___VA_ARGS__ is empty. I looked at using map.h from this answer, but that also doesn't work with the SWIG preprocessor.
How can I modify my definition of REFLECTABLE so that the SWIG preprocessor just generates the flat structure I'm looking for?
I came up with a solution in the end that required two changes:
There's more indirection to the recursion, we paste two tokens together to get the token EXPAND_STOP or EXPAND_MORE, depending on whether or not the __VA_ARGS__ is empty during this evaluation. This selection takes place because SWIG evaluates
#define STOP(...) MORE
as MORE when there are arguments, but simply as STOP when there are no arguments.
The TOKEN macro then adds the required indirection for the token paste operator to work.
There's a fake first argument to both EXPAND_MORE and EXPAND_STOP that prevents the same behaviour from stopping us eating the call to EXPAND_STOP with no arguments.
#define REM(...) __VA_ARGS__
#define PAIR(x) REM x
#define TOKEN_(x,y) x##y
#define TOKEN(x,y) TOKEN_(x, y)
#define STOP(...) MORE
%define EXPAND_MORE(fake, ...)
EXPAND(__VA_ARGS__)
%enddef
#define EXPAND_STOP(fake, ...)
%define EXPAND(tail, ...)
PAIR(tail);
TOKEN(EXPAND_, STOP(__VA_ARGS__)) ## (fake, ##__VA_ARGS__)
%enddef
#define REFLECTABLE(...) EXPAND(__VA_ARGS__)
Although possibly not the most elegant solution this is tested and working with SWIG 3.0.2.
I have two C functions that I'm exposing through SWIG to my Java layer and both have an input param with a const void * data type ("val) that needs to be a uint8_t for the addCategory function but a char for the addAttribute function. I'm currently, in the SWIG Interface file, using the %apply to map the const void * C type to a short on the Java side. Is there a way to modify the SWIG interface file to support both a char (String) and a uint8_t (short) for the const void * input parameter?
C Functions from header file:
int
addCategory(query_t *query, type_t type, const void *val);
int
addAttribute(query_t *query, type_t type, const void *val);
SWIG Interface File:
%module Example
%include "stdint.i"
void setPhy_idx(uint32_t value);
%include "arrays_java.i"
void setId(unsigned char *value);
%{
#include "Example.h"
%}
%apply char * { unsigned char * };
%apply char * { void * };
%apply uint8_t { const void * }
%apply int32_t { int32_t * }
%include "Example.h"
You can't directly do this - what type would be used in this place in Java? You need to help SWIG decide that in some way.
You have (at least) three possible solutions:
Use a type hierarchy - The base type will be what the function takes, the subclasses will get wrapped also. You could do this on the C++ side, or on the Java side using SWIG's typemap facilities. I think this is needlessly complicated though, so I've not made an example here.
Use overloads (or even different functions, with different names altogether - you could use %rename to make them back into overloads in Java even if they have different names in C)
Use a union. This will get wrapped with set and get functions by SWIG:
%module test
union values {
unsigned char *string;
void *generic;
uint8_t someOtherThing;
uint32_t number;
};
void func(values v);
This results in a Java class called values, which func() takes and can pass one of the members of the union through. Clearly you'd want to %apply appropriate typemaps for the members of the union.
I am having trouble importing my C++ functions. If I declare them as C functions I can successfully import them. When explicit loading, if any of the functions are missing the extern as C decoration I get a the following exception:
First-chance exception at 0x00000000 in cpp.exe: 0xC0000005: Access violation.
DLL.h:
extern "C" __declspec(dllimport) int addC(int a, int b);
__declspec(dllimport) int addCpp(int a, int b);
DLL.cpp:
#include "DLL.h"
int addC(int a, int b) {
return a + b;
}
int addCpp(int a, int b) {
return a + b;
}
main.cpp:
#include "..DLL/DLL.h"
#include <stdio.h>
#include <windows.h>
int main() {
int a = 2;
int b = 1;
typedef int (*PFNaddC)(int,int);
typedef int (*PFNaddCpp)(int,int);
HMODULE hDLL = LoadLibrary(TEXT("../Debug/DLL.dll"));
if (hDLL != NULL)
{
PFNaddC pfnAddC = (PFNaddC)GetProcAddress(hDLL, "addC");
PFNaddCpp pfnAddCpp = (PFNaddCpp)GetProcAddress(hDLL, "addCpp");
printf("a=%d, b=%d\n", a,b);
printf("pfnAddC: %d\n", pfnAddC(a,b));
printf("pfnAddCpp: %d\n", pfnAddCpp(a,b)); //EXCEPTION ON THIS LINE
}
getchar();
return 0;
}
How can I import c++ functions for dynamic loading? I have found that the following code works with implicit loading by referencing the *.lib, but I would like to learn about dynamic loading.
Thank you to all in advance.
Update:
bindump /exports
1 00011109 ?addCpp##YAHHH#Z = #ILT+260(?addCpp##YAHHH#Z)
2 00011136 addC = #ILT+305(_addC)
Solution:
Create a conversion struct as
found here
Take a look at the
file exports and copy explicitly the
c++ mangle naming convention.
PFNaddCpp pfnAddCpp = (PFNaddCpp)GetProcAddress(hDLL, "?addCpp##YAHHH#Z");
Inevitably, the access violation on the null pointer is because GetProcAddress() returns null on error.
The problem is that C++ names are mangled by the compiler to accommodate a variety of C++ features (namespaces, classes, and overloading, among other things). So, your function addCpp() is not really named addCpp() in the resulting library. When you declare the function with extern "C", you give up overloading and the option of putting the function in a namespace, but in return you get a function whose name is not mangled, and which you can call from C code (which doesn't know anything about name mangling.)
One option to get around this is to export the functions using a .def file to rename the exported functions. There's an article, Explicitly Linking to Classes in DLLs, that describes what is necessary to do this.
It's possible to just wrap a whole header file in extern "C" as follows. Then you don't need to worry about forgetting an extern "C" on one of your declarations.
#ifdef __cplusplus
extern "C" {
#endif
__declspec(dllimport) int addC(int a, int b);
__declspec(dllimport) int addCpp(int a, int b);
#ifdef __cplusplus
} /* extern "C" */
#endif
You can still use all of the C++ features that you're used to in the function bodies -- these functions are still C++ functions -- they just have restrictions on the prototypes to make them compatible with C code.