Linking scope of loaded symbols when using swig to wrap C++ for python - swig

I have a C++ library that I am wrapping with SWIG to make accessible in python. It is my understanding (from experience) that when SWIG wraps a C++ library in python, upon loading it places the C++ library symbols in a "local" scope. That is - a scope which does not enable future dynamically linked libraries to find the symbols.
(I'm getting this definition of "local" from man dlopen(3) )
Is there any way to get SWIG to place these symbols into the "global" scope, such that any future dynamically linked libraries can find them?

You can make python dlopen shared objects with the RTLD_GLOBAL flag by calling setdlopenflags in sys, e.g.:
sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL)
before your module is loaded. (There's a discussion on swig-users about this)

Related

Can I use STL in a DriverKit driver?

Can I use for example std::vector in DriverKit driver in macOs / XCode?
DriverKit has some container class like OSArray
https://developer.apple.com/documentation/driverkit/osarray?language=objc
If I create a new "DriverKit driver" project and include <vector> then I get build errors. Those error comes from including <cstring> and the error is
No member named 'strcpy' in the global namespace
No member named 'strcat' in the global namespace
As far as I can tell, you aren't supposed to. The headers you end up pulling in there aren't from the DriverKit SDK, they're the default ones that come with Xcode's compiler toolchain. They assume a normal macOS build environment, not the dext environment. The DriverKit SDK doesn't include C++ stdlib headers, and only contains some stripped-down headers for a subset of the C standard library. Hence the missing strcpy and strcat.
Linking against libc++ also fails, even the version included in the DriverKit SDK. I don't know why there is a version of that library included with the DriverKit SDK, but it's clearly not intended for being used in dexts.
There's presumably nothing in particular stopping you from including some other container library, or even directly including parts of an STL implementation. You may need to manually wire up memory allocation calls though.

Tcl version change issue from 8.4 to 8.5.12

I have a problem with changing tcl version from 8.4 to 8.5.12 on RHEL machine. Our product uses TclDevKit components like Tcldom, Tclxml, etc. Also we are using Incr Tcl (Itcl). I am trying to create pkgIndex.tcl file in Itcl in order to find Itcl when that package is required as follown:
package ifneeded Itcl 3.4 [list load [file join $dir "libitcl-O.a"] Itcl ]
but when I use
package require Itcl
Getting report: couldn't load file "/somepath/itcl/lib/libitcl-O.a": /somepath/lib/libitcl-O.a: invalid ELF header
It seems I can't load files with .a extention, but the same is done with previous version of tcl (8.4) and it works fine. I googled a lot, read a lot of documentation, but it doesn't help to go further.
Please help.
Thanks in advance
Libraries come in two general sorts, static libraries and shared libraries. On Linux, static libraries have the extension .a by default, and shared libraries have the extension .so (plus optionally some numbers to indicate the version). Only shared libraries will work with Tcl's load command and even then they have to be designed to work that way (with an appropriate Foobar_Init function, as documented).
When dealing with stub-exporting extensions (fairly rare) or Tcl and Tk themselves, the linking is done in two parts. There's a stub library, normally called somethingstub.a, and there's a main shared library. The main shared library contains the implementation of the code; all that is in the stub library is an ABI/API adaptor so that you can avoid binding your code to an explicit version of the implementation library. By building an extension stub-enabled and linking against the stub library, you gain the ability to have your extension loaded into future versions of Tcl/Tk without any recompilation or relinking steps at all. (You also become able to put the extension inside a starkit for deployment, as those use a rather unusual way of managing shared libraries that the stub mechanism conceals from you.)

Launch interactive OCaml session with library (Yojson) available

I've installed the Yojson library for OCaml via GODI:
http://martin.jambon.free.fr/yojson.html
I want to start an interactive ocaml session (i.e. via the ocaml command) and execute functions from the Yojson library e.g.
Yojson.Safe.from_string;;
How do I do this? The above command gives "Error: Unbound module Yojson". I've worked out how to compile via ocamlc with Yojson available, but I want to launch an interactive session instead.
I know this seems like a horrible beginners question but Yojson comes with no samples and minimal instructions so I'm really stumped. I've tried various combinations of "#load" and compiler switches and I'm stuck.
The tool you are after is called findlib. It is included in the base GODI installation. The tools that come with findlib allow you to easily compile against most OCaml libraries and use those libraries from a toplevel session (ocaml). The findlib documentation is fairly comprehensive, but here is a quick summary to get started.
To start using findlib from within a toplevel session:
#use "topfind";;
This will display a brief usage message. Then you can type:
#list;;
This will show you a list of all of the available packages. Yojson will likely be among them. Finally:
#require "yojson";;
where yojson is replaced by the appropriate entry shown by #list;;. Yojson's modules should be available for you to use at this point.

Understand foreign function interface (FFI) and language binding

Mixing different programming languages has long been something I don't quite understand. According to this Wikipedia article, a foreign function interface (or FFI) can be done in several ways:
Requiring that guest-language functions which are to be host-language callable be specified or implemented in a particular way; often using a compatibility library of some sort.
Use of a tool to automatically "wrap" guest-language functions with appropriate glue code, which performs any necessary translation.
Use of wrapper libraries
Restricting the set of host language capabilities which can be used cross-language. For example, C++ functions called from C may not (in general) include reference parameters or throw exceptions.
My questions:
What are the differences between the
1st, 2nd and 3rd ways? It seems to
me they are all to compile the code of
the called language into some
library with object files and header
files, which are then called by the
calling language.
One source it links says,
implementing an FFI can be done in
several ways:
Requiring the called functions in the target language implement a
specific protocol.
Implementing a wrapper library that takes a given low-language
function, and "wraps" it with code to do data conversion to/from the
high-level language conventions.
Requiring functions declared native to use a subset of high-level functionality (which is compatible with the low-level language).
I was wondering if the first way in
the linked source is the same as the
first way in Wikipedia?
What does the third way in this
source mean? Does it corresponds to the 4th way in Wikipedia?
In the same source, when comparing the three ways it lists, it seems to say
the job of filling the gap between
the two languages is gradually
shifted from the called language
to the calling language. I was
wondering how to understand that? Is this shifting also true for the four ways in Wikipedia?
Are Language binding and FFI
equivalent concepts? How are they
related and differ?
a binding from a programming language
to a library or OS service is an API
providing that service in the
language.
I was wondering which way in the quotation from Wikipedia or from the source each of the following examples belongs to?
Common Object Request Broker Architecture (CORBA)
Calling C in C++, by the extern "C" declaration in C++ to
disable name mangling.
Calling C in Matlab, by MATLAB Interface to Shared Libraries, i.e., first compiling C code to shared library via general C
compiler such as gcc, and then
loading, calling a function from
and unloading the shared library
via Matlab functions
loadlibrary(), calllib() and
unloadlibrary().
Calling C in Matlab, by Creating C/C++ Language MEX-Files
Calling Matlab in C, by mcc compiler
Calling C++ in Java, by JNI, and Calling Java in C++, also by JNI
Calling C/C++ in other languages, Using SWIG
Calling C in Python, by Ctypes module.
Cython
Calling R in Python, by RPy
Programming Language Bindings to OpenGL from various languages, such as Python, Fortran and Java
Bindings for a C library, such as Cairo, from various languages,
such as C++, Python, Java, Common Lisp
May be a specific example will help. Let us take the host language as Python and the guest language as C. This means that Python will be calling C functions.
The first option is to write the C library in a particular way. In the case of Python the standard way would be to have the C function written with a first parameter of Py_Object * among other conditions. For example (from here):
static PyObject *
spam_system(PyObject *self, PyObject *args)
{
const char *command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return Py_BuildValue("i", sts);
}
is a C function callable from Python. For this to work the library has to be written with Python compatibility in mind.
If you want to use an already existing C library, you need another option. One is to have a tool that generates wraps this existing library in a format suitable for consumption by the host language. Take Swig which can be used to tie many languages. Given an existing C library you can use swig to effectively generate C code that calls your existing library while conforming to Python conventions. See the example for building a Python module.
Another option to us an already existing C library is to call it from a Python library that effectively wraps the calls at run time, like ctypes. While in option 2 compilation was necessary, it is not this time.
Another thing is that there are a lot of options (which do overlap) for calling functions in one language from another language. There are FFIs (equivalent to language bindings as far as I know) which usually refer to calling between multiple languages in the same process (as part of the same executable, so to speak), and there are interprocess communication means (local and network). Things like CORBA and Web Services (SOAP or REST) and and COM+ and remote procedure calls in general are of the second category and are not seen as FFI. In fact, they mostly don't prescribe any particular language to be used at either side of the communication. I would loosely put them as IPC (interprocess communication) options, though this is simplification in the case of network based APi like CORBA and SOAP.
Having a go at your list, I would venture the following opinions:
Common Object Request Broker Architecture: IPC, not FFI
Calling C in C++, by the extern "C" declaration in C++ to disable name mangling. ****
Calling C in Matlab, by MATLAB Interface to Shared Libraries Option 3 (ctypes-like)
Calling C in Matlab, by Creating C/C++ Language MEX-Files Option 2 (swig-like)
Calling Matlab in C, by mcc compiler Option 2 (swig-like)
Calling C++ in Java, by JNI, and Calling Java in C++ by JNI Option 3 (ctypes-like)
Calling C/C++ in other languages, Using SWIG Option 2 (swig)
Calling C in Python, by Ctypes Option 3 (ctypes)
Cython Option 2 (swig-like)
Calling R in Python, by RPy Option 3 (ctypes-like) in part, and partly about data exchange (not FFI)
The next two are not foreign function interfaces at all, as the term is used. FFi is about the interaction between two programming languages and should be capable of making any library (with suitable restrictions) from one language available to the other. A particular library being accessible from one language does not an FFI make.
Programming Language Bindings to OpenGL from various languages
Bindings for a C library from various languages

How do they write different language wrappers for same library?

Generally a library will be released in a single language (for example C). If the library tuns out to be useful then many language wrappers for that library will be written. How exactly do they do it?
Kindly someone throw little light on this topic. If it is too language dependent pick language of your choice and explain it.
There are a few options that come to mind:
Port the original C library to the language/platform of your choice
Compile the C library into something (like a DLL) that can be invoked from other components
Put the library on the web, expose an API over HTTP and wrap that on the client
If I wanted to wrap a C library with a managed (.NET) layer, I'd compile the library into a DLL, exposing the APIs I wanted. Then, I'd use P/Invoke to call those APIs from my C# code.