I am attempting to use the LMDB C API with Cython.
I want to import the following definitions from the header file:
typedef struct MDB_env MDB_env;
int mdb_env_create(MDB_env **env);
So I created a .pxd file:
cdef extern from 'lmdb.h':
struct MDB_env:
pass
int mdb_env_create(MDB_env **env)
And I am using it in a Cython script:
cdef MDB_env *e
x = mdb_env_create(&e)
This code compiles fine, but If I run it, I get:
ImportError: /home/me/.cache/ipython/cython/_cython_magic_15705c11c6f56670efe6282cbabe4abc.cpython-36m-x86_64-linux-gnu.so: undefined symbol: mdb_env_create
This happens both in a Cython .pyx + .pxd setup and in a prototype typed in IPython.
If I import another symbol, say a constant, I can access it. So I seem to be looking at the right header file.
I don't see any discrepancy between my syntax and the documentation, but I am clearly doing something wrong. Can somebody give me a hint?
Thanks.
To compile it with IPythons-magic (would be nice if you would mention this explicitly in your question) you have to provide library-path (via -L-option) and library name (via -l-option) of the built c-library you want to wrap, see also the documentation:
%%cython -L=<path to your library> -l=<your_library>
The library you are trying to wrap is not a header-only library. That means that some symbols (e.g. mdb_env_create) are only declared but not defined in the header. When you build the library, the definitions of those symbols can be found in the resulting artifact, which should be provided to the linker when your extension is built. These definitions is what is needed when the program runs.
If you don't do it, the following happens on Linux: When the extension (the *.so-file) is built,the linker allows undefined symbols per default - so this step is "successful" - but the failure is only postponed. When the extension is loaded via import, Python loads the corresponding *.so with help of ldopen and in this step loader checks that the definitions of all symbols are known. But we didn't provide a definition of mdb_env_create so, the loader fails with
undefined symbol: mdb_env_create
It is differently for symbols which are defined in the header-file, for example enums MDB_FIRST&Co - the compiled library isn't necessary and thus the extension can be loaded, as there are no undefined symbols.
Related
If I made a python file named hello.py that has a script made like this.
msg = input("insert your message here: ")
script = '''
def say_something():
print("{msg}")
'''
exec(script)
say_something()
And then I tried to use Cython
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules=cythonize("Hello.py")
)
It will show an error like this: undeclared name not builtin: say_something
I do understand why this happens but I'm not really an expert with python and C just yet. This is just an example, but it's similar to what I'm trying to do with one of my projects. Is there any way I could resolve this? I want to find a way to convert the script string into C as well.
I was trying to build an editable python script.
Cython compiles the Python functions to a native binary that does what the CPython interpreter should do. exec is a function that execute arbitrary code at runtime (which is generally a very bad idea for speed, maintainability/readability and security). Cython does not support exec because it would mean that the could would be compiled at runtime. Thus, the code executed by exec cannot be a Cython code. However, the exec function can still be used to execute a pure-Python code. The error can be removed by turning off the Cython.Compiler.Options.error_on_unknown_names in the setup script (just before calling setup) as pointed out by #DavidW. With this Cython will not complain when it does not find a function defined by exec (or similar methods). Please keep in mind that CPython can only be used in this case instead of Cython (which partially defeat the purpose of using Cython in the first place).
The following code defines a simple Cython function (using Ipython magic, for convenience).
%load_ext cython
%%cython
def f(float x, float y=2):
return x+y
Then, calling help(f) gives this message:
Help on built-in function f in module _cython_magic_e37eeabbc63d5167217465ba978239fc:
f(...)
Note that the arguments of f are not shown. Also, the tab-completion does not work either for the argument names in ipython (e.g. typing f(x then tab).
If I define this function without using Cython:
def g(x, y=2):
return x+y
Calling help(g) gives this and the tab-completion works as expected:
Help on function g in module __main__:
g(x, y=2)
Is there a way to get this behavior with the Cython function? I tried with def, cdef, cpdef, with and without ipython magic, with no success.
Disagree with the agreed answer.
While enabling binding does have the side-effect of documentation strings showing up in code, it also binds all other Python class attributes to Cython extension classes which makes for lower performance, more memory used for each extension object and so on.
The correct flag to enable docstrings only without side effects is embedsignature=True.
It can either be used as decorator - #cython.embedsignature(True) on top of all functions, or part of cython directives in setup.py Extension to apply to all Cython functions - {'embedsignature': True}
From docs:
embedsignature (True / False)
If set to True, Cython will embed a textual copy of the call signature in the docstring of all Python visible functions and
classes. Tools like IPython and epydoc can thus display the signature,
which cannot otherwise be retrieved after compilation. Default is
False.
import cython
#cython.binding(True)
def f(float x, float y=2):
# ...
now help(f) gives
Help on cython_function_or_method in module cy_exc:
f(x, y=2.0)
The documentation says
When enabled, functions will bind to an instance when looked up as a class attribute (hence the name) and will emulate the attributes of Python functions, including introspections like argument names and annotations. Default is False.
You can enable the compilation option in other ways (for example, if you want it enabled everywhere).
You might also want to look at this related question
I have a Clojurescript program running in the browser.
It imports a number of libraries, and then I want to allow the user to enter some small clojurescript "glue-code" that calls those libraries.
I can see (from https://cljs.github.io/api/cljs.js/eval) that you call eval with four arguments, the first being the state of the environment, which is an atom. But can I actually turn my current environment with all the functions I've required from elsewhere, into an appropriate argument to eval?
Update :
I thought that maybe I could set the namesspace for the eval using the :ns option of the third, opts-map, argument. I set it to the namespace of my application :
:ns "fig-pat.core"
But no difference.
Looking at the console, it's definitely the case that it's trying to do the evaluation, but it's complaining that names referenced in the eval-ed code are NOT recognised :
WARNING: Use of undeclared Var /square
for example. (square is a function I'm requiring. It's visible in the application itself ie. the fig-pat.core namespace)
I then get :
SyntaxError: expected expression, got '.'[Learn More]
Which I'm assuming this the failure of eval-ed expression as a whole.
Update 2 :
I'm guessing this problem might actually be related to : How can I get the Clojurescript namespace I am in from within a clojurescript program?
(println *ns*)
is just printing nil. So maybe Clojurescript can't see its own namespace.
And therefore the :ns in eval doesn't work?
Calling eval inside a clojurescript program is part of what is called "self-hosted clojurescript".
In self-hosted clojurescript, namespaces are not available unless you implement a resolve policy. It means that have to let the browser know how to resolve the namespace e.g. loads a cljs file from a cdn.
It's not so trivial to implement namespace resolving properly.
This is explained in a cryptic way in the docstring of load-fn from cljs.js namespace.
Several tools support namespaces resolving in self-host cljs running in the browser e.g Klipse and crepl
I am developing Haxe code that I convert in C# and insert into a Unity project.
The conversion works fine and I am able to use the generated class when I import it alone in Unity.
To make it work, I have to bring in Unity the whole generated src folder, including the Type.cs file.
However, when I import the "Post Processing Stack" (a basic Unity extension) I get errors due to name Conflicts. The Type class is also a basic C# class and It is used by the Post Processing scripts.
The haxe-Type takes priority and breaks the compilation:
Assets/PostProcessing/Runtime/PostProcessingBehaviour.cs(406,30): error CS1502: The best overloaded method match for `System.Collections.Generic.Dictionary<Type,System.Collections.Generic.KeyValuePair<UnityEngine.Rendering.CameraEvent,UnityEngine.Rendering.CommandBuffer>>.Add(Type, System.Collections.Generic.KeyValuePair<UnityEngine.Rendering.CameraEvent,UnityEngine.Rendering.CommandBuffer>)' has some invalid arguments
Assets/PostProcessing/Runtime/PostProcessingBehaviour.cs(406,34): error CS1503: Argument `#1' cannot convert `System.Type' expression to type `Type'
I don't know if it is possible to solve this issue by playing around with C#/Unity/Mono search paths.
I as wondering wether it is more appropriate to (optionally) wrap all haxe top-level classes into the haxe namespace, or a special haxe-defaultnamespace, or prefix them for example to class HType.
Name conflicts for this basic types are likely to emerge in many other contexts, not only in Unity.
I found the solution in the Haxe documentation for C#:
https://github.com/HaxeFoundation/HaxeManual/wiki/Haxe-C%23
-D no-root generate package-less haxe types in the haxe.root namespace to avoid conflicts with other types in the root namespace
This way, all the classes that were at global level will be generated under namespace haxe.root.
Since I made some progress, I changed the title and made a second edit describing my new problem. You may choose to ignore Edit1
I have been trying to run python code from C code. And for this purpose I have been using Cython.
The semantics of my system is such that there is a binary (whos source I can not access) that calls a C function defined in a file (source is accessible) and within this function I need to call python functions, do some processing and return the result to binary.
To achieve this purpose, there are two approaches that I came across:
http://docs.python.org/release/2.5.2/ext/callingPython.html ===> This approach suggests to have the python callback function passed to the C side, so that the callback is called as necessary, but this doesn't work for me as I don't have access to the binary's source (which is used to run the entire system)
https://stackoverflow.com/a/5721123/1126425 ==> I have tried this approach and I get this error when the cython function is called:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb47deb70 (LWP 2065)]
0x007fd38a in PySys_GetObject () from /usr/lib/libpython2.6.so.1.0
http://www.linuxjournal.com/article/8497?page=0,0 ==> This is in fact the basis for cython's functionality but again when I use the examples described there, I get errors similar to 2.
I have no idea how to resolve these errors. Any help would be much appreciated.
Thanks!!
Edit1:
here is a simple scenario that reflects situation:
external.c
#include <external.h>
int callback(int param1,int param2)//Function that the binary calls
{
/*SomeTasks*/
cython_func();//Function defined in the following .pyx file
/*SomeTasks*/
}
cython_file.pyx
cdef void cython_function():
print "Do Nothing!"
I am linking the shared library file created by cython with the library generated by compiling the above C code and then that library is used by the binary...
Edit2:
The segmentation fault goes away when I added Py_Initialize(); before calling cython_function(). But now I am getting the undefined symbol error as : symbol lookup error: lib_c_code.so: undefined symbol: cython_function
Here lib_c_code.so is the shared library created out of the external.c file above. I have tried including the .h file created by the cython compiler in external.c but it still didn't work out.. Here is how I am compiling lib_c_code.so:
gcc -shared -dynlib -lm -W1 -o lib_c_code.so $(OBJDIR)/*.o -lc -lm -lpy_code
and the libpy_code.so is the shared object file that was created out of the cython_file.pyx file as:
cython cython_file.pyx -o cython_file.c
gcc $(IFLAGS) -I/usr/include/python2.6 -fPIC -shared cython_file.c -lpython2.6 -lm -o libpy_code.so
Also, I can see the symbol cython_function in the lib_c_code.so file when I do : nm -g lib_c_code.so..
Any ideas please?
I have to guess here that there's a callback registration function to which you can pass the function pointer, in which case you can simply forego the C file and define a cdef function directly in your Cython code, and pass that with the callback registration function. Use with gil in case you manipulate any Python objects in it.
cdef extern from "external.h":
ctypedef int (*Cb_Func)(int param1, int param2)
void register_callback(Cb_Func func)
cdef int my_callback(int param1,int param2) with gil:
<implementation>
register_callback(my_callback)
This is also explained in the Cython user manual here: http://docs.cython.org/src/userguide/external_C_code.html