I am trying to use the GSL library in a Cython program but don't seem to have the paths correctly specified; I encounter the following error when I try writing a simple example:
%load_ext cythonmagic
%%cython -lgsl -lgslcblas
cdef extern from "gsl/gsl_ran_poisson_pdf.h":
double gsl_ran_poisson_pdf(int x, double mu)
def poison(int x, double mu):
return gsl_ran_poisson_pdf(x,mu)
/Users/name/.ipython/cython/_cython_magic_189673701925d12059c18b75663da8bd.c:317:10: fatal error:
'gsl/gsl_mode.h' file not found
#include "gsl/gsl_mode.h"
I get the same error using CythonGSL and the demo program here: http://nbviewer.ipython.org/github/twiecki/CythonGSL/blob/master/examples/cython_gsl_ipythonnb.ipynb
The GSL libraries are located in the following directories:
-I/usr/local/include
-L/usr/local/lib -lgsl
I know that similar questions have been asked on SO before, but I couldn't find one relevant to my situation and system (I'm using OS-X). Any help would be appreciated.
Thanks!
Related
If I made a python file named hello.py that has a script made like this.
msg = input("insert your message here: ")
script = '''
def say_something():
print("{msg}")
'''
exec(script)
say_something()
And then I tried to use Cython
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules=cythonize("Hello.py")
)
It will show an error like this: undeclared name not builtin: say_something
I do understand why this happens but I'm not really an expert with python and C just yet. This is just an example, but it's similar to what I'm trying to do with one of my projects. Is there any way I could resolve this? I want to find a way to convert the script string into C as well.
I was trying to build an editable python script.
Cython compiles the Python functions to a native binary that does what the CPython interpreter should do. exec is a function that execute arbitrary code at runtime (which is generally a very bad idea for speed, maintainability/readability and security). Cython does not support exec because it would mean that the could would be compiled at runtime. Thus, the code executed by exec cannot be a Cython code. However, the exec function can still be used to execute a pure-Python code. The error can be removed by turning off the Cython.Compiler.Options.error_on_unknown_names in the setup script (just before calling setup) as pointed out by #DavidW. With this Cython will not complain when it does not find a function defined by exec (or similar methods). Please keep in mind that CPython can only be used in this case instead of Cython (which partially defeat the purpose of using Cython in the first place).
I am new to Cython and I wanted to wrap a simple c-function to python. In the C-File i use: #include<complex.h>
When i try to build the wrapper i get following error:
lib/mandelbrot_c.c(10): error C2146: Syntaxfehler: missing";" before "c"
lib/mandelbrot_c.c(10): error C2065: "c": not declared
lib/mandelbrot_c.c(10): error C2065: "z": not declared
...
The problem is that the macro does not work i guess.
The code used in the c-file is: double complex c, z;
The files can be found here:
https://github.com/HaidStefan/python_wrap_c_libraries/tree/dev/cython
Does anyone know a solution for this? Sorry if this question is dumb but i am new to cython and I'm trying to make a benchmark between different wrappers.
Thanks a lot in advance
Stefan
The complex library from the C99 standard library is not supported fom the MSVC compiler, which is used to build cython.
See: Compiling C code in Visual Studio 2013 with complex.h library
I am attempting to use the LMDB C API with Cython.
I want to import the following definitions from the header file:
typedef struct MDB_env MDB_env;
int mdb_env_create(MDB_env **env);
So I created a .pxd file:
cdef extern from 'lmdb.h':
struct MDB_env:
pass
int mdb_env_create(MDB_env **env)
And I am using it in a Cython script:
cdef MDB_env *e
x = mdb_env_create(&e)
This code compiles fine, but If I run it, I get:
ImportError: /home/me/.cache/ipython/cython/_cython_magic_15705c11c6f56670efe6282cbabe4abc.cpython-36m-x86_64-linux-gnu.so: undefined symbol: mdb_env_create
This happens both in a Cython .pyx + .pxd setup and in a prototype typed in IPython.
If I import another symbol, say a constant, I can access it. So I seem to be looking at the right header file.
I don't see any discrepancy between my syntax and the documentation, but I am clearly doing something wrong. Can somebody give me a hint?
Thanks.
To compile it with IPythons-magic (would be nice if you would mention this explicitly in your question) you have to provide library-path (via -L-option) and library name (via -l-option) of the built c-library you want to wrap, see also the documentation:
%%cython -L=<path to your library> -l=<your_library>
The library you are trying to wrap is not a header-only library. That means that some symbols (e.g. mdb_env_create) are only declared but not defined in the header. When you build the library, the definitions of those symbols can be found in the resulting artifact, which should be provided to the linker when your extension is built. These definitions is what is needed when the program runs.
If you don't do it, the following happens on Linux: When the extension (the *.so-file) is built,the linker allows undefined symbols per default - so this step is "successful" - but the failure is only postponed. When the extension is loaded via import, Python loads the corresponding *.so with help of ldopen and in this step loader checks that the definitions of all symbols are known. But we didn't provide a definition of mdb_env_create so, the loader fails with
undefined symbol: mdb_env_create
It is differently for symbols which are defined in the header-file, for example enums MDB_FIRST&Co - the compiled library isn't necessary and thus the extension can be loaded, as there are no undefined symbols.
Since I made some progress, I changed the title and made a second edit describing my new problem. You may choose to ignore Edit1
I have been trying to run python code from C code. And for this purpose I have been using Cython.
The semantics of my system is such that there is a binary (whos source I can not access) that calls a C function defined in a file (source is accessible) and within this function I need to call python functions, do some processing and return the result to binary.
To achieve this purpose, there are two approaches that I came across:
http://docs.python.org/release/2.5.2/ext/callingPython.html ===> This approach suggests to have the python callback function passed to the C side, so that the callback is called as necessary, but this doesn't work for me as I don't have access to the binary's source (which is used to run the entire system)
https://stackoverflow.com/a/5721123/1126425 ==> I have tried this approach and I get this error when the cython function is called:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb47deb70 (LWP 2065)]
0x007fd38a in PySys_GetObject () from /usr/lib/libpython2.6.so.1.0
http://www.linuxjournal.com/article/8497?page=0,0 ==> This is in fact the basis for cython's functionality but again when I use the examples described there, I get errors similar to 2.
I have no idea how to resolve these errors. Any help would be much appreciated.
Thanks!!
Edit1:
here is a simple scenario that reflects situation:
external.c
#include <external.h>
int callback(int param1,int param2)//Function that the binary calls
{
/*SomeTasks*/
cython_func();//Function defined in the following .pyx file
/*SomeTasks*/
}
cython_file.pyx
cdef void cython_function():
print "Do Nothing!"
I am linking the shared library file created by cython with the library generated by compiling the above C code and then that library is used by the binary...
Edit2:
The segmentation fault goes away when I added Py_Initialize(); before calling cython_function(). But now I am getting the undefined symbol error as : symbol lookup error: lib_c_code.so: undefined symbol: cython_function
Here lib_c_code.so is the shared library created out of the external.c file above. I have tried including the .h file created by the cython compiler in external.c but it still didn't work out.. Here is how I am compiling lib_c_code.so:
gcc -shared -dynlib -lm -W1 -o lib_c_code.so $(OBJDIR)/*.o -lc -lm -lpy_code
and the libpy_code.so is the shared object file that was created out of the cython_file.pyx file as:
cython cython_file.pyx -o cython_file.c
gcc $(IFLAGS) -I/usr/include/python2.6 -fPIC -shared cython_file.c -lpython2.6 -lm -o libpy_code.so
Also, I can see the symbol cython_function in the lib_c_code.so file when I do : nm -g lib_c_code.so..
Any ideas please?
I have to guess here that there's a callback registration function to which you can pass the function pointer, in which case you can simply forego the C file and define a cdef function directly in your Cython code, and pass that with the callback registration function. Use with gil in case you manipulate any Python objects in it.
cdef extern from "external.h":
ctypedef int (*Cb_Func)(int param1, int param2)
void register_callback(Cb_Func func)
cdef int my_callback(int param1,int param2) with gil:
<implementation>
register_callback(my_callback)
This is also explained in the Cython user manual here: http://docs.cython.org/src/userguide/external_C_code.html
I tried messing around with Win32 binaries lately (this is for a big project of mine).
So after some weeks of research, I now have a solid understanding of how Assembly works, how it is converted into binary code and how x86/x64 opcodes work.
The last piece to the puzzle is figuring out how to properly call Win32 API methods.
I actually asked a question on here in relation to this, and the answer I got was, I should try and compile an Assembly or C program that does this. So I went ahead and tried this in Assembly (I'm using FASM by the way):
format PE console
entry start
section '.idata' import data readable writable
include 'win32a.inc'
library kernel,'kernel32.dll'
import kernel,\
GetStdHandle,'GetStdHandle',\
WriteConsoleA,'WriteConsoleA'
section '.data' data readable writable
string db 'Hello!', 0h
output dd ?
section '.code' code readable executable
start: push -11
call GetStdHandle
pushd 0
pushd output
pushd 7
pushd string
pushd eax
call WriteConsoleA
This is one of the many versions of this code actually. The main problem is, when I call methods like "ExitProcess", generally other functions from the kernel32.dll library, things seem to work out. It's the IO functions that bug me...
I don't understand what's wrong with this code, I don't get any compile-time errors, though when I run it, it just crashes.
So my next idea was, since this didn't work, to try the same in C.
I'm using Cygwin as a compiler and linker...
#include <windows.h>
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow) {
MessageBox(NULL, "Hello, world!", "Test", MB_OK);
return 0;
}
This code yielded the same result, the application crashed.
Now, I am not looking for any C/C++ code. My original question of interest was to know how calling extern library function looks like in x86/x64 binary (assembled) code. But I would be very thankful for any resources regarding this topic.
Thank you in advance.
-Tom S.
Your problem is that you need to call ExitProcess at the end to properly end the process. Since you are not doing that, the code currently will continue executing and eventually segfaults because it attempts to execute junk bytes.