I'm trying to follow this part of the Cython docs https://cython.readthedocs.io/en/latest/src/userguide/fusedtypes.html :
$ ipython
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: %load_ext cython
In [2]: %%cython -a
...:
...: ctypedef fused char_or_float:
...: char
...: float
...:
Error compiling Cython file:
------------------------------------------------------------
...
ctypedef fused char_or_float:
^
------------------------------------------------------------
/home/ignoring_gravity/.cache/ipython/cython/_cython_magic_8280d4aa00f7cb88541f787d632e437c.pyx:2:0: Compiler crash in AnalyseDeclarationsTransform
Compiler crash traceback from this point on:
File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit
File "/home/ignoring_gravity/.venv/lib/python3.9/site-packages/Cython/Compiler/ParseTreeTransforms.py", line 1611, in visit_ModuleNode
node.body.stats.extend(self.extra_module_declarations)
AttributeError: 'NoneType' object has no attribute 'stats'
In [3]: import cython
In [4]: cython.__version__
Out[4]: '0.29.30'
I can't tell, however, why this fails for me. Am I not doing the same thing as in the docs?
Related
I am trying to make the following code in cython to compile
from cython_gsl cimport *
import atexit
cdef gsl_rng_type * rng_T = gsl_rng_default
cdef gsl_rng * rng_r
gsl_rng_env_setup()
rng_r = gsl_rng_alloc(rng_T)
#atexit.register
def free_gsl_rng():
gsl_rng_free(rng_r)
But I always get the error
Assignment to non-lvalue 'atexit'
Corresponding .pxd file
from cython_gsl cimport *
cdef gsl_rng * rng_r
I am actually compiling this with SageMath 8.7 by
sage setup.py build_ext --inplace
Here is my setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
from Cython.Build import cythonize
import cython_gsl
extensions = [
Extension("gsl_rand", ["gsl_rand.pyx"],
libraries=cython_gsl.get_libraries(),
library_dirs=[cython_gsl.get_library_dir()],
include_dirs=[cython_gsl.get_cython_include_dir()]
),
]
setup(
name='Simulation of k-cut on conditional Galton-Watson trees',
cmdclass={'build_ext': build_ext},
include_dirs = [cython_gsl.get_include()],
ext_modules=cythonize(extensions),
)
Full error log here
sage setup.py build_ext --inplace
Compiling gsl_rand.pyx because it changed.
[1/1] Cythonizing gsl_rand.pyx
/home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/xing/Dropbox/Research/2017/k-cut/GW/sage/moments/cython-v5/gsl_rand.pxd
tree = Parsing.p_module(s, pxd, full_module_name)
warning: /home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/cython_gsl/gsl_integration.pxd:65:9: 'GSL_EMAXITER' redeclared
warning: /home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/cython_gsl/gsl_integration.pxd:67:9: 'GSL_EROUND' redeclared
warning: /home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/cython_gsl/gsl_integration.pxd:69:9: 'GSL_ESING' redeclared
warning: /home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/cython_gsl/gsl_integration.pxd:71:9: 'GSL_EDIVERGE' redeclared
warning: gsl_rand.pyx:14:0: Overriding cdef method with def method.
Error compiling Cython file:
------------------------------------------------------------
...
# cython: profile=False
from cython_gsl cimport *
import atexit
^
------------------------------------------------------------
gsl_rand.pyx:5:7: Assignment to non-lvalue 'atexit'
Error compiling Cython file:
------------------------------------------------------------
...
cdef gsl_rng * rng_r
gsl_rng_env_setup()
rng_r = gsl_rng_alloc(rng_T)
#atexit.register
^
------------------------------------------------------------
gsl_rand.pyx:13:0: Object of type 'int (void (*)(void) nogil) nogil' has no attribute 'register'
Traceback (most recent call last):
File "setup.py", line 29, in <module>
ext_modules=cythonize(extensions),
File "/home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 1097, in cythonize
cythonize_one(*args)
File "/home/xing/Downloads/software/sage/8.7/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 1220, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: gsl_rand.pyx
atexit is getting cimported with from cython_gsl cimport *. This appears to happen with the line from libc.stdlib cimport *. This name therefore conflicts with the Python atexit module. I think this is a good example of why from something [c]import * is not recommended, both in your code and in cython_gsl's code. It gets slightly more confusing because Cython has two types of names (for Python and C) so you get odd error messages.
The best solution is to either do cimport cython_gsl or to cimport the specific symbols you need: from cython_gsl cimport gsl_rng, etc. Remember to change both the pyx and pxd file.
The worse workround is to rename the atexit module when you import it: import atexit as ae.
MCVE
In terminal while loading IPython/Jupyter Notebook as .json:
$ python
Python 3.5.2 | packaged by conda-forge | (default, Jul 26 2016, 01:32:08)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> import io
>>> with io.open('Untitled.ipynb','r',encoding='utf-8') as sf:
... data = json.load(sf) # error on this line
... print(data)
...
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character '\xf3' in position 189292: ordinal not in range(128)
How to make this work?
Also:
>>> print(sys.getdefaultencoding())
utf-8
>>> print(sys.stdout.encoding)
ANSI_X3.4-1968 # I assume this is the main reason why it does not work
What have I tried?
export PYTHONIOENCODING=UTF-8 does not work.
import importlib; importlib.reload(sys); sys.setdefaultencoding("UTF-8") does not work.
sys.stdout = codecs.getwriter('utf-8')(sys.stdout.buffer, 'strict') gives TypeError: write() argument must be str, not bytes
Related: 1, 2, 3
I have a Cython code that runs and works fine under WinPython but giving me issues when I switch from WinPython to Anaconda3. I am using Python 3.4
The test code is :
# cython: boundscheck=False
# cython: wraparound=False
# cython: cdivision=True
cimport cython
cimport numpy as np
import numpy as np
from numpy cimport ndarray as ar
from libc.math cimport *
cpdef ar[double, ndim=1, mode='c'] test(ar[double, ndim=1, mode='c'] x):
cdef:
int n = x.shape[0]
Py_ssize_t i
ar[double, ndim=1, mode='c'] y = np.zeros(n)*np.nan
with nogil:
for i in range(0, n):
y[i] = x[i]+1
return y
The compilation code is:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy as np
ext_modules = [Extension('test', ['tech/test.pyx'], include_dirs=[np.get_include()],
define_macros=[('NPY_NO_DEPRECATED_API', None)],
extra_compile_args=['-O3', '-march=native', '-ffast-math'],
libraries=['m']
)]
setup(
name="Test Function",
cmdclass={'build_ext': build_ext},
ext_modules=ext_modules
)
I am using MinGw to compile, so under the ...\Anaconda3\Lib\distutils folder I have a file that has
[build]
compiler = mingw32
In addition in the PATH environment variables, I have:
default:
...\Anaconda3
...\Anaconda3\Scripts
also added:
...\Anaconda3\libs (this contains python34)
also added: mingw32 files copied from WinPython that contain gcc etc:
...\Anaconda3\Tools\tools\mingw32\bin (this contains gcc)
...\Anaconda3\Tools\tools\mingw32\x86_64-w64-mingw32\bin
when I try:
python setup.py build_ext --inplace
the code runs fine until the test.c, test.o and test.def have been generated. thereafter this is what I get:
C:\Anaconda3\Tools\tools\mingw32\bin\gcc.exe -shared -s build\temp.win-amd64-3.4\Release\tech\test.o build\temp.win-amd64-3.4\Release\tech\test.def -LC:\Anaconda3\libs -LC:\Anaconda3\PCbuild\amd64 -lm -lpython34 -lmsvcr100 -
o P:\Documents\Temp\python-master\python-master\common\test.pyd
build\temp.win-amd64-3.4\Release\tech\test.o:test.c:(.text+0x8e): undefined reference to `__imp_PyExc_TypeError'
build\temp.win-amd64-3.4\Release\tech\test.o:test.c:(.text+0x10e): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-3.4\Release\tech\test.o:test.c:(.text+0x259): undefined reference to `__imp__PyThreadState_Current'
....
....
C:/Anaconda3/Tools/tools/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/../../../../x86_64-w64-mingw3 ild\temp.win-amd64-3.4\Release\tech\test.o: bad reloc address 0x0 in section `.data'
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\Anaconda3\\Tools\\tools\\mingw32\\bin\\gcc.exe' failed with exit status 1
First of all, I couldn't find a Anacond3\PCBuild\amd64 directory/file anywhere.
I tried to look all over the internet, but I couldn't find any reference to __imp_PyExc_TypeError, __imp__PyThreadState_Current, __imp_PyExc_ValueError.
What could be going wrong with the final conversion to *.pyd file?
I suspect Anaconda doesn't support mingw-64 yet.
Historically mingw-64 was indeed a hopeless idea until Cark Kleffner came with his static version.
Things may (should ?) change when Carl's updated version will be available as mingwpy wheel.
You can run
conda remove libpython
to make Anaconda use Visual Studio instead of mingw, which may work better.
sfml.pxd:
cdef extern from "SFML/Window.hpp" namespace "sf":
cdef cppclass VideoMode:
VideoMode(unsigned int, unsigned int) except +
cdef cppclass Window:
Window(VideoMode, String) except +
void display()
display.pyx:
cimport sfml
cdef class Window:
cdef sfml.Window* _this
def __cinit__(self, unsigned int width, unsigned int height):
self._this = new sfml.Window(sfml.VideoMode(width, height), "title")
def __dealloc__(self):
del self._this
def display(self):
self._this.display()
setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("display", ["display.pyx"],
language='c++',
libraries=["sfml-system", "sfml-window"])
]
)
The error when running python setup.py build:
running build
running build_ext
cythoning display.pyx to display.cpp
Traceback (most recent call last):
File "setup.py", line 10, in <module>
libraries=["sfml-system", "sfml-window"])
File "/usr/lib/python3.3/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.3/distutils/dist.py", line 917, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/lib/python3.3/distutils/command/build.py", line 126, in run
self.run_command(cmd_name)
File "/usr/lib/python3.3/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 163, in run
_build_ext.build_ext.run(self)
File "/usr/lib/python3.3/distutils/command/build_ext.py", line 354, in run
self.build_extensions()
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 170, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 317, in cython_sources
full_module_name=module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 608, in compile
return compile_single(source, options, full_module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 549, in compile_single
return run_pipeline(source, options, full_module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 386, in run_pipeline
from . import Pipeline
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Pipeline.py", line 7, in <module>
from .Visitor import CythonTransform
File "Visitor.py", line 10, in init Cython.Compiler.Visitor (/build/src/Cython-0.19/Cython/Compiler/Visitor.c:15987)
ImportError: No module named 'ExprNodes'
Apparently, it can't find something called 'ExprNodes', but I don't think that my Cython installation is broken, because I managed to successfully wrap a different C++ library some time ago, and I didn't run into this problem.
I'm using Cython 0.19.
I would appreciate any help/insight that you could offer.
Thanks.
Looking more closely at the traceback, I see that Cython fails inside it's own compiled code. It may be a bug indeed, sorry for missing it the first time.
What can you do:
Create a clean virtualenv, install Cython there and check if it works. (Version 0.19.1 is the latest).
Create another virtualenv, but this time install Cython using python setup.py install --no-cython-compile.
If either of these fails, please post your detailed configuration (linux distro and version, python version, gcc version, etc.) to the cython-devel mailing list.
BTW does your old successful project still compile?
I have a question to Ctypes and do not know what I do wrong. And yes, I am a newbie for Python and searched the other posts in here. So any advice is appreciated.
What do I want to do :
I simply want to load the FXCM C++ APP fundtions into Python 3.3 so I can call them for connecting to their server.
As it seems Ctypes seems to be the best tool. So a simple code in Python :
import os
dirlist = os.listdir('ForexConnectAPIx64/bin')
from pprint import pprint
pprint(dirlist)
from ctypes import *
myDll = cdll.LoadLibrary ("ForexConnectAPIx64/bin/ForexConnect.dll")
gives a result :
Traceback (most recent call File "C:\Users\scaberia3\Python_Projects \FTC\ListDir_Test.py", line 20, in <module>
myDll = cdll.LoadLibrary ("ForexConnectAPIx64/bin/ForexConnect.dll")
File "C:\Python33\lib\ctypes\__init__.py", line 431, in LoadLibrary
return self._dlltype(name)
File "C:\Python33\lib\ctypes\__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] Das angegebene Modul wurde nicht gefunden (Module not found)
['ForexConnect.dll',
'fxmsg.dll',
'gsexpat.dll',
'gslibeay32.dll',
'gsssleay32.dll',
'gstool2.dll',
'gszlib.dll',
'java',
'log4cplus.dll',
'msvcp80.dll',
'msvcr80.dll',
'net',
'pdas.dll']
Means the path is correct ForextConnect.dll is present and I might do some very simple wrong, but have no clue what.
You can either use Dependency Walker to figure out the correct sequence to manually load the DLLs, or simply add the directory to the system search path:
dllpath = os.path.abspath('ForexConnectAPIx64/bin')
os.environ['PATH'] += os.pathsep + dllpath
myDLL = CDLL('ForexConnect.dll')