I have some algorythms that are implemented in Python but I need to make them portable to access them with Java JNI or directly from C. They don't have scary dependencies but they are updated on a release schedule so reimplementation is not an option. Jython has let me down because the algorythms are in Python 3. I'm trying to use Cython, which looks promising on Windows 10. I have read about on other threads and I have Visual Studio Build Tools for 16.11 (1929) which is the version my Python is also compiled with per this question.
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("C:\\Users\\stephen\\pyp\\VS.pyx")
)
and I invoke this with
python setup.py build_ext --inplace --compiler=msvc
It seems to compile but fails at the linker step:
(pyp) C:\Users\stephen\pyp>python setup.py build_ext --inplace
running build_ext
building 'VS' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\stephen\pyp\include -IC:\Users\stephen\AppData\Local\Programs\Python\Python310\include -IC:\Users\stephen\AppData\Local\Programs\Python\Python310\Include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /TcC:\Users\stephen\pyp\VS.c /Fobuild\temp.win32-3.10\Release\Users\stephen\pyp\VS.obj
VS.c
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\stephen\pyp\libs /LIBPATH:C:\Users\stephen\AppData\Local\Programs\Python\Python310\libs /LIBPATH:C:\Users\stephen\AppData\Local\Programs\Python\Python310 /LIBPATH:C:\Users\stephen\pyp\PCbuild\win32 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\lib\x86 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\lib\x86 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86 /EXPORT:PyInit_VS build\temp.win32-3.10\Release\Users\stephen\pyp\VS.obj /OUT:build\lib.win32-3.10\VS.cp310-win_amd64.pyd /IMPLIB:build\temp.win32-3.10\Release\Users\stephen\pyp\VS.cp310-win_amd64.lib
Creating library build\temp.win32-3.10\Release\Users\stephen\pyp\VS.cp310-win_amd64.lib and object build\temp.win32-3.10\Release\Users\stephen\pyp\VS.cp310-win_amd64.exp
VS.obj : error LNK2001: unresolved external symbol __imp__PyBaseObject_Type
VS.obj : error LNK2001: unresolved external symbol __imp__PyObject_SetAttr
VS.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_InternFromString
VS.obj : error LNK2001: unresolved external symbol __imp__PyCFunction_Type
VS.obj : error LNK2001: unresolved external symbol __imp__PyOS_snprintf
VS.obj : error LNK2001: unresolved external symbol __imp__PyThreadState_Get
VS.obj : error LNK2001: unresolved external symbol __imp__PyExc_Exception
<...>
VS.obj : error LNK2001: unresolved external symbol __imp___Py_TrueStruct
VS.obj : error LNK2001: unresolved external symbol __imp__PyExc_SystemError
VS.obj : error LNK2001: unresolved external symbol __imp__PyObject_Call
VS.obj : error LNK2001: unresolved external symbol __imp__PyUnicode_FromStringAndSize
VS.obj : error LNK2001: unresolved external symbol __imp___PyObject_GetDictPtr
build\lib.win32-3.10\VS.cp310-win_amd64.pyd : fatal error LNK1120: 64 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x86\\link.exe' failed with exit code 1120
Seems there's a library missing here that contains Python symbols but my knowledge of the windows software ecosystem is letting me down, any help appreciated.
Related
I am using cython to convert py file to pyd file.
My test code is:
# funcA.py
class Window:
def exec(self):
pass
pass
And the setup.py file is:
#setup.py
from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension('funcA', ['funcA.py']),
]
setup(
name='App',
ext_modules=ext_modules,
include_dirs=["App"],
cmdclass={'build_ext': build_ext},
)
After python setup.py build_ext --inplace, a bug is reported:
Error compiling Cython file:
------------------------------------------------------------
...
class Window:
def exec(self):
^
------------------------------------------------------------
funcA.py:4:8: Expected an identifier
building 'funcA' extension
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IApp -ID:\Anaconda3\envs\work\include -ID:\Anaconda3\envs\work\Includ
e "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include" "-IC:\Prog
ram Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x
86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /TcfuncA.c /Fobuild\temp.win-amd64-cpython-39\Release\funcA.obj
funcA.c
funcA.c(1): fatal error C1189: #error: Do not use this file, it is the result of a failed Cython compilation.
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
But, if I replace the funcA.py to:
#new funcA.py
class Window:
pass
Everything is OK.
Why exec function would couse this bug?
Any suggestion is appreciated~~~
------------- update --------------------------------
Thanks #DavidW's suggestion.
The solution is adding #cython: language_level=3 on the top of script.
Depending on what Cython version you're using (you don't say...) Cython defaults to Python 2 behaviour. In Python 2 exec was a keyword, so cannot be used as a function name.
Either:
pick a different function name,
put Cython in Python 3-like mode by setting language_level to 3. Bear in mind this may change some other things (e.g. print, the scope of list comprehensions, ...)
Use Cython 3 alpha version where it does default to Python 3 semantics by default.
Basically, I have a cython code where i want to call C's inline assembly. I tried below:
cdef extern from *:
"""
#include <stdio.h>
void print_endln(){
asm("nop");
}
"""
void print_endln()
print_endln()
But I get below error:
ids.obj : error LNK2001: unresolved external symbol asm
Location\iaa.cp39-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe' failed with exit code 1120
My OS is windows 10 x64, python version Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec 7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32
I compile the code as python setup.py build_ext --inplace && python program.py
Is there something wrong I am doing?
I try to cythonize a .py script. It is a PyQt5 gui with a large number of QToolButtons, and a working EventFilter. The c module is built successfully, however, the compilation fails with the following error:
d:\stuff\mapform2a.c(11338) : fatal error C1002: compiler is out of heap space in pass 2
LINK : fatal error LNK1257: code generation failed
error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe' failed with exit status 1257
The compiler comes from Visual Studio 2019. Python 3.5.5 (yes, old, I know, but I have reasons...).
Is there any way to increase the heap space when trying to "cythonize -i script.py" ?
Cython documentation is really not clear on this (for a non-C-expert at least...)
EDIT
The full log is as follows:
C:\temp\MapForm>python setup.py build_ext --inplace Compiling
MapForm2A.py because it changed. [1/1] Cythonizing MapForm2A.py
C:\Anaconda3\lib\site-packages\Cython\Compiler\Main.py:369:
FutureWarning: Cython directive 'language_level' not set, using 2 for
now (Py2). This will change in a later release! File:
C:\temp\MapForm\MapForm2A.py tree = Parsing.p_module(s, pxd,
full_module_name) running build_ext building 'MapForm2A' extension
creating build creating build\temp.win-amd64-3.5 creating
build\temp.win-amd64-3.5\Release C:\Program Files (x86)\Microsoft
Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL
/DNDEBUG /MD -IC:\Anaconda3\include -IC:\Anaconda3\include
"-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE"
"-IC:\Program Files (x86)\Microsoft Visual Studio
14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows
Kits\10\include\10.0.18362.0\winrt" /TcMapForm2A.c
/Fobuild\temp.win-amd64-3.5\Release\MapForm2A.obj MapForm2A.c creating
C:\temp\MapForm\build\lib.win-amd64-3.5 C:\Program Files
(x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo
/INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO
/LIBPATH:C:\Anaconda3\libs /LIBPATH:C:\Anaconda3\PCbuild\amd64
"/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio
14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files
(x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64" "/LIBPATH:C:\Program
Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64"
/EXPORT:PyInit_MapForm2A
build\temp.win-amd64-3.5\Release\MapForm2A.obj
/OUT:build\lib.win-amd64-3.5\MapForm2A.cp35-win_amd64.pyd
/IMPLIB:build\temp.win-amd64-3.5\Release\MapForm2A.cp35-win_amd64.lib
MapForm2A.obj : warning LNK4197: export 'PyInit_MapForm2A' specified
multiple times; using first specification Creating library
build\temp.win-amd64-3.5\Release\MapForm2A.cp35-win_amd64.lib and
object build\temp.win-amd64-3.5\Release\MapForm2A.cp35-win_amd64.exp
Generating code c:\temp\mapform\mapform2a.c(7545) : fatal error C1002:
compiler is out of heap space in pass 2 LINK : fatal error LNK1257:
code generation failed error: command 'C:\Program Files
(x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe'
failed with exit status 1257
I can only add that the process gets stuck on the "Generating code" message for about 90 seconds before raising the C1002 exception.
The setup file is rather standard:
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("MapForm2A.py")
)
The module is pure PyQt5 (pyuic5 output) with no other dependencies and works fine if interpreted directly with no Cython.
EDIT: SOLUTION (Maybe someone will need it).
Thanks to #DavidW (discussion in the comments below).
Setup.py has to be modified in the following way:
from distutils import _msvccompiler
_msvccompiler.PLAT_TO_VCVARS['win-amd64'] = 'amd64'
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("MapForm2A.py"),
)
The first two lines force 64-bit toolchain.
For the purposes of giving a little more explanation to something solved in the comments: the basic problem looks to be that you're compiling something large and complicated and MSVC has run out of memory at the linking step.
Microsoft has a page about this error which suggests a number of options, with the main one being to use a 64-bit compiler. (Note that this is independent of whether you're compiling a 32-bit or 64-bit module - it's merely the choice of compiler executable)
When compiling Python extension modules (especially with setup.py) the compiler setup is typically selected by distutils. Unfortunately, it looks like distutils chooses to force a 32-bit compiler (see https://github.com/python/cpython/blob/e488e300f5c01289c10906c2e53a8e43d6de32d8/Lib/distutils/_msvccompiler.py#L160).
My suggestion was to dig into the distutils internals at the top of setup.py (before any real setup takes place) to override this setting
from distutils import _msvccompiler
_msvccompiler.PLAT_TO_VCVARS['win-amd64'] = 'amd64'
Essentially all you're really doing is passing the option amd64 to the vcvarsall.bat script that microsoft supply to set up their compiler, thus getting a 64-bit compiler to build a 64-bit extension.
Below is my small piece of code :
#include <my_global.h>
#include <mysql.h>
int main(int argc, char **argv)
{
printf("MySQL client version: %s\n", mysql_get_client_info());
exit(0);
}
As evident, the only objective is to get this small program up and running, so nothing much to boast about.But to my dismay it became a headache for me. First I got an error which I posted as this SO question which I fortunately resolved. Now I am getting a stream of errors as
described below :
-------------- Build: Debug in mysqlconntest (compiler: GNU GCC Compiler)---------------
gcc.exe -Wall -fexceptions -g -I"C:\Program Files (x86)\MySQL\MySQL Server 5.7\include" -I"C:\Program Files (x86)\MySQL\MySQL Server 5.7\include" -c C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp -o obj\Debug\main.o
In file included from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:2:0:
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:211:19: error: conflicting declaration 'typedef int mode_t'
In file included from c:\program files (x86)\codeblocks\mingw\bin\../lib/gcc/mingw32/4.7.1/../../../../include/io.h:20:0,
from C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:32,
from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:2:
c:\program files (x86)\codeblocks\mingw\bin\../lib/gcc/mingw32/4.7.1/../../../../include/sys/types.h:99:17: error: 'mode_t' has a previous declaration as 'typedef _mode_t mode_t'
In file included from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:2:0:
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:212:19: error: conflicting declaration 'typedef SSIZE_T ssize_t'
In file included from c:\program files (x86)\codeblocks\mingw\bin\../lib/gcc/mingw32/4.7.1/../../../../include/io.h:20:0,
from C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:32,
from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:2:
c:\program files (x86)\codeblocks\mingw\bin\../lib/gcc/mingw32/4.7.1/../../../../include/sys/types.h:118:18: error: 'ssize_t' has a previous declaration as 'typedef _ssize_t ssize_t'
In file included from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:2:0:
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h: In function 'tm* localtime_r(const time_t*, tm*)':
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:644:25: error: 'localtime_s' was not declared in this scope
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h: In function 'tm* gmtime_r(const time_t*, tm*)':
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/my_global.h:650:22: error: 'gmtime_s' was not declared in this scope
In file included from C:\Users\Sajith\Documents\CodeBlocks\mysqlconntest\main.cpp:3:0:
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h: At global scope:
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:252:3: error: 'NET' does not name a type
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:277:25: error: 'SCRAMBLE_LENGTH' was not declared in this scope
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:415:48: error: use of enum 'enum_session_state_type' without previous declaration
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:419:47: error: use of enum 'enum_session_state_type' without previous declaration
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:441:45: error: use of enum 'mysql_enum_shutdown_level' without previous declaration
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:448:12: error: use of enum 'enum_mysql_set_option' without previous declaration
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:580:28: error: 'NET' has not been declared
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:634:21: error: 'MYSQL_ERRMSG_SIZE' was not declared in this scope
C:\Program Files (x86)\MySQL\MySQL Server 5.7\include/mysql.h:635:19: error: 'SQLSTATE_LENGTH' was not declared in this scope
Process terminated with status 1 (0 minute(s), 0 second(s))
15 error(s), 0 warning(s) (0 minute(s), 0 second(s))
Below are the things which I have done.
Went through this tutorial to create a libmysql.a file.
Verified my include directoried and linker libraries.
Linkers
Include Directories
This made me think that I didn't do my homework properly before I jumped into MySQL API programming. What am I missing here? What are the very basic things that I should ensure before I compile a MYSQL application. I was able to compile this program in my Linux machine successfuly. So I guess that this is Windows specific problem. Any help appreciated.
Note: I am using Code::Blocks (with MingGW toolchain) for an IDE and Windows 8.1 64Bit for an OS. MySQL is 32 bit and my application too is 32 bit.
With MinGW, you didn't need to create a glue. You could link the lib and dll files from MySQL to your C files. You can do it by adding the directory of the library files and the directory of the MySQL header files in your compilation. You could do it by gcc 'name.c' -o 'output.exe' -L "*Lib Directory*" -I "*Header Files Directory*"
so for no reason at all, today when I tried to save my css I recieved the following error:
[Error 2] The system cannot find the file specified
[cmd: [u'node', u'C:\\Users\\Weber\\AppData\\Roaming\\Sublime Text 2\\Packages/JSLint/linter.js', u'--predef', u"['angular', 'document', '$', '_', 'JQuery', 'FB']", u'--sloppy', u'--indent', u'2', u'--node', u'--nomen', u'--vars', u'--plusplus', u'--stupid', u'--todo', u'C:\\Users\\Weber\\AppData\\Roaming\\Sublime Text 2\\Projects\\Blog\\stylesheet.css']]
[dir: C:\Users\Weber\AppData\Roaming\Sublime Text 2\Projects\Blog]
[path: C:\Python27\;C:\Program Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\AMD APP\bin\x86_64;C:\Program Files (x86)\AMD APP\bin\x86;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files (x86)\Panda Security\WaAgent\Common;C:\Program Files (x86)\QuickTime\QTSystem\;C:\cygnus\cygwin-b20\H-i586-cygwin32\bin;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;c:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;c:\Program Files\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files\Java\jdk1.7.0_51\bin]
[Finished]
And so yesterday it worked, but after doing some python and c++ stuff, I came back to HTML/CSS and I get this error with CSS, not HTML. HTML saves fine.
Edit: when I save it saves the file but the console pops up saying this error, which gets greatly annoying.
Thanks.
The JSLint plugin is upset about some configuration. You can either delete the plug-in and reinstall it, or update the settings:
Type ctrl + P to open packages menu and select 'list packages', then pick JSlint. Look for the config file and edit it.