I was looking for a method to compile multiple pyx-files using only setup.py file. The solution was found in the official documentation:
ext_modules = [Extension("*", ["*.pyx"])]
setup(ext_modules=cythonize(ext_modules))
This will compile all pyx-files within the current directory and create a shared-object for each single file. However, someone suggested this alternative:
ext_modules = [Extension("*", [file for file in os.listdir("folder_where_your_pyx_files_are")
if file.endswith('.pyx'])]
setup(ext_modules=cythonize(ext_modules))
Which will compile all pyx-files into one shared-object.
However none of the imports are working properly.(e.g. if there imports between the files, none of them will work)
So my question is:
Is there a use case for compiling multiple pyx-file into one extension module ?
Note: I am new to cython and have little knowledge about Extension module.
Related
I have an installed python module with cython extensions. Now I am writing a second (different) cython module that wants to import extensions from the installed cython module. However, it is not able to find the definition files of the first module.
The first module has .../python3.8/site-packages/plexim[version]/EGG-INFO/SOURCES.txt as follows:
setup.py
...
plexsim/models.cpp
plexsim/models.pxd
...
Which looks good as the pxd is packed with the module.
In the other module I want to import the pxd from models.pxd. However, when I try to install my other extensions module it cannot find the extension definition when doing
from plexsim.models cimport *
How do I package the data correctly such that the other module sees the definition from the already installed module?
My setup.py looks a follows
setup(
package_dir = {"" : "imi"
},
namespace_packages = find_namespace_packages (include = ["imi.*"]),
package_data = {"": "*.pxd *.pyx".split(),
ext_modules = cythonize(
exts,
language_level = 3,
compiler_directives = cdirectives,
nthreads = mp.cpu_count(),
),\
Thanks
After hours of debugging I figured the path-issue out. Adding __init__.pxd to the module seemed to have cured the problem. For future reference be mindful of whether setuptools actually find the pxd files.
Calling a file resulting from the concatenation (bash: cat ... >> app.js) of the following three files:
/usr/share/ceylon/1.2.0/repo/ceylon/language/1.2.0/ceylon.language-1.2.0.js
modules/com/example/helloworld/1.0.0/com.example.helloworld-1.0.0-model.js
modules/com/example/helloworld/1.0.0/com.example.helloworld-1.0.0.js
with the command nodejs app.js does nothing. The same when used in a web page. How do have I to call that javascript program so that it runs without using require.js ?
Please give the rules how ceylon modules and the run function and other functions contained within translate to javascript and are to be called.
How can I get one javascript file from compilation of several ceylon modules without concatenating them manually or with require.js?
The above is without using google closure compiler.
Given the size of 1.6 MB of the language module, it makes no sense to run ceylon-js without using google closure compiler.
Compiling "ceylon.language-1.2.0.js" alone with google closure compiler results in a lot of warnings.
java -jar compiler.jar --compilation_level ADVANCED_OPTIMIZATIONS --js /usr/share/ceylon/1.2.0/repo/ceylon/language/1.2.0/ceylon.language-1.2.0.js --js_output_file lib-compiled.js
How can I get rid of those warnings?
In what order do I have to chain together files resulting from ceylon-js with the model file and the language file to compile them in advanced mode with google closure compiler for dead code elimination.
These are 3 questions, really.
A Ceylon module is compiled to a CommonJS module. Concatenating the resulting files won't work because each file is on CommonJS format, which is a big function that returns an object with the exported declarations.
You can compile the modules with the --no-module option to get just the generated code, without it being wrapped in CommonJS format. For the language module, you can copy the file and just delete the first line and the last 5 lines.
I do not yet know how to get rid of the warnings you mention in the second question.
And as for the third question, I would recommend putting the language module first, then the rest of the files. If you have any toplevel declarations with the same name in different modules, you'll have conflicts (only the last declaration will remain), even if they're not shared, since they're all in the same module/unit.
Well, I think require.js can run the compilation of the modules to one file and then run the google-closure-compiler, see: http://www.requirejs.org/docs/optimization.html
I upgraded my Cython to version 0.23.4 and my code now leads to an AttributeError when I put the .pyx and .pxd files in subdirectories of my working directory.
The minimal example containing the error is as follows:
Main python file in working directory:
import pyximport;
import os,sys;
pyximport.install()
sys.path.insert(0, os.getcwd()+'/pxd')
sys.path.insert(0, os.getcwd()+'/pyx')
from X import *
xObj = X(5)
pyx/X.pyx (i.e. in subdirectory pyx/):
cdef class X:
def __init__(self,var):
self.var = var
pxd/X.pxd (i.e. in subdirectory pxd/):
cdef class X:
cdef public int var
Running Main.py gives the following error:
AttributeError: 'X.X' object has no attribute 'var'
Note, the code runs fine if I move the X.pyx and X.pxd files into my working directory. But this is very inconvenient due to a large number of files.
What can I do to get the code running, whilst having the X.pyx and X.pxd in the subdirectories?
Cython expects the .pyx and .pxd files to be in the same directory as each other, (which doesn't have to be your working directory). As it currently stands I think you're importing only "X.pyx", and it isn't realising that "X.pxd" is related to it.
Therefore you could create a directory called "cython_files" (to keep your files out of your working directory), and in that directory you'd put both "X.pxd" and "X.pyx". You could then add "cython_files" to your path, and you wouldn't get any attribute errors.
Better yet, you could add an "__init__.py" to your "cython_files" directory and then it's treated as a Python module and you can do from cython_files.X import X, and not have to add anything to your path.
Edit for clarity: #romenic's answer identifies the same issue, but suggests a slightly different solution to it (which I suspect works) - this answer argues that OP really shouldn't organize their files in the way they do, rather than trying to work round it.
The issue is that Cython cannot find your header files (.pxd) if you split them into different directories, unless you explicitly import the declarations
from pxd_file cimport class_declaration
assuming that the path to pxd_file is visible to the compiler. In the case of example you have given this simply translates to
from X cimport X
Have a look at these Cython's internal examples.
With the Less plugin I got a nice Less -> Css compilation whenever I save the file. How to get the same behavior with Babel to have ES6 code transpiled to ES5? Thanks
Edited after #lukas-kabrt suggestion:
The less2css Sublime plugin, utilizes the on_post_save() event of the Sublime API, to listen for a file Save and then trigger the code that will compile the .less file automatically. Something similar can be accomplished for Babel ES6-to-ES5 compilation, by using this event in a plugin, and linking it to an external command that will auto compile the current file using Babel. This is a simple plugin for ST2, written in Python, that accomplishes this:
import sublime, sublime_plugin,os
from os.path import dirname, realpath
class BuildonSave(sublime_plugin.EventListener):
def on_post_save(self, view):
es6File = view.file_name()
filename, file_extension = os.path.splitext(es6File)
if file_extension == ".es6":
view.window().run_command('exec',{'cmd': ["/usr/local/bin/babel", es6File, "-o", filename+".js", "--source-maps", "inline"] })
Note: This plugin looks for .es6 suffixed files and compiles them to the appropriate .js file, e.g. myfile.es6, will be transpiled into an ES5 myfile.js file.
The cmd location /usr/local/bin/babel can be obtained on your system by running the which babel command on the terminal, if you're on Mac OS X.
Source code can also be found here: https://gist.github.com/kostasx/1d55c62edcee88375fc8
You can checkout the Sublime Plugin API documentation, if you want to mingle with the code above a little more.
When you run from the command line
$ cython -a mycode.pyx
you get a really nice HTML "annotation" file with yellow shading to indicate slow python operations vs fast C operations. You also get this same HTML file as a link every time you compile Cython code in Sage. My questions are: (1) Can I get this HTML file if I'm compiling using distutils? (2) Can I get this HTML file if I'm compiling using pyximport? Thanks!!
Thanks to larsmans's comment and the Cython email list, I now have many satisfying options to generate the "annotate" HTML file without leaving IPython:
(1) Use subprocess...
import subprocess
subprocess.call(["cython","-a","myfilename.pyx"])
(2) Turn on the global annotate flag in Cython myself, before compiling:
import Cython.Compiler.Options
Cython.Compiler.Options.annotate = True
(3) Pass annotate=True into cythonize() [when using the distutils compilation method].
It seems that pyximport does not have its own direct option for turning on annotation.