When you run from the command line
$ cython -a mycode.pyx
you get a really nice HTML "annotation" file with yellow shading to indicate slow python operations vs fast C operations. You also get this same HTML file as a link every time you compile Cython code in Sage. My questions are: (1) Can I get this HTML file if I'm compiling using distutils? (2) Can I get this HTML file if I'm compiling using pyximport? Thanks!!
Thanks to larsmans's comment and the Cython email list, I now have many satisfying options to generate the "annotate" HTML file without leaving IPython:
(1) Use subprocess...
import subprocess
subprocess.call(["cython","-a","myfilename.pyx"])
(2) Turn on the global annotate flag in Cython myself, before compiling:
import Cython.Compiler.Options
Cython.Compiler.Options.annotate = True
(3) Pass annotate=True into cythonize() [when using the distutils compilation method].
It seems that pyximport does not have its own direct option for turning on annotation.
Related
I have a cython source file in which I would like to import the local python module.
This cython source file is translated using cython (python3 syntax) into a c++ source, which in turn is compiled into a library, and then used from the main C++ program.
When the main program is being executed the import of the local python module fails because the location of thereof module is not known to the executed code. I tried using python3's local import features in my pyx file but to no avail.
The only working solution I came up with (and the most obvious one) is to update python's module search path using sys.path.append. The problem is that I have to hardcode this path, which is ugly.
I tried to find any hints if it is possible to retrieve within cython code location of the source file (I could derive an absolute path from it) but without success. Usual pythonic ways to do that fail - for instance, one of the reasons is that __file__ evaluates to built-in, and retrieval of the absolute path at runtime gives the path where the executable is being run.
Sidenote: one of the searches I did was by querying GitHub search engine for occurrences of sys.path.append in cython files. Interestingly, all results either have paths hardcoded or they are not related to the location of the cython source file within the file system.
So my question is if it is possible within cython code to reliably retrieve the location of its source file?
Disclaimer: I could imagine instrumenting the build system to pass preprocessor variable set to the path in question while building the C++ file derived from the cython one, and then access this within the code, but this looks like an overkill.
Example:
bulba.py
def fn():
print('blah')
bulbulator.pyx
# tag: cpp
# tag: py3only
import sys
sys.path.append('/absolute_path_to_folder_with_bulba_py') # <-- this is the key part. I'd like to replace the hardcoded path with something better
from bulba import fn
fn()
bulbulator.pyx is translated into cpp with:
cython -3 --cplus bulbulator.pyx
lib_wrapper.cpp (this library, and executable which links against it, have a location different than that of py/pyx source code and its translated c++ part)
// import headers generated by cython
#include "bulbulator_api.h"
#include "bulbulator.h"
// global initialization of the cythonized part
__attribute__((constructor))
static void
__library_init()
{
if (int err = PyImport_AppendInittab("bulbulator", PyInit_bulbulator); err != 0)
{
std::fprintf(stderr, "PyImport_AppendInittab(bulbulator) failed with status code=%d\n", err);
std::exit(1);
}
Py_Initialize();
if (import_bulbulator() == -1) // <-- here it fails if I comment out sys.path.append, because bulbulator needs to know the location of bulba.py
{
PyErr_Print();
}
}
I would rather put the bulba.py next to the exe, but it is also possible to back in an absolute path into the Cython-extension, using for example a compile time environment variable (see cython --help for more details), let's call it ADDITIONAL_SYS_PATH:
import sys
sys.path.append(ADDITIONAL_SYS_PATH)
from bulba import fn
fn()
And now running Cython via:
cython -3 --cplus -E ADDITIONAL_SYS_PATH="the path to dir"
will set the right value to ADDITIONAL_SYS_PATH.
Cython saves the name of the pyx-file in the resulting cpp (for example for run-time error reporting), but this name doesn't include the full path, so we need to provide the path manually.
Calling a file resulting from the concatenation (bash: cat ... >> app.js) of the following three files:
/usr/share/ceylon/1.2.0/repo/ceylon/language/1.2.0/ceylon.language-1.2.0.js
modules/com/example/helloworld/1.0.0/com.example.helloworld-1.0.0-model.js
modules/com/example/helloworld/1.0.0/com.example.helloworld-1.0.0.js
with the command nodejs app.js does nothing. The same when used in a web page. How do have I to call that javascript program so that it runs without using require.js ?
Please give the rules how ceylon modules and the run function and other functions contained within translate to javascript and are to be called.
How can I get one javascript file from compilation of several ceylon modules without concatenating them manually or with require.js?
The above is without using google closure compiler.
Given the size of 1.6 MB of the language module, it makes no sense to run ceylon-js without using google closure compiler.
Compiling "ceylon.language-1.2.0.js" alone with google closure compiler results in a lot of warnings.
java -jar compiler.jar --compilation_level ADVANCED_OPTIMIZATIONS --js /usr/share/ceylon/1.2.0/repo/ceylon/language/1.2.0/ceylon.language-1.2.0.js --js_output_file lib-compiled.js
How can I get rid of those warnings?
In what order do I have to chain together files resulting from ceylon-js with the model file and the language file to compile them in advanced mode with google closure compiler for dead code elimination.
These are 3 questions, really.
A Ceylon module is compiled to a CommonJS module. Concatenating the resulting files won't work because each file is on CommonJS format, which is a big function that returns an object with the exported declarations.
You can compile the modules with the --no-module option to get just the generated code, without it being wrapped in CommonJS format. For the language module, you can copy the file and just delete the first line and the last 5 lines.
I do not yet know how to get rid of the warnings you mention in the second question.
And as for the third question, I would recommend putting the language module first, then the rest of the files. If you have any toplevel declarations with the same name in different modules, you'll have conflicts (only the last declaration will remain), even if they're not shared, since they're all in the same module/unit.
Well, I think require.js can run the compilation of the modules to one file and then run the google-closure-compiler, see: http://www.requirejs.org/docs/optimization.html
In tcl how does one find out the path of the package loaded?
% tclsh
% package require csv
I want to find out the path from which csv was loaded.
In python, one can find the path of a module using
>>> import os
>>> print os.__file__
'/a/b/python2.2.1/linux26_x86_64/lib/python2.2/os.pyc'
I am looking for a similar command in tcl
It's not that simple: a package in Tcl appears to be a more abstract thing than that in Python.
First, there are two kinds of packages: "classic" and "modules" which have different underlying mechanisms for finding what to load in response to the package require ... command.
Next, both kinds of packages are able to do whatever they wish to provide their functionality. It means they can be (but not limited to):
Pure Tcl packages, source'ing just one Tcl file or any number of files.
Packages implemented in C or another compiled language, which are in the form of dynamic library which gets loaded when the package is required.
A combination of the above, when there's a C library and a layer of Tcl code around it (usually providing helper/convenience commands).
Hence the question per se has little sense as only modules are represented by exactly one self-contained file but "classic" packages are free to implement themselves as they see fit.
On the other hand, each package normally provides (using one way or another) certain information to the package subsystem which can be retreived (and parsed) using the package ifneeded command. For instance, on my Windows system with ActiveState Tcl 8.5.x, I have:
% package require csv
0.7.2
% package ifneeded csv 0.7.2
package provide csv 0.7.2;source -encoding utf-8 {C:/Program Files/Tcl/lib/teapot/package/tcl/teapot/tcl8/8.3/csv-0.7.2.tm}
Note that what package ifneeded returns is just a Tcl script which is meant to be evaluated to get the package loaded, so parsing of this information is bound to be inherently ad-hoc and fragile.
For Tcl packages you can view list of all loadedable path dirs by command:
join $::auto_path \n
This manual addresses auto_path and other loadable library variables: https://www.systutorials.com/docs/linux/man/n-auto_path/
New or missing loadable package search directory can be added within tclsh:
lappend auto_path /new_directoty
I am trying to use SWIG to generate wrappers for some of my C++ function calls.
Also, I am trying to do build my own TCL shell so I need to static link the generated SWIG libraries. I have my own main function with a Tcl_AppInit call where I do some prior setup.
To do this what function should I include in my program's Tcl_AppInit call? I found that SWIG_init is not the right function. I even tried Cell_Init where cell is the name of the class in my code, but that doesn't help either.
How do I static link SWIG object files with my own main function and Tcl_Appinit call?
Currently when I use the following command to link my executabel I get the following error:
g++ -o bin/icde src/core/*.o src/read/*.o src/swig/*.o src/icde/*.o -ltk -ltcl
I get the following error:
src/icde/main.o: In function `AppInit(Tcl_Interp*)':
main.cpp:(.text+0xa9): undefined reference to `Cell_Init(Tcl_Interp*)'
collect2: ld returned 1 exit status
I checked the src/swig/cell.o file which has the Cell_Init function or not using objdump:
~> objdump -d src/swig/cell.o | grep Cell_Init
00006461 <Cell_Init>:
646c: 75 0a jne 6478 <Cell_Init+0x17>
I am not sure if I am doing something wrong while linking.
------------------- UPDATE ----------------------------
I found that including the swig/swig.cxx file directly in the main file which calls the Tcl_AppInit function resolves the linking issue. Is there a reason for this.
Isn't it possible to create and seprately link the swig file and the file with the main function?
In general, with SWIG you'll end up with a bunch of generated source files that you compile. The normal thing you do then is package them up into a shared library (with appropriate bound dependencies on other shared libraries) that can be imported into a Tcl runtime with the load command.
But you don't want that this time. Instead, you want the object files that you would use to make that shared lib, and you want to include them in the instructions to build an executable along with the object file that holds your main and Tcl_AppInit. You also need to make sure that when linking your main executable that you make it dependent on those external shared libraries; executable building requires that you satisfy all dependencies and make all symbols be bound to their definitions. (You can use a static library to make this easier: it combines a bunch of object files into one file. There's very little difference to just using the object files from it though; in particular, static libraries aren't bound to their dependencies.)
Finally, you do want to include a call to Cell_Init in your Tcl_AppInit. That's the right place to put it (well, as long as you're not arranging for the package to be loaded into sub-interpreters). If it was failing before, that was because you'd got your linking wrong. (Tip: linkers work best when objects and libraries on the link line only depend on things later on the link line. Getting the link order right is a bit of a black art when you've got a complex build!)
I am trying to use Memchan package in my application. I was able to compile and link it statically. But unfortunately I don't know how to load this package in my application.
% rs
Internal error detected during start: can't find package Memchan
can't find package Memchan
while executing
"package require Memchan"
I traced this to the pkgIndex.tcl in Memchan2.3 directory:
% cat pkgIndex.tcl
package ifneeded Memchan 2.3 [list load [file join $dir libMemchan2.3.so]]
I have two questions:
How do I load the statically linked version libMemchan2.3.a ?
Is there a special syntax for calling package require Memchan when one calls a statically linked library?
You've got a statically linked memchan package? Well, that means you need a different package index, whose contents should be this:
package ifneeded Memchan 2.3 {load {} Memchan}
The load has an empty first argument so that statically-linked libraries are considered, but without the filename, a second argument is needed in order to locate the initialization function (which will be Memchan_Init with the above value).
Alternatively, just do this at the start of your script:
load {} Memchan
That will cause the internal package provide to be done anyway, so that any future package require Memchans will just succeed immediately on the grounds that the package is already in use.
[Background info]: As you can see, a package index file is actually very simple: it just provides some instructions to say that if you need a particular package of a particular version, here's a script to make it available. The only real nuance is that the $dir variable describes the location of the package index file while the file is being loaded.