How to make docstrings from cythonized module available to Pylance? - cython

I am working on a fully cython compiled python package, using setuptools and some reStructured Text docstrings to generate the pdf documentation using sphinx-autodoc with good success.
The docstrings are not displayed when loading the package in Jupyter Notebooks (using Pylance language server in VSCode). I did not test but suspect that the issue exists in other IDEs or frontends either. Switching to e.g. Jedi as a language server in VSCode did not help.
Using the help(...) command on the routines/classes yields the expected docstring, so the issue does not seem to be with the cython compilation.
There are quite a few issues about formatting issues popping up when searching for Docstrings and Pylance, the issue never seems to be that they simply can not be retrieved.
Since sphinx (autodoc) does not throw any errors and generates a full-package encompassing documentation set fine, this seems a lot like a problem with the language servers to me.
However, I suspect that there may be any compiler directive or setuptools option that I am unaware of that could help - but I do not know about these.
I do have
compiler_directives={'embedsignature': True}
in the setup.py and since the docstrings do show up when using help(..), I do not think this is an issue.
I have also added
import Cython.Compiler.Options
Cython.Compiler.Options.docstrings = True
even though that is indicated as being the default, and it did not change the behavior at all.
Scavenging through the options of Pylance in VSCode was unfruitful, too - I did not expect much there anyways, since obviously the display of the docstrings does work for plenty (all?) other packages that I looked at. Everything is on defaults.

Related

NativeScript, Code Sharing and different environments

Note: this is not a dupe of this or this other question. Read on: this question is specific to the Code-Sharing template.
I am doing some pretty basic experiments with NativeScript, Angular and the code sharing templates (see: #nativescript/schematics).
Now I am doing some exploration / poc work on how different "build configuration" are supported by the framework. To be clear, I am searching for a simple -and hopefully official- way to have the application use a different version of a specific file (let's call it configuration.ts) based on the current platform (web/ios/android) and environment (development/production/staging?).
Doing the first part is obviously trivial - after all that is the prime purpose of the code sharing schematics. So, different versions of the same file are identified by different extensions. This page explain things pretty simply.
What I don't get as easily is if the framework/template supports any similar convention-based rule that can be used to switch between debug/release (or even better development/staging/production) versions of a file. Think for example of a config.ts file that contains different parameters based on the environment.
I have done some research in the topic, but I was unable to find a conclusive answer:
the old and now retired documentation for the appbuilder platform mentions a (.debug. and .release.) naming convention for files. I don't think this work anymore.
other sources mention passing parameters during the call to tns build / tns run and then fetching them via webpack env variable... See here. This may work, but seems oddly convoluted
third option that gets mentioned is to use hooks to customize the build (or use a plugin that should do the same)
lastly, for some odd reason, the #nativescript/schematics seems to generate a default project that contains two files called environment.ts and environment.prod.ts. I suspect those only work for the web version of the project (read: ng serve) - I wasn't able to get the mobile compiler to recognize files that end with debug.ts, prod.ts or release.ts
While it may be possible that what I am trying to do isn't just supported (yet?), the general confusion an dissenting opinions on the matter make me think I may be missing something.. somewhere.
In case this IS somehow supported, I also wonder how it may integrate with the NativeScript Sidekick app that is often suggested as a tool to ease the build/run process of NativeScript applications (there is no way to specify additional parameters for the tns commands that the Sidekick automates, the only options available are switching between debug/release mode), but this is probably better to be left for another question.
Environment files are not yet supported, passing environment variables from build command could be the viable solution for now.
But of course, you may write your own schematics if you like immediate support for environment files.
I did not look into sharing environment files between web and mobile yet - I do like Manoj's suggestion regarding modifying the schematics, but I'll have to cross that bridge when I get there I guess. I might have an answer to your second question regarding Sidekick. The latest version does support "Webpack" build option which seems to pass the --bundle parameter to tns. The caveat is that this option seems to be more sensitive to typescript errors, even relatively benign ones, so you have to be careful and make sure to fix them all prior to building. In my case I had to lock the version of #types/jasmine in package.json to "2.8.6" in order to avoid some incompatibility between that and the version of typescript that Sidekick's cloud solution is using. Another hint is to check "Clean Build" after npm dependency changes are made. Good luck!

Cannot resolve module 'json'

I'm dealing with a very weird issue. For some reason, since a few days, my application doesn't seem to be loading JSON files anymore inside modules.
The app was created with create-react-app, and has been in development for a while without any issues. I've had a dependency in the project (world-countries) that basically only exports a json file. This dependency is not a recent addition.
So, here's the thing. From one day to another, I get the following error while starting/building the app:
Module not found: Error: Cannot resolve module 'json' in /Users/…
As far as I know, I made no changes that would result in this, and create-react-app's webpack version is still the same.
Requiring a random JSON file in the root of my app goes well, somehow this odd behaviour only seems to apply to json files within node_modules in the app.
Some other points:
By this point, I have reinstalled dependencies and node.js multiple times, checked file permissions, etc. To no avail.
Another dependency that also used a json file internally also had this issue, but I managed to circumvent that.
I've explicitly installed json-loader in case something with webpack got messed up, but this didn't make any difference. In fact, I find it curious that _the missing dependency is "json" and not json-loader.
I have a feeling that this is a weird issue caused by some file limit, as when I run the dev server, I get errors relating to too many open files. My project isn't that big, so this seems weird, and I have a feeling it might have something to do with some npm linked modules I have, but there is no recursion or anything.
I'm still quite perplexed, and have no idea what the issue might be. Any suggestions on how to fix this are very welcome
Some people seemed to have had a related issue regarding importing json files in the past. It wasn't in a dependency but I believe it is related.
I took the time to scaffold a new project, import the world-countries modules and log it, and it was working fine.
I would recommend you to update re-scaffold or update all your dependencies, but most notably react-scripts which is version 0.9.5 for me. My create-react-app-version is also 1.3.0.

Linking to specific glibc version in Cython

I have a Cython extension which I've compiled on Ubuntu 14 and uploaded as an Anaconda package. I'm trying to install the package on another machine which is running Scientific Linux (6?) which ships with an older version of glibc. When I try to import the module I get an error that looks (something like) this:
./myprogram: /lib/libc.so.6: version `GLIBC_2.14' not found (required by ./myprogram)
When I say "something like" - the "myprogram" is actually the .so name of the extension.
From what I understand this error is because I have a newer version of glibc on the build system which has an updated version of the memcpy function.
This page has a good description of the problem, and some rather impractical solutions: http://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc
There is also a much simpler answer proposed here: How can I link to a specific glibc version?
My question is: how to I apply this solution to my Cython extension? Assuming the __asm__ solution works (as given in the second link) what's the best way to get it into the C generated by Cython?
Also, more generally, how to other modules avoid this issue in the first place? For example, I installed and ran a pre-built copy of numpy without any issues.
This turned out to be quite simple.
Create the following header, glibc_fix.h:
__asm__(".symver memcpy,memcpy#GLIBC_2.2.5")
Then include it by using CFLAGS="-include glibc_fix.h". This can be set as an environment variable, or defined in setup.py.
Additionally, it turns out numpy doesn't do anything special in this regard. if I compile it myself it links with the newer version on my system.

Problems converting a C header to D

I'm trying to translating the MySql C connector 6.02 headers to D, but I get some weird crashes.
My guess is I've made some mistakes on translating the structs or function (I'm not very good at C).
I used implib /system on the libmysql.dll to create a lib file.
I couldn't get htod.exe to work. Using -hs (include system files) complained it couldn't find system files.
Coffimplib.exe didn't have an option to prepend _ to exported internal names.
I couldn't find a free version of coff2omf.
mysql.d is the wrapper. I've included the C definition before each wrapped definition to easier spot bugs. The file includes the mysql dll and converted libfile too.
When compiling mytest_fails.d it crashes. mytest_works.d only has an assert, and this makes it work.. Compiling mytest_works with -release makes it crash too.
I've been using dmd 2.051
Download mytest.zip from share1t.com
Update: I've also asked some question regarding this on the D.learn newsgroup, but I don't think anyone has gone through the code.
C Const
Compiler extensions
Connot get htod.exe to work
The weird crashes
stdcall is a Windows function calling convention (very different from the C calling convention). HTOD failed to mark several functions with extern(Windows). This is normal, since HTOD isn't equipped to handle macros (STDCALL is defined as a macro, I can see that from the leftover comments in msyql.d).
Here's an updated mysql.d file:
http://dl.dropbox.com/u/9218759/mysql.d
Now, you need the proper import library in OMF format. I'd generally advise that you do not use implib for this. I've had several problems with it and others have reported having problems using it. Using coffimplib is the way to go. But first, you will need a COFF import library.
If you need it, the Mysql release with the COFF import library can be downloaded from here: http://dev.mysql.com/downloads/mirror.php?id=377977#mirrors (Libraries in DLL form marked for Visual Studio usually come with a COFF import library).
But I'm providing you the translated COFF import library in OMF format here: http://dl.dropbox.com/u/9218759/libmysql.lib
I've tried both of your test cases and they both seem to work fine now. In case of problems, try to check the translated header file again (mysql.d), it's possible that I might have missed to specify all the calling conventions properly.

Inter-module exception name resolution through boost python does not work?

Here is my problem:
I have two C++ modules, A and B, which are built as dynamically-linked libraries. A offers basic math functions, and custom exception types. B is a higher level module that uses A.
B::someFunction() calls a function from A, and tries to catch custom exception A:MyExceptionFromA in order to convert it into a custom type B:MyExceptionFromB (since users of module B do not need to know about the implementation details of A).
Everything works fine as long as I remain in the C++ domain. However, if I expose B::someFunction() in python via boost python, the exception is not caught anymore in the C++ module.
I can catch std::runtime_error, from which A:MyExceptionFromA derives, and call typeid(e).name() to get the retrieve the correct mangled name, so I know the correct exception is thrown. Therefore I suspect that the problem comes from resolving this mangled symbol into the correct exception type.
I have found this link, which explains that "python uses [the insular] model to open extension modules, so that extension module writers don't need to know what symbols other extension modules might be using.". I'm suspecting this is part of the problem/solution, but I do not know enough about symbol resolution to figure out how to solve my problem.
Any ideas?
I found a work-around to my problem. Based on this and link text, I figured out that adding
import sys, dl
sys.setdlopenflags(dl.RTLD_NOW|dl.RTLD_GLOBAL)
before my includes solves the problem, by forcing python to open libraries in immediate, global mode. But I'm still hoping for an alternative solution, if there's one. As mentioned in the first link, I'm suspicious that this could have unforeseen effects (I already know that name clashing could be a problem, and I suspect performance can be affected as well, but are there other side effects?)