I am trying to create namespace packages for a modular project.
The core system has the following packages
homie
homie.api
homie.events
homie.mods
I want to install the respective modules as sub-packages of homie.mods.
Therefor I provided the respective homie.__init__.py and homie.mods.__init__.py with the following content:
from pkg_resources import declare_namespace
declare_namespace(__name__)
My testing module is structured as follows:
homie
homie/__init__.py
homie/mods
homie/mods/__init__.py
homie/mods/test
homie/mods/test/__init__.py
Where homie/__init__.py and homie/mods/__init__.py contain:
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
and homie/mods/test/__init__.py contains
print('Works!')
The core package is being set up with the following setup.py:
from setuptools import setup
<snip>
setup(
name='homie',
version=version,
author=author,
author_email=author_email,
install_requires=[
'docopt',
'setproctitle',
'homeinfo',
'homeinfo-crm',
'openimmo',
'zmq'
],
packages=[
'homie',
'homie.api',
'homie.events',
'homie.mods'
],
namespace_packages=['homie', 'homie.mods'],
<snip>
which I install first.
Sencondly I install the module using its setup.py:
#! /usr/bin/env python3
from setuptools import setup
setup(name='homie-test',
version=version,
author=author,
author_email=email,
install_requires=['homie'],
packages=['homie.mods.test'],
namespace_packages=['homie', 'homie.mods'],
license=open('LICENSE.txt').read()
)
When trying to import stuff, I get the following:
>>> from homie import api, events, mods
>>> from homie.mods import test
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'test'
>>>
What am I missing?
Related
For example, I have a python module, and the structure is
moduleA
__init__.py
functionA.py
moduleB
__init__.py
functionB.py
setup.py
How can I package the moduleA using cython?
What I have tested:
test
__init__.py
foo.py
setup.py
#----------------------------------------------------
## __init__.py
from foo import *
#---------------------------------------------------
## foo.py
def hello():
print("hello")
# ----------------------------------------------------------
## setup.py
from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension('test.foo', ['test/foo.py']),
]
setup(
name='Test',
ext_modules=ext_modules,
include_dirs=["test"],
cmdclass={'build_ext': build_ext},
)
After python setup.py build_ext, a new fold appear:
build
lib.win-amd64-3.7
test
foo.cp37-win_amd64.pyd
temp.win-amd64-3.7
Release
test
foo.cp37-win_amd64.exp
foo.cp37-win_amd64.lib
foo.obj
Then, cd ./build/lib.win-amd64-3.7, and open python interpreter. import test is OK, but I can use hello() by test.hello(), test.foo.hello(), test.foo().
What's wrong with my process?
The environment is:
win 10
python 3.7
cython 0.29.26
Update----------------------------------------------------
The above problem can be solved by `python setup.py build_ext --inplace'. Thanks for DavidW's suggestion.
My next question is how to pack a python module?
For example, I have a module as:
App
ModuleA
__init__.py
funcA.py
ModuleB
__init__.py
funcB.py
__init__.py
setup.py
I can pack the aoove module by the following setup.py:
from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension('App.ModuleA.funcA', ['App/ModuleA/funcA.py']),
Extension('App.ModuleB.funcB', ['App/ModuleB/funcB.py']),
]
setup(
name='App',
ext_modules=ext_modules,
include_dirs=["App"],
cmdclass={'build_ext': build_ext},
)
I manually collect the ext_modules. But, if the module has many file, how can I automatically collect the ext_modules?
I can only write the following stranger code:
name = 'App.'
moduleName = os.listdir('./App')
for mName in moduleName:
name += mName
...
Is there any other elegant methods?
I am trying to use Cython to convert my python script to a .c file.
I have the following files:
test_cython.pyx
print("Hello Cython")
setup.py
import distutils.core
import Cython.Build
distutils.core.setup(
ext_modules = Cython.Build.cythonize("test_cython.pyx"))
This creates the "test_cython.c" using
python setup.py build_ext --inplace
like I want it to, however when I subsequently try to import the test_cython it says that there is no module named test_cython. Any help would be appreciated!
You forgot to name your package!
import distutils.core
import Cython.Build
distutils.core.setup(
name='test_cython',
ext_modules = Cython.Build.cythonize("test_cython.pyx"))
I'm trying to set up a cython setup that will compile
a python source code to an executable (It should embed the main method within) - currently I have managed to set it up as an importable module but not as a standalone executable.
I saw that there is a compiler option Options.embed that should handle this. (In this post it said that it should be set to the function that the interpreter should call - main)
This is the module code:
def main():
print('Cython Demo')
if __name__ == '__main__':
main()
This is the setup "compile.py" code
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
from Cython.Build import cythonize
from Cython.Compiler import Options
Options.docstrings = False
Options.emit_code_comments = False
Options.embed = "main"
ext_modules = cythonize([
Extension("cython_demo.mymod.moduleA",["/home/myuser/PycharmProjects/cython_demo/mymod/moduleA.py"])],
compiler_directives=dict(always_allow_keywords=True,language_level = '3'))
setup(
name = 'My Program Name',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
Unfortunately after compiling the python code and trying to run the executable by calling:
./moduleA.cpython-36m-x86_64-linux-gnu.so
i get the segmentation error.
Segmentation fault (core dumped)
I saw that the main function is there by running grep "int main" on the file. What may be the problem?
When I'm importing the module from somewhere else and running main directly - it works:
import moduleA
moduleA.main()
Thanks!
I want to pass some extra options to the Cython compiler by using extra_compile_args.
My setup.py:
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'Test app',
ext_modules = cythonize("test.pyx", language="c++", extra_compile_args=["-O3"]),
)
However, when I run python setup.py build_ext --inplace, I get the following warning:
UserWarning: got unknown compilation option, please remove: extra_compile_args
Question: How does one use extra_compile_args correctly?
I use Cython 0.23.4 under Ubuntu 14.04.3.
Use the more traditional way without cythonize to supply extra compiler options:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
name = 'Test app',
ext_modules=[
Extension('test',
sources=['test.pyx'],
extra_compile_args=['-O3'],
language='c++')
],
cmdclass = {'build_ext': build_ext}
)
Mike Muller's answer works, but builds extensions in the current directory, not next to the .pyx file when --inplace is given as in:
python3 setup.py build_ext --inplace
So my workaround is to compose a CFLAGS string and override the env variable:
os.environ['CFLAGS'] = '-O3 -Wall -std=c++11 -I"some/custom/paths"'
setup(ext_modules = cythonize(src_list_pyx, language = 'c++'))
There's another way to do this, I found it to be the best one from other two presented because with this you still can use all regular cythonize arguments in usual way:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
name="Test app",
ext_modules=cythonize(
Extension(
"test_ext", ["test.pyx"],
extra_compile_args=["-O3"],
language="c++",
),
),
)
sfml.pxd:
cdef extern from "SFML/Window.hpp" namespace "sf":
cdef cppclass VideoMode:
VideoMode(unsigned int, unsigned int) except +
cdef cppclass Window:
Window(VideoMode, String) except +
void display()
display.pyx:
cimport sfml
cdef class Window:
cdef sfml.Window* _this
def __cinit__(self, unsigned int width, unsigned int height):
self._this = new sfml.Window(sfml.VideoMode(width, height), "title")
def __dealloc__(self):
del self._this
def display(self):
self._this.display()
setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("display", ["display.pyx"],
language='c++',
libraries=["sfml-system", "sfml-window"])
]
)
The error when running python setup.py build:
running build
running build_ext
cythoning display.pyx to display.cpp
Traceback (most recent call last):
File "setup.py", line 10, in <module>
libraries=["sfml-system", "sfml-window"])
File "/usr/lib/python3.3/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.3/distutils/dist.py", line 917, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/lib/python3.3/distutils/command/build.py", line 126, in run
self.run_command(cmd_name)
File "/usr/lib/python3.3/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.3/distutils/dist.py", line 936, in run_command
cmd_obj.run()
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 163, in run
_build_ext.build_ext.run(self)
File "/usr/lib/python3.3/distutils/command/build_ext.py", line 354, in run
self.build_extensions()
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 170, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/usr/lib/python3.3/site-packages/Cython/Distutils/build_ext.py", line 317, in cython_sources
full_module_name=module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 608, in compile
return compile_single(source, options, full_module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 549, in compile_single
return run_pipeline(source, options, full_module_name)
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Main.py", line 386, in run_pipeline
from . import Pipeline
File "/usr/lib/python3.3/site-packages/Cython/Compiler/Pipeline.py", line 7, in <module>
from .Visitor import CythonTransform
File "Visitor.py", line 10, in init Cython.Compiler.Visitor (/build/src/Cython-0.19/Cython/Compiler/Visitor.c:15987)
ImportError: No module named 'ExprNodes'
Apparently, it can't find something called 'ExprNodes', but I don't think that my Cython installation is broken, because I managed to successfully wrap a different C++ library some time ago, and I didn't run into this problem.
I'm using Cython 0.19.
I would appreciate any help/insight that you could offer.
Thanks.
Looking more closely at the traceback, I see that Cython fails inside it's own compiled code. It may be a bug indeed, sorry for missing it the first time.
What can you do:
Create a clean virtualenv, install Cython there and check if it works. (Version 0.19.1 is the latest).
Create another virtualenv, but this time install Cython using python setup.py install --no-cython-compile.
If either of these fails, please post your detailed configuration (linux distro and version, python version, gcc version, etc.) to the cython-devel mailing list.
BTW does your old successful project still compile?