c++ - How to import depending cython modules from parent folder in python

Question

Imagine the following folder structure:

/project
  run.py
  /Helper
    Helper.pxd
    Helper.pyx
    setup.py
  /Settings
    Settings.pxd
    Settings.pyx
    setup.py

Helper uses the types Settings and PySettings defined in Settings.pxd. Therefor in Helper.pxd I'm doing:

from Settings cimport Settings, PySettings

In the setup.py in the Helper directory I put

include_path=[".", "../Settings/"]

in the cythonize command. Thus Helper knows about Settings and everything compiles.

In run.py I'd like to import Settings as well as Helper. The Settings import works fine but when doing

import Helper.Helper

I get the following error:

Traceback (most recent call last):
  File "run.py", line 1, in <module>
    import Helper.Helper
  File "../Settings/Settings.pxd", line 6, in init Helper
AttributeError: module 'Settings' has no attribute 'PySettings'

This issue disappears as soon as everything is in the same directory.

For the sake of completeness you'll find the whole code below:

/project/run.py

import Settings.Settings as S
import Helper.Helper as H

settings = S.PySettings()
settings.doSomething()

H.PyMyFunction(settings)

/project/Helper/Helper.pxd

from Settings cimport Settings, PySettings

cdef extern from "../../cppCode/Helper/Helper.h":
  void myFunction(Settings settings)

/project/Helper/Helper.pyx

#distutils: sources = [../../cppCode/Helper/Helper.cpp, ../../cppCode/Settings/Settings.cpp]
#distutils: language = c++

def PyMyFunction(PySettings settings):
  myFunction(settings.c_settings)

/project/Helper/setup.py

from distutils.core import setup
from Cython.Build import cythonize
from setuptools.extension import Extension

extensions = [
  Extension("Helper", ["Helper.pyx"])
]

setup(ext_modules=cythonize(extensions, include_path=[".", "../Settings/"]))

/project/Settings/Settings.pxd

cdef extern from "../../cppCode/Settings/Settings.h":
  cdef cppclass Settings:
    Settings() except +
    void doSomething()

cdef class PySettings:
  cdef Settings c_settings

/project/Settings/Settings.pyx

#distutils: sources = ../../cppCode/Settings/Settings.cpp
#distutils: language = c++

cdef class PySettings:
  def __cinit__(self):
    self.c_settings = Settings()

  def doSomething(self):
    self.c_settings.doSomething()

/project/Settings/setup.py

from distutils.core import setup
from Cython.Build import cythonize
from setuptools.extension import Extension

extensions = [
  Extension("Settings", ["Settings.pyx"])
]

setup(ext_modules=cythonize(extensions))

Answer 1

"include_path" is for C includes. It has no influence on where Python looks for pxd files.

You usually want one setup file, at the top level. A better way of doing it is probably:

setup.py
project/
  run.py
  __init__.pxd
  Settings/
    __init__.pxd
    Settings.pyx/pxd
  Helper/
    __init__.pxd
    Helper.pyx/pxd

This will create a single module called "package" which will all be built at once and installed at once. Once installed your user would be able to do from project import whatever.

You'll notice I've added some __init__.pxd files. These serve basically the same purpose as __init__.py files, except they identify folders as (sub)packages for Cython's cimport mechanism.

Helper .pxd then starts:

from project.Settings.Settings cimport Settings, PySettings

I've used a full rather than relative import because cimport and relative imports seems a bit unreliable.

Setup.py is as follows:

from setuptools.extension import Extension
from setuptools import setup
from Cython.Build import cythonize

ext_modules = cythonize("project/*/*.pyx")

setup(ext_modules=ext_modules)

You are welcome to explicitly use Extension but for something simple it seemed easier to send the file pattern directly to cythonize.

---

I've only tested the Cythonize stage of this since I don't have the C++ files needed to compile and run it.