15f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)""" 25f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)Import hooks; when installed with the install() function, these hooks 35f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)allow importing .pyx files as if they were Python modules. 45f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 55f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)If you want the hook installed every time you run Python 65f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)you can add it to your Python version by adding these lines to 75f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)sitecustomize.py (which you can create from scratch in site-packages 85f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)if it doesn't exist there or somewhere else on your python path):: 95f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import pyximport 115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyximport.install() 125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)For instance on the Mac with a non-system Python 2.3, you could create 145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)sitecustomize.py with only those two lines at 155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)/usr/local/lib/python2.3/site-packages/sitecustomize.py . 165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)A custom distutils.core.Extension instance and setup() args 185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)(Distribution) for for the build can be defined by a <modulename>.pyxbld 195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)file like: 205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# examplemod.pyxbld 225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def make_ext(modname, pyxfilename): 235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) from distutils.extension import Extension 245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return Extension(name = modname, 255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sources=[pyxfilename, 'hello.c'], 265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) include_dirs=['/myinclude'] ) 275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def make_setup_args(): 285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return dict(script_args=["--compiler=mingw32"]) 295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)Extra dependencies can be defined by a <modulename>.pyxdep . 315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)See README. 325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)Since Cython 0.11, the :mod:`pyximport` module also has experimental 345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)compilation support for normal Python modules. This allows you to 355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)automatically run Cython on every .pyx and .py module that Python 365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)imports, including parts of the standard library and installed 375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)packages. Cython will still fail to compile a lot of Python modules, 385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)in which case the import mechanism will fall back to loading the 395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)Python source modules instead. The .py import mechanism is installed 405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)like this:: 415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyximport.install(pyimport = True) 435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)Running this module as a top-level script will run a test and then print 455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)the documentation. 465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)This code is based on the Py2.3+ import protocol as described in PEP 302. 485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)""" 495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)import sys 515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)import os 525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)import glob 535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)import imp 545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)mod_name = "pyximport" 565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)assert sys.hexversion >= 0x2030000, "need Python 2.3 or later" 585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)PYX_EXT = ".pyx" 605f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)PYXDEP_EXT = ".pyxdep" 615f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)PYXBLD_EXT = ".pyxbld" 625f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 635f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)DEBUG_IMPORT = False 645f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 655f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def _print(message, args): 665f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if args: 675f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) message = message % args 685f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) print(message) 695f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 705f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def _debug(message, *args): 715f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if DEBUG_IMPORT: 725f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _print(message, args) 735f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 745f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def _info(message, *args): 755f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _print(message, args) 765f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 775f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# Performance problem: for every PYX file that is imported, we will 785f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# invoke the whole distutils infrastructure even if the module is 795f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# already built. It might be more efficient to only do it when the 805f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# mod time of the .pyx is newer than the mod time of the .so but 815f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# the question is how to get distutils to tell me the name of the .so 825f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# before it builds it. Maybe it is easy...but maybe the peformance 835f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# issue isn't real. 845f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def _load_pyrex(name, filename): 855f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) "Load a pyrex file given a name and filename." 865f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 875f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def get_distutils_extension(modname, pyxfilename, language_level=None): 885f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# try: 895f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# import hashlib 905f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# except ImportError: 915f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# import md5 as hashlib 925f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# extra = "_" + hashlib.md5(open(pyxfilename).read()).hexdigest() 935f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# modname = modname + extra 945f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) extension_mod,setup_args = handle_special_build(modname, pyxfilename) 955f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if not extension_mod: 965f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if not isinstance(pyxfilename, str): 975f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # distutils is stupid in Py2 and requires exactly 'str' 985f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # => encode accidentally coerced unicode strings back to str 995f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxfilename = pyxfilename.encode(sys.getfilesystemencoding()) 1005f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) from distutils.extension import Extension 1015f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) extension_mod = Extension(name = modname, sources=[pyxfilename]) 1025f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if language_level is not None: 1035f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) extension_mod.cython_directives = {'language_level': language_level} 1045f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return extension_mod,setup_args 1055f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1065f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def handle_special_build(modname, pyxfilename): 1075f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) special_build = os.path.splitext(pyxfilename)[0] + PYXBLD_EXT 1085f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ext = None 1095f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setup_args={} 1105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if os.path.exists(special_build): 1115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # globls = {} 1125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # locs = {} 1135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # execfile(special_build, globls, locs) 1145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # ext = locs["make_ext"](modname, pyxfilename) 1155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) mod = imp.load_source("XXXX", special_build, open(special_build)) 1165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) make_ext = getattr(mod,'make_ext',None) 1175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if make_ext: 1185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ext = make_ext(modname, pyxfilename) 1195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert ext and ext.sources, ("make_ext in %s did not return Extension" 1205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) % special_build) 1215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) make_setup_args = getattr(mod,'make_setup_args',None) 1225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if make_setup_args: 1235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setup_args = make_setup_args() 1245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert isinstance(setup_args,dict), ("make_setup_args in %s did not return a dict" 1255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) % special_build) 1265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert set or setup_args, ("neither make_ext nor make_setup_args %s" 1275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) % special_build) 1285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ext.sources = [os.path.join(os.path.dirname(special_build), source) 1295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for source in ext.sources] 1305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return ext, setup_args 1315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def handle_dependencies(pyxfilename): 1335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) testing = '_test_files' in globals() 1345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) dependfile = os.path.splitext(pyxfilename)[0] + PYXDEP_EXT 1355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # by default let distutils decide whether to rebuild on its own 1375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # (it has a better idea of what the output file will be) 1385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # but we know more about dependencies so force a rebuild if 1405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # some of the dependencies are newer than the pyxfile. 1415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if os.path.exists(dependfile): 1425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) depends = open(dependfile).readlines() 1435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) depends = [depend.strip() for depend in depends] 1445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # gather dependencies in the "files" variable 1465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # the dependency file is itself a dependency 1475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) files = [dependfile] 1485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for depend in depends: 1495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) fullpath = os.path.join(os.path.dirname(dependfile), 1505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) depend) 1515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) files.extend(glob.glob(fullpath)) 1525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # only for unit testing to see we did the right thing 1545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if testing: 1555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _test_files[:] = [] #$pycheck_no 1565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # if any file that the pyxfile depends upon is newer than 1585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # the pyx file, 'touch' the pyx file so that distutils will 1595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # be tricked into rebuilding it. 1605f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for file in files: 1615f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) from distutils.dep_util import newer 1625f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if newer(file, pyxfilename): 1635f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("Rebuilding %s because of %s", pyxfilename, file) 1645f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) filetime = os.path.getmtime(file) 1655f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) os.utime(pyxfilename, (filetime, filetime)) 1665f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if testing: 1675f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _test_files.append(file) 1685f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1695f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def build_module(name, pyxfilename, pyxbuild_dir=None, inplace=False, language_level=None): 1705f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert os.path.exists(pyxfilename), ( 1715f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) "Path does not exist: %s" % pyxfilename) 1725f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) handle_dependencies(pyxfilename) 1735f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1745f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) extension_mod,setup_args = get_distutils_extension(name, pyxfilename, language_level) 1755f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_in_temp=pyxargs.build_in_temp 1765f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sargs=pyxargs.setup_args.copy() 1775f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sargs.update(setup_args) 1785f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_in_temp=sargs.pop('build_in_temp',build_in_temp) 1795f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1805f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import pyxbuild 1815f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod, 1825f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_in_temp=build_in_temp, 1835f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=pyxbuild_dir, 1845f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setup_args=sargs, 1855f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=inplace, 1865f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) reload_support=pyxargs.reload_support) 1875f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert os.path.exists(so_path), "Cannot find: %s" % so_path 1885f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1895f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) junkpath = os.path.join(os.path.dirname(so_path), name+"_*") #very dangerous with --inplace ? yes, indeed, trying to eat my files ;) 1905f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) junkstuff = glob.glob(junkpath) 1915f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for path in junkstuff: 1925f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if path!=so_path: 1935f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 1945f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) os.remove(path) 1955f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except IOError: 1965f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _info("Couldn't remove %s", path) 1975f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 1985f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return so_path 1995f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2005f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def load_module(name, pyxfilename, pyxbuild_dir=None, is_package=False, 2015f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_inplace=False, language_level=None, so_path=None): 2025f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 2035f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if so_path is None: 2045f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if is_package: 2055f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module_name = name + '.__init__' 2065f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 2075f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module_name = name 2085f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) so_path = build_module(module_name, pyxfilename, pyxbuild_dir, 2095f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=build_inplace, language_level=language_level) 2105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) mod = imp.load_dynamic(name, so_path) 2115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if is_package and not hasattr(mod, '__path__'): 2125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) mod.__path__ = [os.path.dirname(so_path)] 2135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert mod.__file__ == so_path, (mod.__file__, so_path) 2145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except Exception: 2155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if pyxargs.load_py_module_on_import_failure and pyxfilename.endswith('.py'): 2165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # try to fall back to normal import 2175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) mod = imp.load_source(name, pyxfilename) 2185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert mod.__file__ in (pyxfilename, pyxfilename+'c', pyxfilename+'o'), (mod.__file__, pyxfilename) 2195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 2205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import traceback 2215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) raise ImportError("Building module %s failed: %s" % 2225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) (name, 2235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) traceback.format_exception_only(*sys.exc_info()[:2]))), None, sys.exc_info()[2] 2245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return mod 2255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# import hooks 2285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)class PyxImporter(object): 2305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """A meta-path importer for .pyx files. 2315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """ 2325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def __init__(self, extension=PYX_EXT, pyxbuild_dir=None, inplace=False, 2335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=None): 2345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.extension = extension 2355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.pyxbuild_dir = pyxbuild_dir 2365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.inplace = inplace 2375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.language_level = language_level 2385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def find_module(self, fullname, package_path=None): 2405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fullname in sys.modules and not pyxargs.reload_support: 2415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None # only here when reload() 2425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 2435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) fp, pathname, (ext,mode,ty) = imp.find_module(fullname,package_path) 2445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fp: fp.close() # Python should offer a Default-Loader to avoid this double find/open! 2455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if pathname and ty == imp.PKG_DIRECTORY: 2465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pkg_file = os.path.join(pathname, '__init__'+self.extension) 2475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if os.path.isfile(pkg_file): 2485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return PyxLoader(fullname, pathname, 2495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) init_path=pkg_file, 2505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=self.pyxbuild_dir, 2515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=self.inplace, 2525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 2535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if pathname and pathname.endswith(self.extension): 2545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return PyxLoader(fullname, pathname, 2555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=self.pyxbuild_dir, 2565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=self.inplace, 2575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 2585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if ty != imp.C_EXTENSION: # only when an extension, check if we have a .pyx next! 2595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 2605f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2615f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # find .pyx fast, when .so/.pyd exist --inplace 2625f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxpath = os.path.splitext(pathname)[0]+self.extension 2635f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if os.path.isfile(pyxpath): 2645f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return PyxLoader(fullname, pyxpath, 2655f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=self.pyxbuild_dir, 2665f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=self.inplace, 2675f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 2685f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2695f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # .so/.pyd's on PATH should not be remote from .pyx's 2705f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # think no need to implement PyxArgs.importer_search_remote here? 2715f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2725f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except ImportError: 2735f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pass 2745f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2755f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # searching sys.path ... 2765f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2775f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #if DEBUG_IMPORT: print "SEARCHING", fullname, package_path 2785f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if '.' in fullname: # only when package_path anyway? 2795f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) mod_parts = fullname.split('.') 2805f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module_name = mod_parts[-1] 2815f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 2825f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module_name = fullname 2835f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyx_module_name = module_name + self.extension 2845f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # this may work, but it returns the file content, not its path 2855f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #import pkgutil 2865f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #pyx_source = pkgutil.get_data(package, pyx_module_name) 2875f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 2885f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if package_path: 2895f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) paths = package_path 2905f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 2915f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) paths = sys.path 2925f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) join_path = os.path.join 2935f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) is_file = os.path.isfile 2945f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) is_abs = os.path.isabs 2955f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) abspath = os.path.abspath 2965f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #is_dir = os.path.isdir 2975f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sep = os.path.sep 2985f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for path in paths: 2995f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if not path: 3005f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) path = os.getcwd() 3015f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) elif not is_abs(path): 3025f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) path = abspath(path) 3035f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if is_file(path+sep+pyx_module_name): 3045f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return PyxLoader(fullname, join_path(path, pyx_module_name), 3055f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=self.pyxbuild_dir, 3065f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=self.inplace, 3075f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 3085f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3095f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # not found, normal package, not a .pyx file, none of our business 3105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("%s not found" % fullname) 3115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 3125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)class PyImporter(PyxImporter): 3145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """A meta-path importer for normal .py files. 3155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """ 3165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def __init__(self, pyxbuild_dir=None, inplace=False, language_level=None): 3175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if language_level is None: 3185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level = sys.version_info[0] 3195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.super = super(PyImporter, self) 3205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.super.__init__(extension='.py', pyxbuild_dir=pyxbuild_dir, inplace=inplace, 3215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=language_level) 3225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.uncompilable_modules = {} 3235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.blocked_modules = ['Cython', 'pyxbuild', 'pyximport.pyxbuild', 3245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 'distutils.extension', 'distutils.sysconfig'] 3255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def find_module(self, fullname, package_path=None): 3275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fullname in sys.modules: 3285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 3295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fullname.startswith('Cython.'): 3305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 3315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fullname in self.blocked_modules: 3325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # prevent infinite recursion 3335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 3345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if _lib_loader.knows(fullname): 3355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return _lib_loader 3365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("trying import of module '%s'", fullname) 3375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if fullname in self.uncompilable_modules: 3385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) path, last_modified = self.uncompilable_modules[fullname] 3395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 3405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) new_last_modified = os.stat(path).st_mtime 3415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if new_last_modified > last_modified: 3425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # import would fail again 3435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return None 3445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except OSError: 3455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # module is no longer where we found it, retry the import 3465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pass 3475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.blocked_modules.append(fullname) 3495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 3505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) importer = self.super.find_module(fullname, package_path) 3515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if importer is not None: 3525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if importer.init_path: 3535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) path = importer.init_path 3545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) real_name = fullname + '.__init__' 3555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 3565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) path = importer.path 3575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) real_name = fullname 3585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("importer found path %s for module %s", path, real_name) 3595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 3605f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) so_path = build_module( 3615f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) real_name, path, 3625f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxbuild_dir=self.pyxbuild_dir, 3635f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level, 3645f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=self.inplace) 3655f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _lib_loader.add_lib(fullname, path, so_path, 3665f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) is_package=bool(importer.init_path)) 3675f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return _lib_loader 3685f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except Exception: 3695f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if DEBUG_IMPORT: 3705f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import traceback 3715f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) traceback.print_exc() 3725f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # build failed, not a compilable Python module 3735f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 3745f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) last_modified = os.stat(path).st_mtime 3755f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except OSError: 3765f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) last_modified = 0 3775f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.uncompilable_modules[fullname] = (path, last_modified) 3785f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) importer = None 3795f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) finally: 3805f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.blocked_modules.pop() 3815f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return importer 3825f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3835f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)class LibLoader(object): 3845f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def __init__(self): 3855f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self._libs = {} 3865f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3875f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def load_module(self, fullname): 3885f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 3895f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) source_path, so_path, is_package = self._libs[fullname] 3905f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except KeyError: 3915f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) raise ValueError("invalid module %s" % fullname) 3925f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("Loading shared library module '%s' from %s", fullname, so_path) 3935f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return load_module(fullname, source_path, so_path=so_path, is_package=is_package) 3945f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3955f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def add_lib(self, fullname, path, so_path, is_package): 3965f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self._libs[fullname] = (path, so_path, is_package) 3975f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 3985f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def knows(self, fullname): 3995f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return fullname in self._libs 4005f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4015f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)_lib_loader = LibLoader() 4025f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4035f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)class PyxLoader(object): 4045f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def __init__(self, fullname, path, init_path=None, pyxbuild_dir=None, 4055f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) inplace=False, language_level=None): 4065f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) _debug("PyxLoader created for loading %s from %s (init path: %s)", 4075f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) fullname, path, init_path) 4085f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.fullname = fullname 4095f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.path, self.init_path = path, init_path 4105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.pyxbuild_dir = pyxbuild_dir 4115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.inplace = inplace 4125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.language_level = language_level 4135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) def load_module(self, fullname): 4155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) assert self.fullname == fullname, ( 4165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) "invalid module, expected %s, got %s" % ( 4175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.fullname, fullname)) 4185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if self.init_path: 4195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # package 4205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #print "PACKAGE", fullname 4215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module = load_module(fullname, self.init_path, 4225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.pyxbuild_dir, is_package=True, 4235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_inplace=self.inplace, 4245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 4255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module.__path__ = [self.path] 4265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 4275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) #print "MODULE", fullname 4285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) module = load_module(fullname, self.path, 4295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) self.pyxbuild_dir, 4305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_inplace=self.inplace, 4315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=self.language_level) 4325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return module 4335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)#install args 4365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)class PyxArgs(object): 4375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_dir=True 4385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_in_temp=True 4395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setup_args={} #None 4405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)##pyxargs=None 4425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def _have_importers(): 4445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) has_py_importer = False 4455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) has_pyx_importer = False 4465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for importer in sys.meta_path: 4475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if isinstance(importer, PyxImporter): 4485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if isinstance(importer, PyImporter): 4495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) has_py_importer = True 4505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) else: 4515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) has_pyx_importer = True 4525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return has_py_importer, has_pyx_importer 4545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def install(pyximport=True, pyimport=False, build_dir=None, build_in_temp=True, 4565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setup_args={}, reload_support=False, 4575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) load_py_module_on_import_failure=False, inplace=False, 4585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=None): 4595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """Main entry point. Call this to install the .pyx import hook in 4605f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) your meta-path for a single Python process. If you want it to be 4615f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) installed whenever you use Python, add it to your sitecustomize 4625f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) (as described above). 4635f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4645f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) You can pass ``pyimport=True`` to also install the .py import hook 4655f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) in your meta-path. Note, however, that it is highly experimental, 4665f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) will not work for most .py files, and will therefore only slow 4675f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) down your imports. Use at your own risk. 4685f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4695f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) By default, compiled modules will end up in a ``.pyxbld`` 4705f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) directory in the user's home directory. Passing a different path 4715f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) as ``build_dir`` will override this. 4725f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4735f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``build_in_temp=False`` will produce the C files locally. Working 4745f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) with complex dependencies and debugging becomes more easy. This 4755f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) can principally interfere with existing files of the same name. 4765f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_in_temp can be overriden by <modulename>.pyxbld/make_setup_args() 4775f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) by a dict item of 'build_in_temp' 4785f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4795f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``setup_args``: dict of arguments for Distribution - see 4805f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) distutils.core.setup() . They are extended/overriden by those of 4815f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) <modulename>.pyxbld/make_setup_args() 4825f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4835f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``reload_support``: Enables support for dynamic 4845f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) reload(<pyxmodulename>), e.g. after a change in the Cython code. 4855f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) Additional files <so_path>.reloadNN may arise on that account, when 4865f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) the previously loaded module file cannot be overwritten. 4875f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4885f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``load_py_module_on_import_failure``: If the compilation of a .py 4895f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) file succeeds, but the subsequent import fails for some reason, 4905f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) retry the import with the normal .py module instead of the 4915f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) compiled module. Note that this may lead to unpredictable results 4925f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for modules that change the system state during their import, as 4935f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) the second import will rerun these modifications in whatever state 4945f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) the system was left after the import of the compiled module 4955f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) failed. 4965f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4975f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``inplace``: Install the compiled module next to the source file. 4985f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 4995f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) ``language_level``: The source language level to use: 2 or 3. 5005f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) The default is to use the language level of the current Python 5015f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) runtime for .py files and Py2 for .pyx files. 5025f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """ 5035f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if not build_dir: 5045f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) build_dir = os.path.join(os.path.expanduser('~'), '.pyxbld') 5055f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5065f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) global pyxargs 5075f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs = PyxArgs() #$pycheck_no 5085f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs.build_dir = build_dir 5095f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs.build_in_temp = build_in_temp 5105f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs.setup_args = (setup_args or {}).copy() 5115f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs.reload_support = reload_support 5125f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyxargs.load_py_module_on_import_failure = load_py_module_on_import_failure 5135f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5145f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) has_py_importer, has_pyx_importer = _have_importers() 5155f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) py_importer, pyx_importer = None, None 5165f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5175f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if pyimport and not has_py_importer: 5185f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) py_importer = PyImporter(pyxbuild_dir=build_dir, inplace=inplace, 5195f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=language_level) 5205f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) # make sure we import Cython before we install the import hook 5215f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import Cython.Compiler.Main, Cython.Compiler.Pipeline, Cython.Compiler.Optimize 5225f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sys.meta_path.insert(0, py_importer) 5235f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5245f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) if pyximport and not has_pyx_importer: 5255f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pyx_importer = PyxImporter(pyxbuild_dir=build_dir, inplace=inplace, 5265f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) language_level=language_level) 5275f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sys.meta_path.append(pyx_importer) 5285f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5295f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) return py_importer, pyx_importer 5305f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5315f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def uninstall(py_importer, pyx_importer): 5325f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """ 5335f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) Uninstall an import hook. 5345f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) """ 5355f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 5365f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sys.meta_path.remove(py_importer) 5375f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except ValueError: 5385f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pass 5395f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5405f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 5415f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) sys.meta_path.remove(pyx_importer) 5425f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except ValueError: 5435f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pass 5445f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5455f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)# MAIN 5465f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5475f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)def show_docs(): 5485f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) import __main__ 5495f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) __main__.__name__ = mod_name 5505f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) for name in dir(__main__): 5515f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) item = getattr(__main__, name) 5525f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) try: 5535f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) setattr(item, "__module__", mod_name) 5545f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) except (AttributeError, TypeError): 5555f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) pass 5565f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) help(__main__) 5575f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) 5585f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles)if __name__ == '__main__': 5595f1c94371a64b3196d4be9466099bb892df9b88eTorne (Richard Coles) show_docs() 560