2006-09-27

Implementing phase 2 of PEP 302

If you read PEP 302 you will notice the mention of a "phase 2" in a couple of places. The second phase of the PEP which was never done because of a lack of time was to refactor the work done by __import__ so that it worked within the PEP 302 machinery.

This means breaking up the various types of imports into their own importers. Built-ins and freeze modules would become meta_path importers. Importing .py, .pyc, and extension modules would all become their own importers. In general this is really nice as stat calls could now suddenly be minimized as a importer factory function could be written that inspects the path it is being requested to work with, note that only extension modules are in the directory, and then return an importer only for extension modules. This would also allow for having a .py only importer which could take an optional flag that skips writing a .pyc file or specifies where to write the .pyc files (giving us the possibility of having PEP 304 implemented).

For my work, by making sure that the importers are broken up as needed and made flexible enough, I can fully control what imports are allowed and thus control what code can do. Basically I need to end up with the ability to allow .py imports with no creation of .pyc files, prevent .pyc imports, be able to restrict extension module imports, and restrict built-in module imports.

I have emailed python-dev to find out if doing all of this in pure Python source code is worth the effort compared to trying to make the C code work. While there are some bootstrapping issues, they are surmountable. The real question is whether it is worth the time and effort to do the work of the rewrite or to just force my way through the C code.

Either way imports will become more flexible.