2006-12-07

Import rewrite close to being semantically complete

I started off this week finishing the last minor details of what I thought I needed to implement (basically supporting __all__, what to do when the bytecode in a .pyc is bad, using the import lock, and frozen packages). Got all of that squared away 46 hours ago.

Then I started to run regrtest.py (Python's regression test suite) using my import code. That brought about its own issues. Most of it was just stuff I didn't think of (what to do when an empty string is passed in as the name of the module to import, which varies based on the type of possible relative import you are doing). Other stuff has been bad assumptions made about PEP 302 loaders.

Anyway, at this point I am three tests short of passing (if you ignore that I had to regenerate the output for test_pkg; boy do I want to get rid of all old-style tests). test_traceback is failing for some reason that I have not gotten into. test_runpy is crapping out because it assumes all loaders define get_code which is actually optional. And test_import is failing because I don't raise ImportWarning when someone tries to import a directory that doesn't have __init__.py and for some odd reload() test.

After that I just need to support case-insensitive file-systems. Once that is in place I will have semantic completeness at least as far as Python's test suite (which is probably pretty good). There might be some issue with people's custom importers/loaders (although I don't know how) or some bad assumptions made by people about some things.

But I am hopeful I can be semantically complete before 2007 rolls around. That way, if I happen to have my laptop by the time I go on holiday I can do my bonus work: rewrite zipimport, implement a sqlite3 importer/loader, and write some benchmarks to see where performance is lagging and to help any future optimizations.

It will definitely be nice to reach semantic completeness. Even if this code becomes like the 'compile' package and just a Python re-implementation of some internals it will still be useful just for defining the semantics of import.