2007-06-04

Now you too can know how import (roughly) works!

About two months ago I started trying to diagram import. The idea was twofold: I wanted something that people could look at so understanding how import worked overall was not a mystery, but I also wanted to show how complicated the thing is to help me argue for some changes I want to make.

To help me figure out what to diagram, I decided to write out the pseudocode to import. I figured I could write the pseudocode quickly to work out the level of detail I wanted for the diagram. That way I would waste less time experimenting in Graphviz which takes longer than experimenting in pseudocode.

The pseudocode turned out so well that I have now scrapped the diagram. I now have pseudocode that is syntactically proper Python with a ton of comments of what is going on. I then just make calls to functions that I have not defined that magically handle nitty-gritty details. This makes it easier to read than importlib as I can just cram it all into a single function without worrying about proper object abstractions, etc.

At this point I have written out the code for how import goes about looking for a module. This covers the use of sys.meta_path, sys.path, sys.path_importer_cache, and sys.path_hooks. I have not covered how the bytecode/source dance works or extension modules. Writing up those two will show why Python makes so many stat calls when it does an import.