2011-03-20

PyCon 2011 wrap-up

tl;dr version: it was awesome! Read on to find out what I did at the sprints and two themes that I found at the conference.

Since I have not had a chance to blog until now, people like Nick Coghlan and Brian Curtin have beaten me to writing good summaries of the VM summit, language summit, conference, and sprints so I won't worry about doing an overall summary of everything.

At the sprints I almost exclusively focused on my Python 3 on PyPI website. After squashing a bunch of bugs I added two more popularity metrics. The first metric is download rate. This is to help balance out against total download count which favours older projects thanks to simply having been available longer. I also added the poll results as found on the front page of www.python.org (look in the upper-right corner of the page). The download rate has about 50% of those projects converted which is great, while the poll results is at 16% which is also very good considering people are explicitly asking for those projects to be converted because they were not at the time of voting. And because people seem to have some odd desire to see the crap code I wrote for this website, you can find it here.

As for the two themes I see gaining more exposure in the upcoming year, one was obviously Python 3 support from projects. There is now a book out by Lennart Regebro [Amazon] (which I wrote the foreword for), a good number of projects have already switched, Django will be switching this summer, and at the sprints Jim Fulton started porting zc.buildout to Python 3 (which happens to be the largest Python 3 holdout according to my website at the moment). In other words the momentum is there for Python 3 and I continue to not worry about it.

The other theme which gained more traction is separating Python the language from CPython the interpreter. PyPy's performance is now good enough to make it a serious consideration for anyone to use. And once they release a version supporting Python 2.7 (IronPython beat them to Python 2.7 compatibility) then they will be in a position to become the de-facto interpreter people use when they don't have specific requirements (e.g., embedding Python in a Java program and thus requiring Jython). What this means (at least to me) is that python-dev is going to need to start to shift towards a stance where the Python language takes precedence over CPython the interpreter. While CPython will continue to be the reference implementation (don't ask if PyPy will become it; I don't think they want that responsibility honestly), certain considerations are going to be needed in order to make sure that things we do in the Hg repository do not end up hurting the other VMs.

To help with this goal of making sure the Python language does not play favourites, I am planning on doing three things. One is a PEP I am drafting up dictating that C extension code in the stdlib must be kept fully compatible with any pure Python equivalent code. It will also say that any extension module with no matching pure Python version must get an exemption to be included in the stdlib. This is to prevent all the VMs from having to re-implement every extension module on their own, essentially leading to four versions of the same module.

The second thing is getting importlib to become the de-facto implementation of import in Python. Every single VM hates implementing import. It's subtle and complicated, making it very tough to implement in a way that is conformant. By moving it to pure Python, importlib will allow all of the VMs to simply use importlib and thus not have to re-implement import.

Lastly, a very long-term goal is to break out Python's stdlib into its own repository. This will make it even more obvious that CPython is the reference implementation of Python and that it is not special in any other way. It will also allow the other VMs to use the same code base to prevent duplicated work (the key contributors on the other VMs are already receiving commit privileges on CPython so they can start fixing bugs now). This is a very long-term project of mine and won't be started until my other two goals mentioned above are finished. In other words my goal is to start at PyCon 2012 with this.