2007-03-04

Answering the question I was asked the most at PyCon

"Will python-dev ever switch to using PyPy?"

I was asked that question enough that if I had a nickel everytime it was posed I could have bought myself a can of pop. People had PyPy on their mind as their 0.99 release announcement happened very shortly before the conference. And with them being only 2 - 3x slower than CPython without their JIT technology, people wanted to know if python-dev would ever be willing to switch over. Well, here is my take on answering that question.

First, PyPy is not as fast or faster than CPython yet. I honestly hope they pull it off, but they are not there yet. No one wants a slower Python (we catch enough slack for our current speed which I think is fast enough). Until they get really damn close to CPython speed this will be an issue.

Second, PyPy does not directly support extension modules (from my understanding). Once again, I hope they meet their goal of supporting them, but they have not yet so that is another hurdle to overcome. This also ties in with needing to be backwards-compatible with CPython.

Third, python-dev does not know PyPy. If we switched it would mean all of the active python-dev developers would have to learn a new codebsae. Now you might be thinking, "yes, but it's in Python and shouldn't be too hard to pick up". Maybe, but you have to remember two things. One is that we are all volunteers and have a lot of time and energy invested in CPython. I don't know how many python-dev regulars would be willing to throw that away and learn a new codebase just to get to the same level of knowledge that they are at now with CPython. Second, I honestly don't know how easy it is to read the PyPy implementation. Considering some of the smartest Python programmers in the world work on it there is a possibility that some stuff in there is really complex and hard to comprehend (but I suspect the code is easy to read).

Fourth, and related to number three, is that of the flexibility of using CPython to test out new language features. With CPython, Guido and everyone else on python-dev know how to try out new language features. If we moved to PyPy we would have the issue of not being able to do that for a period of time (possibly a long period) until we came up to speed on the PyPy implementation.

And fifth, PyPy is not run by python-dev. No one has told me that they asked the PyPy guys this question as to whether they even wanted to do this. There is an assumption in this question that the PyPy guys want to give up control of PyPy to python-dev and work under its rules, practices, idiosyncrasies, and scheduling. This is the same answer we have given when people have asked if we would ever move to IronPython if Mono becames widespread; python-dev needs to be able to control the implementation of the Python language so it can experiment with new features without having its hands tied.

Now, as to whether we would be willing to incorporate PyPy code into Python's trunk, the answer is always "sure, if it meets our standards". That means it needs to meet PEP 7 & 8, we can maintain it, it has docs and tests, and that we actually want the feature that the code implements. This is the same requirements that were recently put to the Stackless guys when someone proposed adding some Stackless features to CPython. I doubt anyone would propose stupid features, so that one is almost a moot point. But the ability to maintain the code is the real sticking point as python-dev inherently takes over any code committed and the last thing any of us want is some nasty piece of code that gives us a great feature also gives us major headaches.

Anyway, this would be my reasoning if the question was ever seriously posed by the PyPy folk to the python-dev folk.