PEP 3108 (stdlib cleanup) progress

As of this moment there are 81 deprecated modules in Python 2.6. I have personally deprecated 56 of them since PEP 3108 was accepted. There are about 15 modules left to still be deprecated and removed. None of these counts cover modules with new names.

Unfortunately there is still work to be done. Those 15 modules left, all but one are there because they require removing their usage from the stdlib in 2.6 (symtable might survive if it gets documentation written for it). There is also the issue of dealing with renamed modules which I have personally not touched yet (I view the deletions as more important since it removes code rather than just shuffling it around; plus 2to3 makes the change-over for new names easier to live with than an entire module disappearing). Still plenty to do between now and b1.

PEP 3108 is accepted! Let the deprecations begin!

Guido officially accepted PEP 3108 the other day! Now the work begins.

Issue 2775 is tracking the PEP work. If people want to help out they can claim something to do there.

I am personally starting with module deletions (audiodev was the first victim). The steps are laid out in the PEP and they are very straight-forward (which is why I made issue 2775 an easy issue). And if you use TextMate it is even easier thanks to the new Python-Dev bundle! There is a command in it that will generate the boilerplate for the deprecation warning for you, complete with inserting the module's name!

If you would like to help please submit patches to the issue. The betas for 2.6 and 3.0 cannot go out until this PEP is implemented!

PEP 3108 finally sent to python-3000

After literally years at trying to get the stdlib lined up to be reorganized and thinned out, PEP 3108 is finally off to python-3000. Here is to hoping it is a relatively painless ordeal.

I hate PEP 263

PEP 263 is the one that allows one to specify the encoding of a Python source file. It is giving me such a headache with trying to bootstrap importlib. Because I must be able to open a source file with the proper encoding I must also have the entire codecs system working properly. This is a problem when the codecs system relies on imports to get to decoders.

Now the UTF-8 decoder is set up by default. But everything else, including ASCII, must be reachable through an import. That can be a problem when it is import that needs that module.

It is also a pain that one must open a file in a generic fashion, read the first two lines of a file, use a regex to try to find a specified encoding, and then reopen the file with the encoding found. That's a lot of stat calls and such that can be expensive. To deal with this and other bootstrapping issues I am going to have to expose some more C code in my special version of 3.0 in order to get this to work. Luckily it has partially been done for me thanks to imp.find_module() which does the proper file opening, albeit using the C import code. Once a pull it out to give me basically a custom source code open() function I should be able to move on to the next failing test.

Currently there are seven failing tests (albeit I can't run importlib properly anyway because of the PEP 263 issues). I have until early June to get this all done. Here is to hoping I can pull this off and properly delegate the stdlib reorg so that it all doesn't impact my personal life (which now includes looking for a new apartment for myself).

Random update from PythonLand

Since I have not blogged in a week but don't have anything that warrants a full-blown post I figured I would collect stuff I have been working on together in a single post.

My presentation to VanPyZ on Py3K went well despite the fact a projector couldn't be found. I had to use a whiteboard in the room which was fine, but my handwriting is atrocious so I am amazed people were able to follow me. Gathering up all of the major changes made me realize how overall the language itself is just getting cleaned up for the better. Where pain comes in is through changes to the stdlib. Hopefully we will be able to have reasonable warnings in the stdlib for people so they can have an easier transition.

Pulling together what needs to be done in 2.6 to ease the transition to 3.0 is being collected in a Google spreadsheet. The details of how warnings will work and such has yet to be worked out. I am personally hoping that there are levels of warnings. One for stuff that you need to change (e.g., stdlib changes to using bytes), stuff that you can change syntactically but that 2to3 could do as well (e.g., the new raise/except syntax), and stuff that would normally require a __future__ statement (e.g., print becoming a function). I honestly think the latter is not needed and that a __future__ statement should just be left out in favoring of relying on 2to3. Since stuff like 'print' and 'exec' are syntax they can be detected perfectly and thus changed perfectly. I say the less warnings we have to pile into 2.6 the better.

I have also formally posted PEP 362 to python-dev. Neal Becker said that Boost.Python could use the Signature object and was wondering if PEP 362 was going to be accepted. I subsequently updated the code to work with 3.0a1 (which happens to be both 2.6 and 3.0 compatible without any source code changes). If python-dev happens to reject PEP 362 I will end up releasing the code to PyPI and start a Google Code project for it.

One thing that my PEP 362 work made me realize is how much of a pain it is to be working on stuff for 2.6 right now that have different behavior between 2.6 and 3.0. The issue I ran into is inspect.getargspec(). In 3.0 it raises an exception if the function has keyword-only arguments or annotations; you should use getfullargspec() instead. It would make sense then that if I just added getfullargspec() to 2.6 with dummy values for keyword-only arguments and annotations. But that seems silly since both keyword-only arguments and annotations will be backported. Neither, though, have been backported. Plus Py3K warnings have not been exposed at the Python level yet (it has just been done in a cursory fashion in C code).

I ended up just using getfullargspec when possible, defaulting to getargspec when necessary. Now I have to decide if I want to try to get python-dev to make a decision on how to handle warnings again, backport features to 2.6, or stick with trying to get importlib bootstrapped into 3.0. So much to do. =)

Abstract Base Classes PEP accepted

Guido has now accepted PEP 3119: Introducing Abstract Base Classes. The discussion on this topic went on for a while; so long, in fact, that it led to PEP 3133. But in the end people were either tolerant of the PEP or liked it.

Personally, I am indifferent. While I provided feedback on the first draft I honestly have no personally need for this. But I do know people do request we have some way of specifying interfaces (or something similar) fairly regularly. At least we have an easier way to tell if you are dealing with a sequence or a mapping data structure.

Deadline today for Py3K PEPs

Today is the day to get in those Py3K PEPs. In order to meet a sometime-in-June alpha for Python 3.0, a deadline for PEPs specifically affecting the release are due today. The only exemption is that of PEPs and such affecting the stdlib.


This week has already seen a flurry of new PEPs come in. PEPs 3121, 3122, and 3141 have been checked in. Plus there are roughly three or four currently being discussed on python-3000. The only benefit of all of these PEPs being out in the wild is that people are actually focusing their discussions and not going all over the place as tends to happen when there is debate over only a single PEP.

PEP 3113 (removal of automatic tuple parameter unpacking)

Anyone at PyCon who saw my lightning talk or was around me throughout the conference will know what this PEP is about: getting rid of tuple parameters. Thanks to everyone at PyCon who provided arguments for their removal (basically Ka-Ping Yee was the only person who provided any argument for them to stay).

Collin Winter has already implemented a transform rule for 2to3 and Guido has already said he supports the idea. It looks like this PEP will get accepted for Py3K! Woohoo!

No more string exceptions for 2.6

I just checked in the next step in PEP 352 for 2.6: no more string exceptions! You now get a TypeError if you try to raise a string exception. There is also a DeprecationWarning raised if you try to catch a string exception.

People have been talking about their removal for a long time (before I got involved in python-dev). Feels good to have made it happen.

stdlib reorg PEP has been added

I just emailed python-3000 an inlined copy of my stdlib reorg PEP (PEP 3108). The PEP should show up online the next time the cronjob for PEPs runs.

Hopefully this won't kick up too much heat for me since I am just trying to help get this thing done. But I am sure there are some suggested module removals people will not be happy with.
Question is whether enough people care about a specific removal to deem the module worth keeping.

Plus I have some suggested groupings of modules in the Open Issues section. Guido never explicitly cleared me to do groupings, but I was thinking of them anyway as I went through the stdlib and almost everyone who commented on my blog suggested the same thing, I figured I should at least toss the option out there for Guido to consider. We will see if he goes for any of them or not.

Started the stdlib reorg PEP

I just finished the initial draft covering modules to remove. A very quick count has roughly 58 modules that I am planning to explicitly suggest for removal! There are also others being suggested but they are listed as open issues and are not on as solid footing for possible removal. Thanks to everyone (especially Ian Bicking and Jacobian) for helping to identify stuff that should go.

I still have to do the list of modules to rename. I have the list but I need to move it to the PEP and organize them by why they are to be renamed. Might get that done tomorrow.