What will forever be exclusive to Python 3?

[2010-08-26: remove PEP 3109 and 3110 as they are both syntactically supported in Python 2.6
 2010-09-01: remove mention of built-ins returning iterators]

A question on Stack Overflow about what is exclusive to Python 3 came up and I realized that there is no clear list of big changes that you cannot access in Python 2.7 through a __future__ import. So I figured I would go through the What's New docs for Python 3.0, 3.1, and 3.2a1 (although the What's New doc has not been written yet) and see what has (not) been backported of significance.

If something is available in Python 2.6 without a __future__ import I will not list it here (e.g., new octal literals, bytes literal, and str.format). I also don't touch the C API. Otherwise stuff that is crossed out has been backported in Python 2.7 or is in Python 2.6 with a __future__ import. Everything else you have to make the switch to Python 3 to get the feature.

• Python 3.0
• print as a function
• from __future__ import print_function since Python 2.6.
• Views and iterators instead of lists
• future_builtins since Python 2.6 contains versions of built-ins that match Python 3 semantics, including returning iterators instead of lists
• Dicts gained equivalent view methods (e.g., dict.viewkeys()) in Python 2.7
• Ordering comparisons
• Int/long unification
• Unicode/bytes clarification
• from __future__ import unicode_literals helps since Python 2.6
• ... but only if you bother to rid your code of all references to str and move to unicode and bytes
• Standard library changes in regards to bytes/strings in Python 3 only
• Dict comprehensions
• Python 2.7
• Link is to a withdrawn PEP, but the general idea holds
• Set comprehensions
• Python 2.7
• Set literals
• Python 2.7
• Removal of old syntax
• exec as a function
• PEP 352: Required Superclass for Exceptions
• PEP 3102: keyword-only arguments
• PEP 3104: nonlocal
• PEP 3107: function annotations
• PEP 3108: Standard library reorganization
• PEP 3115: Metaclasses
• PEP 3132: extended iterable unpacking
• PEP 3134: Exception Chaining and Embedded Tracebacks
• PEP 3135: new super
• PEP 3137: memoryview
• Python 2.7
• Python 3.1
• PEP 372: ordered dictionary to collections
• Python 2.7
• PEP 378: Format Specifier for Thousands Separator
• Python 2.7
• Fields in format() can be auto-numbered
• E.g., "{} {}".format(2, 5) == "2 5"
• Might seem minor, but damn does it make a difference!
• Added in Python 2.7
• contextlib.nested no longer needed
• Another minor but handy feature
• Python 2.7
• Floats print their shortest representation
• Python 2.7
• tkinter.ttk
• Added as ttk in Python 2.7
• importlib
• Single function, importlib.import_module, in Python 2.7
• sysconfig
• Python 2.7
• io rewritten in C
• Python 2.7
• --with-computed-gotos
• Python 3.2a1
• GIL heavily reworked to perform better
• Bazillions of improvements to unittest
• Available externally as unittest2
• Updated in Python 2.7
• argparse
• Python 2.7
• PEP 391: dictionary-based configuration for logging
• Python 2.7
• PEP 3147: PYC repository directories
• ... bunch of stuff I probably overlooked from Misc/NEWS

So what does the list tell us? First, a ton of syntactic cleanup only appears in Python 3, which is not surprising. Second, there are still plenty of reasons, both from a development perspective but also a performance one, to look forward to moving over to Python 3 when you can.

Everyone should switch to Distribute and off of setuptools

[edit: mention how the fix in Python 2.6.3 may have not belonged in 2.6.3 since it is a micro release]

Python 2.6.3 has a couple of bugs still lingering that warrant a brown bag 2.6.4 release (should be out before the end of the month). One of the "bugs", and the entire reason I am doing this blog post, involves distutils and setuptools. Turns out that for Python 2.6.3 a change was made to distutils that broke setuptools for building extension modules. While the change that broke setuptools is being viewed as improper for a micro release (code shouldn't break in a micro release unless it was really bad semantics being fixed), it did bring to my attention that a lot of people do not know about Distribute.

A problem is that setuptools is no longer maintained. Luckily there is already a solution to this predicament that the wider Python community might not be fully aware of.  Tarek Ziadé forked setuptools and created Distribute, with its first release two months ago, explicitly to provide a library compatible with setuptools that is being actively maintained (and thus has bug fixes). Distribute is a drop-in replacement for setuptools, complete with being able to import it under the setuptools name so that everything will continue to work as if you had setuptools itself installed sans some bugs.

Because Distribute fixes bugs and is a backwards-compatible drop-in replacement I HIGHLY encourage people to change their installs of setuptools to Distribute. Everything will continue to work as Distribute installs itself under the setuptools name to maintain backwards-compatibility. Gentoo has even switched. Plus Distribute 0.6.3 supports Python 3.

So please, if you use setuptools then upgrade to Distribute.

One down, three to go

I managed to knock off another failing test for integrating importlib into Py3K; test_pkg. That leaves three: test_coding, test_pep263, and test_runpy. The first two have something to do with the BOM and probably how I am trying to figure out the encoding of the source file. Test_runpy has something to do with relative imports. Hopefully solving test_coding or test_pep263 will knock the other one off, meaning I only have two bugs left to deal with by June 1.

I hate PEP 263

PEP 263 is the one that allows one to specify the encoding of a Python source file. It is giving me such a headache with trying to bootstrap importlib. Because I must be able to open a source file with the proper encoding I must also have the entire codecs system working properly. This is a problem when the codecs system relies on imports to get to decoders.

Now the UTF-8 decoder is set up by default. But everything else, including ASCII, must be reachable through an import. That can be a problem when it is import that needs that module.

It is also a pain that one must open a file in a generic fashion, read the first two lines of a file, use a regex to try to find a specified encoding, and then reopen the file with the encoding found. That's a lot of stat calls and such that can be expensive. To deal with this and other bootstrapping issues I am going to have to expose some more C code in my special version of 3.0 in order to get this to work. Luckily it has partially been done for me thanks to imp.find_module() which does the proper file opening, albeit using the C import code. Once a pull it out to give me basically a custom source code open() function I should be able to move on to the next failing test.

Currently there are seven failing tests (albeit I can't run importlib properly anyway because of the PEP 263 issues). I have until early June to get this all done. Here is to hoping I can pull this off and properly delegate the stdlib reorg so that it all doesn't impact my personal life (which now includes looking for a new apartment for myself).

Transitioning from 2.6 to 3.0 is beginning to solidify

Guido began a discussion on python-dev earlier this week about how to backport the bytes type. This has led to some clarification on how he sees people transitioning their code which I figured some people might be interested in.

The current proposal on how to handle the bytes type is a good illustration of what the current plan is. Basically b'' and bytes will become direct aliases for '' and str, respectively. The idea is that instead of having to backport all of that code and worry about supporting it all at the C level, people get a simple way to syntactically state that they are moving (or have moved) code over for compatibility with 3.0. Makes our lives easier in terms of maintaining code while letting users have a simple transition.

And the use of the -3 flag has been clarified. Guido said he wants the flag to augment 2to3, not replicate what it does. That means that warnings raised by -3 should only be for stuff that 2to3 can't handle directly. That means that even if you code is warnings-free with the -3 flag turned on there is no guarantee your code will run properly as 2to3 will be needed to handle syntactic changes that you will not be warned about.

So where does that leave us in terms of expected steps to go from 2.6 code to 3.0? As of right now the steps look to be:

1. Run your code with normal warnings turned on. You should have no warnings in your code, including PendingDeprecationWarnings. Completing this step will make sure your code is good for the 2.x line. You should be doing this even if you don't care about 3.0.
   
2. Run with the -3 flag turned on. Any warnings that crop up now should also be dealt with. At this point your code should be syntactically written for 2.x but partially semantically ready for 3.0 (along with still working with 2.x).
   
3. Run 2to3 on your code. This should make it mostly syntactically ready for 3.0.
4. Run your unit tests. Any remaining porting issues should crop up at this point.

Obviously this is all a work-in-progress. Until 2.6 is out the door any of this could fluctuate. But I think it all looks reasonable.

Python 3.0a2 is out!

Go to the release page to download it. As Guido mentions in his blog post, probably the biggest change in this release is making the 'bytes' type immutable and adding bytearray as a mutable version of 'bytes'.

What to expect in Python 3.0a2

While Python 3.0a2 will most likely not be released by the end of tomorrow as Guido asked, that doesn't mean I can't tell you, dear reader, what the big changes are in the next alpha.

And really there is only one big change that is planned; the new bytes/buffer dichotomy. If you were to download Py3K right now you would notice that the old 'str' type has been renamed str8. It was left there for instances where transitioning over to the new 'str' (which is the old 'unicode', mind you) was either too painful at the time or how to handle the transition in that instance was not really clear.

But at some point between a1 and now, Guido thought about what it would be like if there was an immutable bytes type. Enough people liked the idea that the thought became an a2 goal.

And so that means str8 becomes bytes, bytes becomes buffer, and buffer becomes memoryview. Where does that leave us? Well, memoryview becomes the Python representation of the new buffer protocol as laid out in PEP 3118. The bytes type stays the same, but gets the name buffer as it is basically a buffer of raw bytes.

But now the name 'bytes' is for an immutable array of raw bytes. As discussed in PEP 3137, this means old uses of the old 'str' type can now be translated over to the new bytes type. So string literals get a 'b' tacked in front of them and becomes instances of the new bytes type. The common methods of strings (e.g., capitalize(), etc.) stick around for compatibility and because often times raw bytes from something is just ASCII. This should help with converting to Py3K from 2.x code as the basic semantics are all the same sans some methods. Most people will want to toss in some code that will decode the bytes into a str instance for printing, etc., but that is still optional.

The other big perks is stuff you get thanks to having an immutable type. You can now use raw bytes as keys in dicts as the bytes type is hashable. You can easily use them for comparison without worrying about it mutating from underneath you. It also means if you specify a bytes literal it will stay that way.

But the transition has not entirely happened yet. str8 is mutating into bytes slowly. Probably the biggest thing left is to rename the types and fix any failing tests. The remaining steps are outlined in this thread. As always help is appreciated.

Importlib update

Importlib has now been (roughly) bootstrapped into Py3K in my py3k-importlib branch. There are some tests that are still failing that I have not tried to fix yet. I also have not removed any C code yet so there is still a chance of some dependency that I don't know about yet.

And there is still the issue of 'warnings' not being a built-in module. Neal Norwitz wrote an initial C version of the critical stuff, but it isn't complete. The biggest issue is that it doesn't pick up anything that is set on 'warnings' itself (e.g., 'filters', 'showwarnings'). That's bad as various attributes in 'warnings' get set externally frequently. My proposed solution is to see if 'warnings' has been imported, and if so use the attributes from there, otherwise fall back on internal C stuff. That way the module is entirely independent of Python code and thus doesn't cause me any bootstrap issues.

Once 'warnings' is built-in I will be able to rip out C code to make sure this whole thing actually works. Then I can fix any bugs I have for importlib itself. That should be it at that point and I should hopefully be able to move my work into Py3K itself.

importlib ported to Py3K

I just finished porting importlib over to Py3K. I basically created a Py3K directory in importlib's sandbox and just copied the requisite files over, ran them through 2to3, and then made changes as needed (first in the original files when possible, and then Py3K-only changes).

The only Py3K-specific changes I had to make was for the bytes type. Stuff such as imp.get_magic() and reading bytecode in binary returns bytes and that just doesn't work for comparing against string literals or passing to ord(). =) I also had some integer division that should have been flooring instead of true division. But otherwise it's now converted.

Unfortunately if I make any changes to the original files I am going to have to forward-port it. What I think would be the best solution is have a script that copies the originals, run them through 2to3, and then apply Py3K-specific patches (which would be checked into svn). That way I can develop on the 2.x code and then just make the Py3K code an after-thought. If I have to tweak any code in the 2.x version I will probably bother with this, but not right now. =)

Giving a talk on Python 3.0a1 in Vancouver on September 4th

On the evening of September 4th I will be giving a presentation on Python 3.0a1 in Vancouver, BC for VanPyZ (see the "Next Talk" on the VanPyZ site for details). We sometimes go for drinks afterwards and just talk. Group has always been really nice and it should be fun!

-Brett

Python 3.0a1 is out!

Guido pushed out a source distribution of Python 3.0a1 this morning! You can read the web page about the details, but basically this is a real alpha release. =) So please give it a spin and let us know if you find bugs.

General update

For some reason I just felt like writing and nothing miraculous has happened in my personal life, which mean I will be scratching my writing itch here on my coding blog. Sorry PythonLand. =)

So Python 3.0a1 gets closer and closer. Lately Guido has been trying to stomp failing tests with various people. I personally fixed test_warnings to be re-entrant (__warningregistry__ being set on individual modules is evil!) today. Skip Montanaro (who has over 200 commits on me according to Ohloh) has been removing support for old OSs. I believe Eric Smith is the one implementing PEP 3101 (the new string formatting implementation). And with the Google sprint coming next week (which I am missing; first sprint I have missed since I joined python-dev) we might actually hit alpha before the month is out!

As for me, I have mostly been hacking on importlib. Once I got the API redesigned to make it easier to have different storage mechanisms for source code I felt hyper-motivated to code. Pretty much all week I have been re-implementing something like an importer or loader and then writing the unit tests the next night. At this point I have rewritten the extension and source file importers with tests. The extension module loader has been rewritten and the basic loading functionality is tested, but I just tossed in PEP 302 optional extension support and that is not tested. The source code loader has been rewritten but the methods needs for the source handler has not been implemented so it is not functional.

But boy has this rewrite simplified the code! I must have been on a slight functional programming kick when I first designed importlib as I was passing around a lot of objects throughout out the execution flow and then querying them for stuff. Now I have a base class and then provide the methods and attributes the base class needs to work. Much cleaner and easier to follow now.

I can't knock functional programming too much, though, as I am still passing key objects around. But I have just thought out more exactly what information can be provided up-front and what needs to be conditionally asked later. That has led to smaller APIs to implement with the trade-off of slightly large method parameters. But since the info is needed anyway, making the info parameters really is not a bad thing.

Overall I am very happy with how the rewrite is going. Granted I should have branched off my import_in_py directory in Python's sandbox, but it's not a big deal as I am just leaving the code in-place that I am making obsolete until I can do a complete switch-over. The real test in all of this, though, is how easy/hard rewriting zipimport is using the new API I came up with. If that turns out to not be overly painful then I think everyone should be happy with the basic support the package will provide for writing custom back-ends that use Python source/bytecode (I have stopped worrying about DSLs based on Python for now as I don't want to over-design and it can still be handled as the code that handles the source/bytecode dance is just a function that is set on the source loader).

The py3k-struni branch is dead!

Long live the py3k branch!

Yes, the main branch for Python 3.0 development is finally properly spelled. Guido made the struni branch the official dev branch this morning. If you happen to have any Py3K branches checked out, please use ``svn switch`` to your main checkout over to the new branch.

Help get 3.0a1 out the door!

As of this writing there are 11 failing tests left in the py3k-struni branch (the branch that is unifying str and unicode) which is holding up the alpha release of Python 3.0. Once those failing tests are fixed that branch will become the Py3K development branch (according to Guido's blog).

So if you have any interest in helping get Py3K out the door please help! Since all the failing tests are stdlib stuff you don't need to worry about digging into the core.

Helping out with the str/unicode unification

Guido has sent out an email asking for some help with getting tests to pass for the str/unicode unification work. If you are interested in helping please read this wiki page.

Python-Dev does care about the 2.x -> 3.0 transition

Martijn Faassen and others (and I do mean others; I am not singling out Martijn, just his blog post since it's easy to point to =) in the community seem to be suggesting as of late that python-dev is not paying enough attention to the difficulties people are going to face in porting code from Python 2.x to Python 3.0. It has led to both Collin and Doug posting on the topic.

And now I am going to speak out about this. I really wish people would not view python-dev and python-3000 as a bunch of nutty coders sitting in some ivory tower who just toss stuff out the window to the crowds below without ever looking. It feels like we are being cast as some nasty aristocrats who are just doing what is fun for us and not worrying enough about others who depend on our work.

If people are thinking that, they are just plain wrong. The first thing you need to remember (that Doug points out in his blog) is that we are also users (the Hair Club For Men tag line is now going off in my head). I don't know of a single active member of python-dev or python-3000 who doesn't have Python code to maintain. There is the entire stdlib that we have to maintain for both Python 2.6 and 3.0 (and trust me, that code uses every trick in the book and spans back to the 1.x days and it has managed to be transitioned). We all have the same worries as everyone else of making sure our code keeps working in Python 2.6 and Python 3.0.

Heck, I am probably one of the few people in this world who has written code from scratch to run on both Python 2.6 and Python 3.0 (my PEP 362 implementation that I did at PyCon). It wasn't bad at all (hardest thing was remembering whether I should use a 'print' statement or function for debugging). And I am about to embark on bootstrapping importlib. With tests, it is 2,394 lines of code according to sloccount. This code is developed for Python 2.6 but is going to be run under 3.0. And I can't cheat and dev in 3.0 since I hope to backport to 2.6. I am going to be very aware of transition cost with this project (although, I do admit upfront, the code is only for 2.6 as I have dependencies on some 2.6 stdlib code, but it could be ported to 2.5 probably without issue).

And people should not think that we don't listen to the community. Obviously people do or else I wouldn't be taking the time to do this blog post for people realize that we do.

But it is tough, as Collin's post shows, when people in the community start to comment like we don't care. It wears on you. People have been talking on python-3000 and python-dev about these issues for over a year now. You get to the point where you do just want to go "talk is cheap" and tell people to either help address their own issues are go away. It's not like everyone is like Guido and gets paid to work on Python. I don't even get to work on the core for my thesis anymore. It's volunteer time and having people complain about your volunteer work is an amazing downer and makes you either stop wanting to volunteer or become bitter and tell people to go away if they don't appreciate what you are doing. I really have to fight the latter from taking hold (the former won't happen because I enjoy working on Python too much). I constantly remind myself that these comments stems for general worries that people have and also from lack of info because not everyone follows development closely and so they go on what hits the blogosphere or c.l.py (it's one of the reasons I constantly try to make python-dev transparent and want to help make it easier to follow and join).

But we have had issues like this before. Python-dev knows how to do backwards-compatibility; I think our track record shows this.

And I think we have shown we care about the community. I personally did the python-dev Summaries for quite a while beyond what made them useful to me just because I knew the community found them useful and I care about the community.

And people also need to realize that this entire thing is an organic process. People are acting like Python 3.0 is releasing in a week along with 2.6 and everything is locked in stone for a shrink-wrapped box release. But we have not even reached alpha! Would you want someone coming to you and getting after you for what you planned to do when you had not even reached alpha yet? Heck, the word "alpha" has not even been uttered for 2.6 on python-dev yet.

The transition process is not in stone. Tools can change. New ideas can come forward. As I said, we are not even at alpha yet for 3.0 so the feature set is not even fully implemented which means needed changes to 2.6 have not occurred. And even when we do hit alpha, we don't have a ship date that we absolutely must hit. A perk of open source is that if a ship date slips, then it slips. If we come up against a problem in the transition process we can stop and deal with it, ship date be damned if necessary.

So please, don't pass judgement on the transition until we have released stuff and said, "go play". And even then, give us (constructive) feedback! Tools can change and we can continue to improve the process long after Python 3.0 and 2.6 ship. But please, if you are worried, just say you are worried but will wait to pass judgement until everything has been finalized and released and give python-dev the benefit of the doubt for now.

OK, I feel better now. =) I hope my points come across clearly. And I hope this doesn't spark some little flame war online; the whole point of this post is to prevent that. Hopefully I won't have to do another post like this again since feeling like you need to defend your volunteer efforts just plain sucks.

Responding to people's reactions to Guido's Py3K blog post

For those of you don't know, Guido did a blog post on the status of Py3K. I suspect most people who keep up on Python stuff know about it at this point as it went far enough to get picked up by LtU. And reactions have varied from positive to negative.

I am not here to tell you about Guido's plans as the blog post covers that. I am also not here to say whether I like his plans as he has been taking input from people for quite some time and I have been part of the feedback-providing group for about as long (I am a co-author of PEP 3100 so I have been helping to make public what Guido keeps in his head for three years come August). I have blogged before about how I am a fan of Py3K and thus am putting my python-dev time into it.

What I am going to try to do with this post is publicly discuss some of the questions and comments I have come across that Guido's post has generated. As always, if this post doesn't answer your question, leave a comment and I will try to respond. And an important note for people who want to check out the Py3K code and play around: what Guido posted about are what is planned, not what is necessarily in the code!

The most common comment I have seen is about the removal of reduce from the built-ins. Now first of all, realize it is just being moved to functools, not being removed entirely. Hopefully that will calm some people down. Nor is this a sign that Python is going to suddenly cut out more and more functional programming support (I think the addition of set comprehensions in Py3K shows that is not the case).

But I think two things are important to understand for this. One is that whether you have wrapped your head around reduce so well that you don't need to write out the 'for' loop equivalent, that does not mean everyone else has. A lot of the defenders of reduce have been saying that once you get it it's a great little function. That's nice, but the stuff that are built-ins should not require practice to comprehend for the majority of people, especially beginners if it can be helped.

Nor should they require you to be into a style of programming that Python is not explicitly designed for. A commenter in the LtU post stated how Python might support functional programming, but it is not designed for it. That is an important point and means that the decision will not always be made from an FP perspective.

I am sure someone is now going, "but what about map and filter, why do they get to stay, can't you use list comprehensions?" The answer to that is simplicity; ``map(fxn, stuff)`` is simpler than ``[fxn(x) for x in stuff]`` (plus map is faster). And both functions are easy to comprehend and so they get to stay. Plus they are used a ton more than reduce is.

I have seen some people not liking ABCs, but it's just a library, so you don't have to use it if you don't want. I suspect a lot of people will just use it as a set of nice mixins to get some methods for free in certain circumstances. But for those who have a use for ABCs they will most likely appreciate it.

And then there are the usual suggestions on how to change the language that has been heard time again (e.g., getting rid of defining 'self' as a parameter). You can see PEP 3099 for an official list of stuff that will not change. But also remember that the chance to submit a PEP for Python 3.0 has past (it as the end of April), so even if your idea is not in PEP 3099, if it requires a PEP it is too late for Python 3.0. There is always Python 3.1, though.

Finally, the transition. I think the transition plan for people is pretty damn good considering how much is changing. Regardless, the Python 2.x branch will exist for several years (I suspect Python 2.7 will be the last full version release but live as a bugfix release for longer than normal). But some people worry about the chicken-and-the-egg problem of big frameworks not switching and thus no one switching over to Py3K.

Yes, some people will never switch. That's fine, enjoy Python 2.x. But others will and continue to develop on Python 3.0 while leaving their 2.x code in the past. Plus stuff will start to be developed only on Python 3.0 that will act as "bait" to get people to switch. And you can probably get away with not going to Python 2.6-only conversion of your code. As long as you can run against Python 2.6 you should be able to convert your code; Python 2.6 is a target, not the base version to be compatible with.

In the end it all doesn't matter. Python 2.x is not going anywhere, so even if Py3K turns out to be a flop Python will live on. But if Py3K does do well (and I expect it will in the end), Python will be better for it. I know that I will end up developing for Py3K exclusively whenever I can and be happier for it. Hopefully most of you will end up being happier as well.

Deadline today for Py3K PEPs

Today is the day to get in those Py3K PEPs. In order to meet a sometime-in-June alpha for Python 3.0, a deadline for PEPs specifically affecting the release are due today. The only exemption is that of PEPs and such affecting the stdlib.


This week has already seen a flurry of new PEPs come in. PEPs 3121, 3122, and 3141 have been checked in. Plus there are roughly three or four currently being discussed on python-3000. The only benefit of all of these PEPs being out in the wild is that people are actually focusing their discussions and not going all over the place as tends to happen when there is debate over only a single PEP.

Why Python 3.0 (and the way we are going about it) is a good thing

On python-3000 there has been a flare-up of people worrying about porting their apps over to Python 3.0. At this point it is looking like Neal Norwitz's C implementation of the 'warnings' module will be combined with a special Py3K warning for when your code does something that differs between 2.6 and 3.0. It will have minimal cost and will be off by default. Combine that with the 2to3 refactoring tool and it should help ease the pain of moving code from 2.x to 3.x.

What has people worried, though, is release schedules. Python 3.0a1 should hopefully be out June or July of this year and with 3.0 final sometime in 2008. Python 2.6, if you follow typical 12-18 month delays between minor releases, should be out sometime in 2007. That puts the amount of time between 2.6 and 3.0 at less than a year. This has people worried.

People want their code to move over to 3.0 easily, which I understand. People with large codebases would probably be happy if we just incrementally changed the 2.x branch so that it just morphed into 3.0 in the future at some point. But the problem with that is it would delay reaching what has been planned for 3.0 by years. We cannot break more than one piece of syntax between minor versions without causing major transition headaches for people. And Guido doesn't want to wait years (and frankly neither do I).

This is why the decision was made to just go all-out and break all of the mistakes made over the years at once. It's like ripping off a bandage; you can either do it very slowly or just pull hard and get it over with. Unfortunately the people with the large chunks of code are saying that pulling hard is just too painful and would still take too long because of the amount of change.

And this is why people are worrying that 2.6 will not help transition well enough or fast enough before 3.0 comes out. That supposedly means that third-party libraries will not be available for 3.0 and thus nothing of value will be written in it, essentially making 3.0 still-born.

And this is where I disagree. It will take YEARS before people start coding in Py3K by default, let alone transitioning over in droves. 3.0 is most likely going to be an early-adopters release for those people who do not need to rely on third-party libraries. It will also be there to work out any issues that come up from the community. Guido has already stated that 3.1 will be based on feedback from 3.0 and he really doesn't expect people to really take until 3.2.

People who want to play with 3.0 will slowly transition there stuff over and then the third-party support will grow. And while all of this is happening the 2.x series will continue to gain features from the 3.x series to make transitioning easier until people just start to code in teh Py3K style naturally. And obviously people who write stuff from scratch that is not dependent on anything but the stdlib will continue to crop up.

In other words, I don't expect everyone to transition in a flash to Python 3.0 from 2.6. I expect it to be very slow. It might take several versions of Python before there is a bastardized 2.x release that has everything that 3.x has as well (or close to it) so that code can run in both versions of Python. This is not a race. It will be some time before various OSs ship with 3.x anyway, so there is no huge rush.

What could happen is apps fork at Python 2.5. They keep that version as their very backwards-compatible version and update for security reasons and such. With new features they can target 2.6 and up, helping them transition to 3.x at some point in the future. This might take some time, but whatever; 2.x will be maintained for quite a while.

But honestly, in the end, I don't really care if Py3K is never as popular as Python 2.x. As long as Py3K is still fun to develop I will continue to work on it.

stdlib reorg PEP has been added

I just emailed python-3000 an inlined copy of my stdlib reorg PEP (PEP 3108). The PEP should show up online the next time the cronjob for PEPs runs.

Hopefully this won't kick up too much heat for me since I am just trying to help get this thing done. But I am sure there are some suggested module removals people will not be happy with.
Question is whether enough people care about a specific removal to deem the module worth keeping.

Plus I have some suggested groupings of modules in the Open Issues section. Guido never explicitly cleared me to do groupings, but I was thinking of them anyway as I went through the stdlib and almost everyone who commented on my blog suggested the same thing, I figured I should at least toss the option out there for Guido to consider. We will see if he goes for any of them or not.