2007-03-05

Transient failures in unit tests are annoying

The buildbots for Python are a great thing. Being able to constantly run tests on a myriad of platforms has helped lower the number of bugs in Python.

But when a test fails because a socket connection timed out or a test just sometimes fails because of various other things outside of the code's control, it is rather annoying. I am thinking that a context manager along the lines of ``with failure_ok(IOErrr, errno=110): ...`` could wrap around calls that have external dependencies. If an exception is raised that matches what is specified, including attributes, then the test can just let itself skip the rest of the test as it was not really a failure but there is no way for the rest of the tests to continue. That should help cut down on the white noise coming from the buildbots.

On a related note, I just submitted a new recipe to the Python Cookbook. It is a context manager for dealing with a temporary file that you write to, use, and then want to delete. The context manager handle the first and last part of those steps for you. The 'with' statement is damn handy when it comes to testing. Essentially makes setUp/tearDown not worth a whole lot.