With my web site consisting of a single page, I suspect some 404s are going to get served rather quickly. So I decided to go ahead and do a custom 404 page. Google Webmaster Tools has a handy 404 widget you can paste into your 404 page that gives suggestions on where the person probably meant to go, along with a site-specific search box. Google Analytics also has some custom 404 code to help you track the bad pages.
And all of this was done by just defining a 404.html template in Django. The only problem I hit was that I couldn't have nested blocks for my Google Analytics code. The 404 code between a normal page view and the 404 are arguments to a single JS function. I wanted the Google Analytics code to be in a block and then have a nested block for the arguments to the JS function. In the end I just stripped out the Google Analytics block and just went with the argument block. Hopefully I won't come across a need to not track a page.