But I am obviously no longer part of the target audience. =) I have been hacking on core code since the fall of 2002 so I am rather removed from being a newbie. What I would love to have people leave a comment on this post about is if there is a specific piece of information that you have found really useful when you have worked on the core code base. And if you never have, what piece of info would you need in order to push you over the edge and into contributing to Python? Please be specific; saying "I don't know where to start" is both generic and a known issue (which I will cover in the tutorial).
I expect I will cover in the tutorial how to:
- Get a checkout of the code
- Coding standards
- Report a bug
- Review a bug
- Create a patch
- Review a patch
- Find stuff in the code base
- Find things to do (issue tracker, increasing code coverage, etc.)
- Tips on how best to develop (probably specific to areas such as the compiler, documentation, etc.)
Posts
22 comments:
The workflow for creating a patch that looked the most attractive was:
1. Create personal Mercurial repository
2. make changes
3. publish patch from repository
4. apply selected patches from Python main source control
* Testing requirements, or lack thereof.
* How to implement/add new functionality/features. (how to get them approved)
* How to get involved with other developers (such as yourself)
* Tools and tricks used by current core developers
Side note: have anyone thought about a contributor buddy system whereas people could "shadow" or be shadowed by someone more senior in the core dev process?
1) Ok, so I have this roundup item with a patch. It passes the tests, it resolves whatever problem, how do I know if I should commit it? It's relatively clear for obvious bugs or broken things, but entirely not for enhnacements or additional functionality. Some of these things obviously have some discussion related to them which has gone on and make them clear, and if not do we just try to make the discussion happen? It's particularly unclear for enhancements like the string.rsplit(), which sat for more than a year with discussion like "It's trivial to implement rsplit in-line, see:" and then went on to provide an implementation that was short but buggy.
2) I've decided to commit a patch. How do I tell if it should also be applied into one of the maint branches?
3) What kind of trouble will I get into if I commit something that really shouldn't have been? Am I going to get education or a reaming? I mean, I heard this story about this gun nut that lost his commit privs because of this, but I'm not talking anything quite so "obviously" bad. :-)
Mostly I came to some of my own answers on the above that worked out well, but were things that caused me a lot of thought when I last had a vacation.
Sean
Can your tutorial end up on the internet? This information is sort of there, but I've definitely been 'lurking' the python-dev mailing list too long. A convenient place to get started would be very helpful for those of us who are semi-motivated to contribute.
After "checkout the code" you should have
* compile
* run tests
My opnion is like kent, writed, how to setup the enviroment to play with the source (*.c and *.py).
Compilers, includes, tests, SCM (CVS/SVN/Hg or etc. ), all those in the 3 mains SO (Win, Lin and Mac).
IMHO is the first step, before the checkout.
Of course my view is based in my experience who is very new in C language.
IMHO, the enviroment to develop in Python is very easy to setup, but for C (like Python core) isn't obviously to know.
Very important things that would help:
* CPython object model
* Important files and codepaths through the interpreter
In "tactical thinking" one often wears "different hats", which simply resemble a different way to look at a problem.
Edward de bono gives each hat a different colour to reflect a different "strategy" to look at a specific problem - this way you can more easily explore other ways towards a problem.
In this specific situation of the python core I'd like a "newbie hat", one that explains what core developers do (their tricks) and exactly how they do so - and what is most important, why. And exactly about this "why", I'd like good reasoning so that i.e. I could understand too (I am quite slow and silly).
Documentation is not only about explaining API and reasoning but also about teaching other people (which is by the way why I get irritated when a few developers are simply to lazy to care for documentation).
Wouldn't a birds-eye view on the concepts and techniques used in the implementation be a good idea? (Or is the target audience supposed to already know these things?) I'm always much more stimulated by architectural issues than by mundane stuff like repositories and coding standards.
It would be really lovely if you're talks could be video'd and posted on google video in a downloadable format. I'm really interested in knowing more about the internals of python, but won't be able to make the sprint. Things that I'm very interested in learning more about are:
1)Data types used to represent python functions.(And ubiquitous internal data types in general)
2)A run down of the various options to automake that yield different builds of python, how to build minimal pythons and such.
So I was not expecting 10 comments to pop up on this without me noticing, so I am just going to do a patch reply to everyone so far.
@manuel
I am not going to cover using Hg for this as that would require me learning how to work with it in my workflow. I just don't have the time and Hg is not an "officially" supported tool for Python development.
@jesse
I added more info on what is expected for tests. I also added covering adding new features to varying levels of complexity. I already planned to cover little tricks. As for the buddy system, Titus has suggested it but that will have to wait until I start my crusade to lower the barrier of entry to developing on the core.
@anonymous (who is sean =)
I was not going to worry about covering commit privileges for people at the sprint as I expected everyone who had them would know what to do. But since people have been known to get privs at sprints I should cover it.
@sean fritz
It's up to the AV guys at the conference if this gets up on YouTube. I have no problem with it being videotaped, but it's going to be long and hands-on, so I am not sure how helpful it will be in video form. As I have said I plan to turn this tutorial into a series of documents that will go up at python.org/dev so even if the tutorial is not recorded the info will not be lost forever.
@kent, lucmult
I will cover how to checkout the code, compile, and run the test suite on Windows, Linux, and OS X.
@bwb
I was not planning on cover technical details on how the core works like the object model. That is something better taught by the C extension docs. But I will cover where to look for various files.
@anonymous
Right, and the tutorial is meant to get attendees to be junior core developers in a couple of hours.
@reggie
Maybe, but that is a bottomless pit of time. Plus the C extension docs cover the C details pretty well. I would rather spend my time getting people up to speed on how to work within the development team than on technical details that are already documented somewhere.
@alexander
I already covered the video thing in another part of this comment. Same with the data structures thing (although if people REALLY want this I can tack it on if there is time, but I can only do so much =). And I will cover some important configure options, but I won't go into too much detail as you should be running against a full-fledged build and not some minimal one.
For people not fluent in C, a list of books or readings on C dev in general, examples of good C, strategies for freeing, locking, etc.
I understand you can't give this good coverage, but you can at least point enthusiastic people in the right direction.
@jdunk
I am probably not the right person to be answering that question. I mean I learned C from K&R, Guy Steele's book, and python-dev. =) Plus I am sure some book or two has come about since I last bought a C book.
It would be great if there was a list of things to do, either in python or c. Jesse posted a comment about how to get features approved but i'm sure there is a long list of functionality that could be added to the standard modules even if it was just in python.
I've always thought some DNS tools would be a great addition to the standard library, but if there was a request for such code i think there would be more specific contributions.
In terms of the core I think a clear outline of what needs to be done would be a great addition. I think that sort of roadmap would make it much easier to help out.
@graham
I plan to not only cover how to get new things that people want in, but also where to go to find out what is needed.
This isn't exactly about python internals, but I put quite a bit of work into a a python compiler for parrot and managed to get pretty darn far before I decided not to mess with C. The thing is, I don't really want to mess with C at all. We have pyrex and pypy but none of that technology has really been incorporated into core python, as far as I can tell. Even the standard compiler package has a serious limitation in that it doesn't give you access to comments, so projects like bicycle repair man have to reinvent the wheel just to write a simple filter. I use python because I don't want to mess with ugly languages like C. Show me how to use python to help improve core aspects of python and I'd be much more interested in chipping in.
@michal
No, Pyrex and PyPy are not in the core for good reason. They are still experimental and have different release schedules.
As for the compiler package, you are trying to use the wrong tool for refactoring. For instance, the 2to3 tool contains a pure Python parser that does handle comments. This is why 2to3 can handle source-to-source translations.
In terms of improving Python by not touching C code, there are various ways. Ignoring the obvious improvement of tests and the stdlib, various code generation tools are in Python. And give me some time and hopefully import will be written almost entirely in Python.
It's very good if those infos is ready before Python Bug Day (next in 23th february).
http://wiki.python.org/moin/PythonBugDay
Maybe the initial parts (steps).
What you think about ?
@lucmult
I think you are being slightly optimistic that I will get it all done in time. =) If I manage to get my slides done early I will post them online for people to use.
I found this this HowTo on 'Writing Python Extensions' very useful. I see it as a good first introduction to the CPython API and some of the concepts involved. Hands-on experience always works best for me.
Did a video get made of this talk? I'd love to see it, even though I wasn't able to make it to PyCon.
@Seth No video, but I have written various documents that live at http://www.python.org/dev/ which cover most of this.
Post a Comment