blog.notmyidea.org/content/Technologie/2010-08-16-an-amazing-summer-of-code-working-on-distutils2.md

# An amazing summer of code working on distutils2


The [Google Summer of Code](http://code.google.com/soc/) I've spent
working on [distutils2](http://hg.python.org/distutils2/) is over. It
was a really amazing experience, for many reasons.

First of all, we had a very good team, we were 5 students working on
distutils2: [Zubin](http://zubin71.wordpress.com),
[Éric](http://wokslog.wordpress.com/),
[Josip](http://gsoc.djolonga.com/),
[Konrad](http://konryd.blogspot.com/) and me. In addition,
[Mouad](http://mouadino.blogspot.com/) have worked on the PyPI testing
infrastructure. You could find what each person have done on [the wiki
page of
distutils2](http://bitbucket.org/tarek/distutils2/wiki/GSoC_2010_teams).

We were in contact with each others really often, helping us when
possible (in \#distutils), and were continuously aware of the state of
the work of each participant. This, in my opinion, have bring us in a
good shape.

Then, I've learned a lot. Python packaging was completely new to me at
the time of the GSoC start, and I was pretty unfamiliar with python good
practices too, as I've been introducing myself to python in the late
2009.

I've recently looked at some python code I wrote just three months ago,
and I was amazed to think about many improvements to made on it. I guess
this is a good indicator of the path I've traveled since I wrote it.

This summer was awesome because I've learned about python good
practices, now having some strong
[mercurial](http://mercurial.selenic.com/) knowledge, and I've seen a
little how the python community works.

Then, I would like to say a big thanks to all the mentors that have
hanged around while needed, on IRC or via mail, and especially my mentor
for this summer, [Tarek Ziadé](http://tarek.ziade.org).

Thanks a lot for your motivation, your leadership and your cheerfulness,
even with a new-born and a new work\!

## Why ?

I wanted to work on python packaging because, as the time pass, we were
having a sort of complex tools in this field. Each one wanted to add
features to distutils, but not in a standard way.

Now, we have PEPs that describes some format we agreed on (see PEP 345),
and we wanted to have a tool on which users can base their code on,
that's [distutils2](http://hg.python.org/distutils2/).

## My job

I had to provide a way to crawl the PyPI indexes in a simple way, and do
some installation / uninstallation scripts.

All the work done is available in [my bitbucket
repository](http://bitbucket.org/ametaireau/distutils2/).

### Crawling the PyPI indexes

There are two ways of requesting informations from the indexes: using
the "simple" index, that is a kind of REST index, and using XML-RPC.

I've done the two implementations, and a high level API to query those
twos. Basically, this supports the mirroring infrastructure defined in
PEP 381. So far, the work I've done is gonna be used in pip (they've
basically copy/paste the code, but this will change as soon as we get
something completely stable for distutils2), and that's a good news, as
it was the main reason for what I've done that.

I've tried to have an unified API for the clients, to switch from one to
another implementation easily. I'm already thinking of adding others
crawlers to this stuff, and it was made to be extensible.

If you want to get more informations about the crawlers/PyPI clients,
please refer to the distutils2 documentation, especially [the pages
about
indexes](http://distutils2.notmyidea.org/library/distutils2.index.html).

You can find the changes I made about this in the
[distutils2](http://hg.python.org/distutils2/) source code .

### Installation / Uninstallation scripts

Next step was to think about an installation script, and an uninstaller.
I've not done the uninstaller part, and it's a smart part, as it's
basically removing some files from the system, so I'll probably do it in
a near future.

[distutils2](http://hg.python.org/distutils2/) provides a way to install
distributions, and to handle dependencies between releases. For now,
this support is only about the last version of the METADATA (1.2) (See,
the PEP 345), but I'm working on a compatibility layer for the old
metadata, and for the informations provided via PIP requires.txt, for
instance.

### Extra work

Also, I've done some extra work. this includes:

  - working on the PEP 345, and having some discussion about it (about
    the names of some fields).
  - writing a PyPI server mock, useful for tests. you can find more
    information about it on the
    [documentation](http://distutils.notmyidea.org).

## Futures plans

As I said, I've enjoyed working on distutils2, and the people I've met
here are really pleasant to work with. So I *want* to continue
contributing on python, and especially on python packaging, because
there is still a lot of things to do in this scope, to get something
really usable.

I'm not plainly satisfied by the work I've done, so I'll probably tweak
it a bit: the installer part is not yet completely finished, and I want
to add support for a real
[REST](http://en.wikipedia.org/wiki/Representational_State_Transfer)
index in the future.

We'll talk again of this in the next months, probably, but we definitely
need a real
[REST](http://en.wikipedia.org/wiki/Representational_State_Transfer) API
for [PyPI](http://pypi.python.org), as the "simple" index *is* an ugly
hack, in my opinion. I'll work on a serious proposition about this,
maybe involving [CouchDB](http://couchdb.org), as it seems to be a good
option for what we want here.

## Issues

I've encountered some issues during this summer. The main one is that's
hard to work remotely, especially being in the same room that we live,
with others. I like to just think about a project with other people, a
paper and a pencil, no computers. This have been not so possible at the
start of the project, as I needed to read a lot of code to understand
the codebase, and then to read/write emails.

I've finally managed to work in an office, so good point for home/office
separation.

I'd not planned there will be so a high number of emails to read, in
order to follow what's up in the python world, and be a part of the
community seems to takes some times to read/write emails, especially for
those (like me) that arent so confortable with english (but this had
brought me some english fu \!).

## Thanks \!

A big thanks to [Graine Libre](http://www.graine-libre.fr/) and [Makina
Corpus](http://www.makina-corpus.com/), which has offered me to come
into their offices from time to time, to share they cheerfulness \! Many
thanks too to the Google Summer of Code program for setting up such an
initiative. If you're a student, if you're interested about FOSS, dont
hesitate any second, it's a really good opportunity to work on
interesting projects\!