first week working on distutils2
#################################

:date: 2010-06-04
:category: tech

As I've been working on
`Distutils2 <http://hg.python.org/distutils2/>`_ during the past
week, taking part of the
`GSOC <http://code.google.com/intl/fr/soc/>`_ program, here is a
short summary of what I've done so far.

As my courses are not over yet, I've not worked as much as I
wanted, and this will continues until the end of June. My main
tasks are about making installation and uninstallation commands, to
have a simple way to install distributions via
`Distutils2 <http://hg.python.org/distutils2/>`_.

To do this, we need to rely on informations provided by the Python
Package Index (`PyPI <http://pypi.python.org/>`_), and there is at
least two ways to retreive informations from here: XML-RPC and the
"simple" API.

So, I've been working on porting some
`Distribute <http://bitbucket.org/tarek/distribute/>`_ related
stuff to `Distutils2 <http://hg.python.org/distutils2/>`_, cutting
off all non distutils' things, as we do not want to depend from
Distribute's internals. My main work has been about reading the
whole code, writing tests about this and making those tests
possible.

In fact, there was a need of a pypi mocked server, and, after
reading and introducing myself to the distutils behaviors and code,
I've taken some time to improve the work
`Konrad <http://bitbucket.org/konrad>`_ makes about this mock.

A PyPI Server mock
~~~~~~~~~~~~~~~~~~

The mock is embeded in a thread, to make it available during the
tests, in a non blocking way. We first used
`WSGI <http://wsgi.org>`_ and
`wsgiref <http://docs.python.org/library/wsgiref.html>`_ in order
control what to serve, and to log the requests made to the server,
but finally realised that
`wsgiref <http://docs.python.org/library/wsgiref.html>`_ is not
python 2.4 compatible (and we *need* to be python 2.4 compatible in
Distutils2).

So, we switched to
`BaseHTTPServer <http://docs.python.org/library/basehttpserver.html>`_
and
`SimpleHTTPServer <http://docs.python.org/library/simplehttpserver.html>`_,
and updated our tests accordingly. It's been an opportunity to
realize that `WSGI <http://wsgi.org>`_ has been a great step
forward for making HTTP servers, and expose a really simplest way
to discuss with HTTP !

You can find
`the modifications I made <http://bitbucket.org/ametaireau/distutils2/changesets>`_,
and the
`related docs <http://bitbucket.org/ametaireau/distutils2/src/tip/docs/source/test_framework.rst>`_
about this on
`my bitbucket distutils2 clone <http://bitbucket.org/ametaireau/distutils2/>`_.

The PyPI Simple API
~~~~~~~~~~~~~~~~~~~

So, back to the main problematic: make a python library to access
and request information stored on PyPI, via the simple API. As I
said, I've just grabbed the work made from
`Distribute <http://bitbucket.org/tarek/distribute/>`_, and played
a bit with, in order to view what are the different use cases, and
started to write the related tests.

The work to come
~~~~~~~~~~~~~~~~

So, once all use cases covered with tests, I'll rewrite a bit the
grabbed code, and do some software design work (to not expose all
things as privates methods, have a clear API, and other things like
this), then update the tests accordingly and write a documentation
to make this clear.

Next step is to a little client, as I've
`already started here <http://github.com/ametaireau/pypiclient>`_
I'll take you updated !