Alexis' log

Introducing cornice

2011-12-06T00:00:00+01:00

Wow, already my third working day at mozilla. Since Monday, I've been working with Tarek Ziadé, on a pyramid REST-ish toolkit named cornice.

Its goal is to take all the hard bits appart from you when implementing a web service, so you can focus on what's important. Cornice provides you facilities for validation of any kind.

The goal is to simplify your work, but we don't want to reinvent the wheel, so it is easily pluggable with validations frameworks, such as Collander.

Handling errors and validation

We have changed the way errors are handled. Here is how it works:

service = Service(name="service", path="/service")


def is_awesome(request):
    if not 'awesome' in request.GET:
        request.errors.add('body', 'awesome', 'You lack awesomeness!')


@service.get(validator=(is_awesome))
def get1(request):
    return {"test": "succeeded"}

All the errors collected during the validation process, or after, are collected before returning the request. If any, a error 400 is fired up, with the list of problems encoutred encoded as a nice json list (we plan to support multiple formats in the future)

As you might have seen, request.errors.add takes three parameters: location, name and description.

location is where the error arised. It can either be "body", "query", "headers" or "path". name is the name of the variable causing problem, if any, and description contains a more detailled message.

Here is an example of a malformed request:

$ # run a demo app

To describe a web service in cornice, you have to write something like this

System Message: ERROR/3 (./content/mozilla/introducing-cornice.rst, line 54)

Content block expected for the "code-block" directive; none found.

.. code-block:: python

Quels usages pour l'informatique ?

2011-12-01T00:00:00+01:00

Quand on termine ses études, on s'en pose un tas, des questions. Sur le métier que l'on veut faire, sur ce que ça signifie, sur le sens et la valeur du travail. Et j'en suis arrivé à faire un constat simple: l'informatique, c'est utile, tant que ça ne viens pas vous pourrir la vie. Oui, parce que de l'informatique on en a partout, des "geeks" et des "accros" aussi, et que ça vient s'immiscer dans nos vies même quand d'autres moyens ou médias sont plus utiles ou pertinents.

Certes, l'informatique nous permet de mieux communiquer et de mieux travailler. Mais à quel prix ? ce n'est pas parce qu'il est "possible" d'industrialiser l'éducation (ou l'agriculture !), que l'on doit le faire. Oui, ça me dérange d'être une des nombreuses personnes à l'œuvre derrière cette soit disant "révolution", qui n'est pas toujours pour le meilleur. Attention, je ne remets pas l'informatique et son intérêt en cause: je me pose des questions quand à la place que je veux lui donner et la place que je souhaites occuper dans son évolution. Ce n'est pas parce qu'on peut tuer avec un marteau (avec un peu de volonté) qu'il s'agit d'un mauvais outil, mais si tout le monde se met à tuer avec des marteaux (y a des malades partout, hein), alors se poser la question de son rôle, en tant que fabricant de marteaux me semble nécessaire (oui, je vous l'accorde, on aura vu des comparaisons plus perspicaces).

Donc: à partir de quel moment l'informatique cesse d'être un outil utile pour transformer nos modes de vies d'une manière qui me dérange ? Peut être avec son arrivée sur des périphériques mobiles ? Peut être quand elle se fait l'instrument du consumérisme et de l'individualisme.

Et alors, on fait quoi ?

Mais si je continue à faire de l'informatique, il y à bien des raison. J'ai d'ailleurs trouvé mon intérêt de par le coté collaboratif qui est permis et développé par l'outil informatique, et notamment par le réseau des réseaux (internet). Faisons ensemble, mes amis. Prouvons que la collaboration à de meilleurs jours à vivre que la compétition. Le web, notamment, est une avancée majeure en ce qui concerne la liberté d'expression et le partage de connaissances (oui, kipédia). Je vous conseille d'ailleurs à ce propos l'excellent discours tenu par Bernard Stiegler paru recemment sur owni.

Et c'est cet avenir qu'il me plait de défendre: l'ouverture d'esprit, la possibilité que chacun puisse contribuer et participer à une base de savoir commune, en apprenant des autres. Mais par pitié, n'imposons pas la technologie là ou elle n'est pas nécessaire, et utilisons là avec tact quand elle peut nous être profitable.

Il me plait de repenser l'informatique comme outil et non plus comme mode de vie. Faisons le l'outil de la collaboration. À l'école, apprenons à nos enfants à collaborer, à susciter le partage, pas uniquement avec l'outil informatique, mais aussi avec celui ci, tout en leurs apprenant à avoir un regard critique sur les informations qu'il reçoivent.

En bref, questionner le rôle que l'on souhaites avoir dans notre société par le biais de l'informatique est nécessaire. Comme d'autres, je suis arrivé à l'informatique par le biais du premier ordinateur familial, il y a de ça une bonne quinzaine d'années. Ça intrigue, on touche un peu à tout (on en fait des conneries !) et on finit par apprendre/comprendre comment ça marche, petit à petit. Cette curiosité n'est d'ailleurs pas le propre de l'informatique puisqu'on la retrouve dans la cuisine, dans le bricolage et dans un tas de domaines de notre vie quotidienne.

Finalement, c'est aimer bidouiller, et comprendre comment ça fonctionne, quitte à sortir les compétences de leur domaine de prédilection (qui à dit que l'informatique ne pouvait être artistique ?) Le mouvement hacker (bidouilleurs) aime à sortir l'informatique de son carcan et l'appliquer ailleurs.

C'est de cette manière que j'ai envie de considérer mon métier, qui avant tout est une passion. Je suis un bidouilleur, j'aime découvrir comment les choses fonctionnent et avoir une panoplie d'outils qui me permettent de répondre à des besoins réels.

Favoriser la collaboration

Et donc, en tant qu'individu, pourquoi faire de l'informatique ? Qu'est-ce qui m'attire dans cet outil ?

Ce qu'on pourrait qualifier de "recherche fondamentale", l'écriture de bibliothèques logicielles, est important mais n'est pas tout. Ce qui importe ce sont les usages qui en découlent. Je souhaite savoir écrire des outils qui sont utiles, qui favorisent la collaboration et participent à l'ouverture des esprits.

Je choisis de faire de l'informatique pour créer les outils qui répondent à des problématiques réelles, pour trouver de meilleures manières de communiquer et de travailler ensemble. Mais, comme me le disait David, d'Outils-Réseaux, on ne crée pas de la coopération: rien ne sert d'essayer de faire coopérer des gens qui ne veulent pas. On peut, cependant, la faciliter, en utilisant les bons outils et en formant les gens à leur utilisation, ainsi qu'aux pratiques collaboratives (qui, je le répète, ne s'arrêtent pas du tout aux frontières informatique).

Le logiciel libre, avant d'être une force pour le marché logiciel, est une application du partage. Une démonstration qu'il est possible de travailler ensemble pour fabriquer quelque chose de fonctionnel et d'utile pour tous. Une sorte d'antithèse de ce modèle capitaliste incarné par les brevets logiciel.

A plusieurs reprises, j'ai été bluffé par la réalité du logiciel libre. Oui, il est facile de collaborer lorsqu'on crée un logiciel, pour peu qu'on explique les tenants et les aboutissants aux participants. Les contributeurs sortent d'on ne sait ou, pour peu que le projet leur soit utile. Je ne parles pas d'outils "corpo compliant" (bien que ça soit probablement aussi le cas), mais d'outils que j'ai pu développer pour mon propre usage, et sur lesquels il à été possible de collaborer avec d'autres.

Parce que l'informatique est utile dans bien des milieux, parce qu'elle peut être (et elle l'est) un vecteur de participation et de collaboration, défendons les valeurs qui nous sont chères (logiciels libres et ouverts!) et construisons des ponts entre les initiatives qui nous parlent (dans mon cas ça parles de fermes autogérées, et d'initiatives d'éducation populaire) et l'informatique. Faisons en sorte de rendre l'informatique accessible et utile dans les milieux ou elle peut apporter quelque chose !

How are you handling your shared expenses?

2011-10-15T00:00:00+02:00

TL;DR: We're kick-starting a new application to manage your shared expenses. Have a look at http://ihatemoney.notmyidea.org

As a student, I lived in a lot of different locations, and the majority of them had something in common: I lived with others. It usually was a great experience (and I think I will continue to live with others). Most of the time, we had to spend some time each month to compute who had to pay what to the others.

I wanted to create a pet project using flask, so I wrote a little (~150 lines) flask application to handle this. It worked out pretty well for my housemates and me, and as we had to move into different locations, one of them asked me if he could continue to use it for the year to come.

I said yes and gave it some more thoughts: We probably aren't the only ones interested by such kind of software. I decided to extend a bit more the software to have a concept of projects and persons (the list of persons was hard-coded in the first time, boooh!).

I then discussed with a friend of mine, who was excited about it and wanted to learn python. Great! That's a really nice way to get started. Some more friends were also interested in it and contributed some features and provided feedback (thanks Arnaud and Quentin!)

Since that, the project now support multiple languages and provides a REST API (android and iphone apps in the tubes!), into other things. There is no need to register for an account or whatnot, just enter a project name, a secret code and a contact email, invite friends and that's it (this was inspired by doodle)!

You can try the project at http://ihatemoney.notmyidea.org for now, and the code lives at https://github.com/spiral-project/ihatemoney/.

Features

In the wild, currently, there already are some implementations of this shared budget manager thing. The fact is that most of them are either hard to use, with a too much fancy design or simply trying to do too much things at once.

No, I don't want my budget manager to make my shopping list, or to run a blog for me, thanks. I want it to let me focus on something else. Keep out of my way.

No user registration

You don't need to register an account on the website to start using it. You just have to create a project, set a secret code for it, and give both the url and the code to the people you want to share it with (or the website can poke them for you).

Keeping things simple

"Keep It Simple, Stupid" really matches our philosophy here: you want to add a bill? Okay. Just do it. You just have to enter who paid, for who, how much, and a description, like you would have done when you're back from the farmer's market on raw paper.

No categories

Some people like to organise their stuff into different "categories": leisure, work, eating, etc. That's not something I want (at least to begin with).

I want things to be simple. Got that? Great. Just add your bills!

Balance

One of the most useful thing is to know what's your "balance" compared to others. In other words, if you're negative, you owe money, if you're positive, you have to receive money. This allows you to dispatch who has to pay for the next thing, in order to re-equilibrate the balance.

Additionally, the system is able to compute for you who has to give how much to who, in order to reduce the number of transactions needed to restore the balance.

API

All of what's possible to do with the standard web interface is also available through a REST API. I developed a simple REST toolkit for flask for this (and I should release it!).

Interested?

This project is open source. All of us like to share what we are doing and would be happy to work with new people and implement new ideas. If you have a nice idea about this, if you want to tweak it or to fill bugs. Don't hesitate a second! The project lives at http://github.com/spiral-project/ihatemoney/

La simplicité volontaire contre le mythe de l'abondance

2011-10-14T00:00:00+02:00

Paul Ariès dans la préface de "La simplicité volontaire contre le mythe de l'abondance"

Je sais aussi qu'à force d'être seulement contre, nous finissons par être "tout contre", tout contre le système, ses débats pourris, ses pratiques immondes, ses pervertions mentales. Cessons de porter notre adversaire sur notre dos, car il finira par nous poignarder, au moment ou nous pensions justement pouvoir le vaincre… Entendons-nous bien, il ne s'agit pas d'apprendre à "positiver" sous prétexte que l'on pourrait "moraliser" ou repeindre en vert le capitalisme, il s'agit, bien au contraire, d'être aussi intraitables lorsque nous créons cet autre monde, que lorsque nous étions ces militants purs mais un peu durs à jouir, il s'agit d'être aussi indomptables mais sans oublier que le capitalisme ne pourra être arrété que par du non-capitaliste… c'est-à-dire en multipliant les expérimentations sociales, en acceptant de faire au maximum sécession, en inventant (réinventant?) une gauche buissonière et maquisarde, en faisant, au quotidien, la part belle à la dérive, à des modes de comportement expérimentaux. Nous ne devons plus accepter de faire de la politique aux conditions de nos adversaires, car c'est la seule façon d'éviter le pessimisme, le ressentiment, une désensibilisation croissante et… la mort lente.

Using dbpedia to get languages influences

2011-08-16T00:00:00+02:00

While browsing the Python's wikipedia page, I found information about the languages influenced by python, and the languages that influenced python itself.

Well, that's kind of interesting to know which languages influenced others, it could even be more interesting to have an overview of the connexion between them, keeping python as the main focus.

This information is available on the wikipedia page, but not in a really exploitable format. Hopefully, this information is provided into the information box present on the majority of wikipedia pages. And… guess what? there is project with the goal to scrap and index all this information in a more queriable way, using the semantic web technologies.

Well, you may have guessed it, the project in question in dbpedia, and exposes information in the form of RDF triples, which are way more easy to work with than simple HTML.

For instance, let's take the page about python: http://dbpedia.org/page/Python_%28programming_language%29

The interesting properties here are "Influenced" and "InfluencedBy", which allows us to get a list of languages. Unfortunately, they are not really using all the power of the Semantic Web here, and the list is actually a string with coma separated values in it.

Anyway, we can use a simple rule: All wikipedia pages of programming languages are either named after the name of the language itself, or suffixed with "( programming language)", which is the case for python.

So I've built a tiny script to extract the information from dbpedia and transform them into a shiny graph using graphviz.

After a nice:

$ python get_influences.py python dot | dot -Tpng > influences.png

The result is the following graph (see it directly here)

While reading this diagram, keep in mind that it is a) not listing all the languages and b) keeping a python perspective.

This means that you can trust the scheme by following the arrows from python to something and from something to python, it is not trying to get the matching between all the languages at the same time to keep stuff readable.

It would certainly be possible to have all the connections between all languages (and the resulting script would be easier) to do so, but the resulting graph would probably be way less readable.

You can find the script on my github account. Feel free to adapt it for whatever you want if you feel hackish.

Pelican, 9 months later

2011-07-25T00:00:00+02:00

Back in October, I released pelican, a little piece of code I wrote to power this weblog. I had simple needs: I wanted to be able to use my text editor of choice (vim), a vcs (mercurial) and restructured text. I started to write a really simple blog engine in something like a hundred python lines and released it on github.

And people started contributing. I wasn't at all expecting to see people interested in such a little piece of code, but it turned out that they were. I refactored the code to make it evolve a bit more by two times and eventually, in 9 months, got 49 forks, 139 issues and 73 pull requests.

Which is clearly awesome.

I pulled features such as translations, tag clouds, integration with different services such as twitter or piwik, import from dotclear and rss, fixed a number of mistakes and improved a lot the codebase. This was a proof that there is a bunch of people that are willing to make better softwares just for the sake of fun.

Thank you, guys, you're why I like open source so much.

Using JPype to bridge python and Java

2011-06-11T00:00:00+02:00

Java provides some interesting libraries that have no exact equivalent in python. In my case, the awesome boilerpipe library allows me to remove uninteresting parts of HTML pages, like menus, footers and other "boilerplate" contents.

Boilerpipe is written in Java. Two solutions then: using java from python or reimplement boilerpipe in python. I will let you guess which one I chosen, meh.

JPype allows to bridge python project with java libraries. It takes another point of view than Jython: rather than reimplementing python in Java, both languages are interfacing at the VM level. This means you need to start a VM from your python script, but it does the job and stay fully compatible with Cpython and its C extensions.

First steps with JPype

Once JPype installed (you'll have to hack a bit some files to integrate seamlessly with your system) you can access java classes by doing something like that:

import jpype
jpype.startJVM(jpype.getDefaultJVMPath())

# you can then access to the basic java functions
jpype.java.lang.System.out.println("hello world")

# and you have to shutdown the VM at the end
jpype.shutdownJVM()

Okay, now we have a hello world, but what we want seems somehow more complex. We want to interact with java classes, so we will have to load them.

Interfacing with Boilerpipe

To install boilerpipe, you just have to run an ant script:

$ cd boilerpipe
$ ant

Here is a simple example of how to use boilerpipe in Java, from their sources

package de.l3s.boilerpipe.demo;
import java.net.URL;
import de.l3s.boilerpipe.extractors.ArticleExtractor;

public class Oneliner {
    public static void main(final String[] args) throws Exception {
        final URL url = new URL("http://notmyidea.org");
        System.out.println(ArticleExtractor.INSTANCE.getText(url));
    }
}

To run it:

$ javac -cp dist/boilerpipe-1.1-dev.jar:lib/nekohtml-1.9.13.jar:lib/xerces-2.9.1.jar src/demo/de/l3s/boilerpipe/demo/Oneliner.java
$ java -cp src/demo:dist/boilerpipe-1.1-dev.jar:lib/nekohtml-1.9.13.jar:lib/xerces-2.9.1.jar de.l3s.boilerpipe.demo.Oneliner

Yes, this is kind of ugly, sorry for your eyes. Let's try something similar, but from python

import jpype

# start the JVM with the good classpaths
classpath = "dist/boilerpipe-1.1-dev.jar:lib/nekohtml-1.9.13.jar:lib/xerces-2.9.1.jar"
jpype.startJVM(jpype.getDefaultJVMPath(), "-Djava.class.path=%s" % classpath)

# get the Java classes we want to use
DefaultExtractor = jpype.JPackage("de").l3s.boilerpipe.extractors.DefaultExtractor

# call them !
print DefaultExtractor.INSTANCE.getText(jpype.java.net.URL("http://blog.notmyidea.org"))

And you get what you want.

I must say I didn't thought it could work so easily. This will allow me to extract text content from URLs and remove the boilerplate text easily for infuse (my master thesis project), without having to write java code, nice!

Un coup de main pour mon mémoire!

2011-05-25T00:00:00+02:00

Ça y est, bientôt la fin. LA FIN. La fin des études, et le début du reste. En attendant je bosse sur mon mémoire de fin d'études et j'aurais besoin d'un petit coup de main.

Mon mémoire porte sur les systèmes de recommandation. Pour ceux qui connaissent last.fm, je fais quelque chose de similaire mais pour les sites internet: en me basant sur ce que vous visitez quotidiennement et comment vous le visitez (quelles horaires, quelle emplacement géographique, etc.) je souhaites proposer des liens qui vous intéresseront potentiellement, en me basant sur l'avis des personnes qui ont des profils similaires au votre.

Le projet est loin d'être terminé, mais la première étape est de récupérer des données de navigation, idéalement beaucoup de données de navigation. Donc si vous pouvez me filer un coup de main je vous en serais éternellement reconnaissant (pour ceux qui font semblant de pas comprendre, entendez "tournée générale").

J'ai créé un petit site web (en anglais) qui résume un peu le concept, qui vous propose de vous inscrire et de télécharger un plugin firefox qui m'enverra des information sur les sites que vous visitez (si vous avez l'habitude d'utiliser chrome vous pouvez considérer de switcher à firefox4 pour les deux prochains mois pour me filer un coup de main). Il est possible de désactiver le plugin d'un simple clic si vous souhaitez garder votre vie privée privée ;-)

Le site est par là: http://infuse.notmyidea.org. Une fois le plugin téléchargé et le compte créé il faut renseigner vos identifiants dans le plugin en question, et c'est tout!

A votre bon cœur ! Je récupérerais probablement des données durant les 2 prochains mois pour ensuite les analyser correctement.

Merci pour votre aide !

Travailler moins pour mieux travailler ?

2011-05-19T00:00:00+02:00

Je viens de passer une semaine et demi quasiment hors-ligne et je dois dire que je suis assez impressionné du résultat: je suis de retour chez mes parents pour le "easter break" et j'en ai profité pour donner un coup aux travaux de la maison (et pour me reposer un brin!).

Bosser en extérieur est un réel plaisir et faire quelque chose de manuel également. Je n'ai pas pour habitude de bricoler autre chose que du logiciel et c'est vraiment quelque chose que j'apprécie.

J'avoue, je mens un peu quand je dis que j'étais complètement déconnecté: j'ai vérifié mes mails assez régulièrement en utilisant mon téléphone (merci Arnaud pour le prêt du gadget!) et j'ai limité ma présence web au strict minimum. Je veux dire par là pas de twitter, pas d'IRC et autres mailing-lists.

Quand je dis hors-ligne, je ne dis pas que je n'ai pas travaillé sur mon ordinateur. J'ai à fournir beaucoup plus de travail que ce que je ne pensait en premier lieu pour la rédaction de mon mémoire et j'ai passé quelques heures par ci par là à lire des articles et livres sur le sujet ainsi que rédigé une bonne partie de mon mémoire durant ces 10 jours. Résultat ? Les heures que j'ai passées à travailler ont été étonnement plus productives que celles que j'ai l'habitude de passer derrière un écran. Je ne parles pas uniquement du fait de procrastiner; évidemment c'est une des causes principales de ma perte de productivité, mais je pense également au fait de laisser le cerveau reposer, au moins en ce qui concerne la partie informatique. Bricoler demande de la concentration et de la réflexion mais permet également de laisser son esprit vagabonder. J'ai pu donc avancer mes sur certaines parties de mon mémoire alors que j'étais en train de poser le bardage sur la garage par exemple.

Passer du temps sur autre chose (qui demandait également de la concentration par ailleurs) m'a permis d'être réellement plus efficace lors de mes phases de rédaction et de lecture.

Je me demande depuis quelques temps quel sera mon futur travail et quelle forme il prendra. Éééh oui, la fin des études arrive pour la fin de l'année, c'est pas une blague. Et je me pose sérieusement la question du travail aux 3/4 temps. La raison principale étant que je ne souhaites pas passer la majeure partie de mon temps à travailler derrière un écran et que la formation que j'ai m'y rattache beaucoup. Ne me comprenez pas de travers: j'aime ce que je fais; mais j'aime aussi ce que je fais à coté de l'informatique: associatif, projets alternatifs, permaculture et autres.

Viens s'ajouter à ça le fait d'avoir une qualité de vie qui me semble pouvoir passer par le fait de travailler moins ("il est marant ce gamin, il à pas encore commencé à travailler qu'il veut bosser moins, regarde moi cette feignasse!") et travailler plus efficacement. Bien sur, on n'est jamais 100% productif et c'est d'autant plus vrai pour moi, alors si gagner en productivité peut passer par travailler moins, pourquoi pas s'y essayer !

Peut être que vous pratiquez déjà le travail au 3/4 temps, que vous avez des retours d'expérience à faire : fausse bonne idée ? Vraie bonne idée ?

Analyse users' browsing context to build up a web recommender

2011-04-01T00:00:00+02:00

No, this is not an april's fool ;)

Wow, it's been a long time. My year in Oxford is going really well. I realized few days ago that the end of the year is approaching really quickly. Exams are coming in one month or such and then I'll be working full time on my dissertation topic.

When I learned we'll have about 6 month to work on something, I first thought about doing a packaging related stuff, but finally decided to start something new. After all, that's the good time to learn.

Since a long time, I'm being impressed by the last.fm recommender system. They're scrobbling the music I listen to since something like 5 years now and the recommendations they're doing are really nice and accurate (I discovered a lot of great artists listening to the "neighbour radio".) (by the way, here is my lastfm account)

So I decided to work on recommender systems, to better understand what is it about.

Recommender systems are usually used to increase the sales of products (like Amazon.com does) which is not really what I'm looking for (The one who know me a bit know I'm kind of sick about all this consumerism going on).

Actually, the most simple thing I thought of was the web: I'm browsing it quite every day and each time new content appears. I've stopped to follow my feed reader because of the information overload, and reduced drastically the number of people I follow on twitter.

Too much information kills the information.

You shall got what will be my dissertation topic: a recommender system for the web. Well, such recommender systems already exists, so I will try to add contextual information to them: you're probably not interested by the same topics at different times of the day, or depending on the computer you're using. We can also probably make good use of the way you browse to create groups into the content you're browsing (or even use the great firefox4 tab group feature).

There is a large part of concerns to have about user's privacy as well.

Here is my proposal (copy/pasted from the one I had to do for my master)

Introduction and rationale

Nowadays, people surf the web more and more often. New web pages are created each day so the amount of information to retrieve is more important as the time passes. These users uses the web in different contexts, from finding cooking recipes to technical articles.

A lot of people share the same interest to various topics, and the quantity of information is such than it's really hard to triage them efficiently without spending hours doing it. Firstly because of the huge quantity of information but also because the triage is something relative to each person. Although, this triage can be facilitated by fetching the browsing information of all particular individuals and put the in perspective.

Machine learning is a branch of Artificial Intelligence (AI) which deals with how a program can learn from data. Recommendation systems are a particular application area of machine learning which is able to recommend things (links in our case) to the users, given a particular database containing the previous choices users have made.

This browsing information is currently available in browsers. Even if it is not in a very usable format, it is possible to transform it to something useful. This information gold mine just wait to be used. Although, it is not as simple as it can seems at the first approach: It is important to take care of the context the user is in while browsing links. For instance, It's more likely that during the day, a computer scientist will browse computing related links, and that during the evening, he browse cooking recipes or something else.

Page contents are also interesting to analyse, because that's what people browse and what actually contain the most interesting part of the information. The raw data extracted from the browsing can then be translated into something more useful (namely tags, type of resource, visit frequency, navigation context etc.)

The goal of this dissertation is to create a recommender system for web links, including this context information.

At the end of the dissertation, different pieces of software will be provided, from raw data collection from the browser to a recommendation system.

Background Review

This dissertation is mainly about data extraction, analysis and recommendation systems. Two different research area can be isolated: Data preprocessing and Information filtering.

The first step in order to make recommendations is to gather some data. The more data we have available, the better it is (T. Segaran, 2007). This data can be retrieved in various ways, one of them is to get it directly from user's browsers.

Data preparation and extraction

The data gathered from browsers is basically URLs and additional information about the context of the navigation. There is clearly a need to extract more information about the meaning of the data the user is browsing, starting by the content of the web pages.

Because the information provided on the current Web is not meant to be read by machines (T. Berners Lee, 2001) there is a need of tools to extract meaning from web pages. The information needs to be preprocessed before stored in a machine readable format, allowing to make recommendations (Choochart et Al, 2004).

Data preparation is composed of two steps: cleaning and structuring ( Castellano et Al, 2007). Because raw data can contain a lot of un-needed text (such as menus, headers etc.) and need to be cleaned prior to be stored. Multiple techniques can be used here and belongs to boilerplate removal and full text extraction (Kohlschütter et Al, 2010).

Then, structuring the information: category, type of content (news, blog, wiki) can be extracted from raw data. This kind of information is not clearly defined by HTML pages so there is a need of tools to recognise them.

Some context-related information can also be inferred from each resource. It can go from the visit frequency to the navigation group the user was in while browsing. It is also possible to determine if the user "liked" a resource, and determine a mark for it, which can be used by information filtering a later step (T. Segaran, 2007).

At this stage, structuring the data is required. Storing this kind of information in RDBMS can be a bit tedious and require complex queries to get back the data in an usable format. Graph databases can play a major role in the simplification of information storage and querying.

Information filtering

To filter the information, three techniques can be used (Balabanovic et Al, 1997):

The content-based approach states that if an user have liked something in the past, he is more likely to like similar things in the future. So it's about establishing a profile for the user and compare new items against it.
The collaborative approach will rather recommend items that other similar users have liked. This approach consider only the relationship between users, and not the profile of the user we are making recommendations to.
the hybrid approach, which appeared recently combine both of the previous approaches, giving recommendations when items score high regarding user's profile, or if a similar user already liked it.

Grouping is also something to consider at this stage (G. Myatt, 2007). Because we are dealing with huge amount of data, it can be useful to detect group of data that can fit together. Data clustering is able to find such groups (T. Segaran, 2007).

References:

Balabanović, M., & Shoham, Y. (1997). Fab: content-based, collaborative recommendation. Communications of the ACM, 40(3), 66–72. ACM. Retrieved March 1, 2011, from http://portal.acm.org/citation.cfm?id=245108.245124&.
Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web: Scientific american. Scientific American, 284(5), 34–43. Retrieved November 21, 2010, from http://www.citeulike.org/group/222/article/1176986.
Castellano, G., Fanelli, A., & Torsello, M. (2007). LODAP: a LOg DAta Preprocessor for mining Web browsing patterns. Proceedings of the 6th Conference on 6th WSEAS Int. Conf. on Artificial Intelligence, Knowledge Engineering and Data Bases-Volume 6 (p. 12–17). World Scientific and Engineering Academy and Society (WSEAS). Retrieved March 8, 2011, from http://portal.acm.org/citation.cfm?id=1348485.1348488.
Kohlschutter, C., Fankhauser, P., & Nejdl, W. (2010). Boilerplate detection using shallow text features. Proceedings of the third ACM international conference on Web search and data mining (p. 441–450). ACM. Retrieved March 8, 2011, from http://portal.acm.org/citation.cfm?id=1718542.
Myatt, G. J. (2007). Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining.
Segaran, T. (2007). Collective Intelligence.

Privacy

The first thing that's come to people minds when it comes to process their browsing data is privacy. People don't want to be stalked. That's perfectly right, and I don't either.

But such a system don't have to deal with people identities. It's completely possible to process completely anonymous data, and that's probably what I'm gonna do.

By the way, if you have interesting thoughts about that, if you do know projects that do seems related, fire the comments !

What's the plan ?

There is a lot of different things to explore, especially because I'm a complete novice in that field.

I want to develop a firefox plugin, to extract the browsing informations ( still, I need to know exactly which kind of informations to retrieve). The idea is to provide some raw browsing data, and then to transform it and to store it in the better possible way.
Analyse how to store the informations in a graph database. What can be the different methods to store this data and to visualize the relationship between different pieces of data? How can I define the different contexts, and add those informations in the db?
Process the data using well known recommendation algorithms. Compare the results and criticize their value.

There is plenty of stuff I want to try during this experimentation:

I want to try using Geshi to visualize the connexion between the links, and the contexts
Try using graph databases such as Neo4j
Having a deeper look at tools such as scikit.learn (a machine learning toolkit in python)
Analyse web pages in order to categorize them. Processing their contents as well, to do some keyword based classification will be done.

Lot of work on its way, yay !

Working directly on your server? How to backup and sync your dev environment with unison

2011-03-16T00:00:00+01:00

I have a server running freebsd since some time now, and was wondering about the possibility to directly have a development environment ready to use when I get a internet connexion, even if I'm not on my computer.

Since I use vim to code, and spend most of my time in a console while developing, it's possible to work via ssh, from everywhere.

The only problem is the synchronisation of the source code, config files etc. from my machine to the server.

Unison provides an interesting way to synchronise two folders, even over a network. So let's do it !

Creating the jail

In case you don't use FreeBSD, you can skip this section.

# I have a flavour jail named default
$ ezjail-admin -f default workspace.notmyidea.org 172.19.1.6
$ ezjail-admin start workspace.notmyidea.org

In my case, because the "default" flavour contains already a lot of interesting things, my jail come already setup with ssh, bash and vim for instance, but maybe you'll need it in your case.

I want to be redirected to the ssh of the jail when I connect to the host with the 20006 port. Add lines in /etc/pf.conf:

workspace_jail="172.19.1.6"
rdr on $ext_if proto tcp from any to $ext_ip port 20006 -> $workspace_jail port 22

Reload packet filter rules

$ /etc/rc.d/pf reload

Working with unison

Now that we've set up the jail. Set up unison on the server and on your client. Unison is available on the freebsd ports so just install it

$ ssh notmyidea.org -p 20006
$ make -C /usr/ports/net/unison-nox11 config-recursive
$ make -C /usr/ports/net/unison-nox11 package-recursive

Install as well unison on your local machine. Double check to install the same version on the client and on the server. Ubuntu contains the 2.27.57 as well as the 2.32.52.

Check that unison is installed and reachable via ssh from your machine

$ ssh notmyidea.org -p 20006 unison -version
unison version 2.27.157
$ unison -version
unison version 2.27.57

Let sync our folders

The first thing I want to sync is my vim configuration. Well, it's already in a git repository but let's try to use unison for it right now.

I have two machines then: workspace, the jail, and ecureuil my laptop.

unison .vim ssh://notmyidea.org:20006/.vim
unison .vimrc ssh://notmyidea.org:20006/.vimrc

It is also possible to put all the informations in a config file, and then to only run unison. (fire up vim ~/.unison/default.prf.

Here is my config:

root = /home/alexis
root = ssh://notmyidea.org:20006

path = .vimrc
path = dotfiles
path = dev

follow = Name *

My vimrc is in fact a symbolic link on my laptop, but I don't want to specify each of the links to unison. That's why the follow = Name * is for.

The folders you want to synchronize are maybe a bit large. If so, considering others options such as rsync for the first import may be a good idea (I enjoyed my university huge upload bandwith to upload 2GB in 20mn ;)

Run the script frequently

Once that done, you just need to run the unison command line some times when you want to sync your two machines. I've wrote a tiny script to get some feedback from the sync:

import os
from datetime import datetime

DEFAULT_LOGFILE = "~/unison.log"
PROGRAM_NAME = "Unison syncer"

def sync(logfile=DEFAULT_LOGFILE, program_name=PROGRAM_NAME):
    # init
    display_message = True
    error = False

    before = datetime.now()
    # call unison to make the sync
    os.system('unison -batch > {0}'.format(logfile))

    # get the duration of the operation
    td = datetime.now() - before
    delta = (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6

    # check what was the last entry in the log
    log = open(os.path.expanduser(logfile))
    lines = log.readlines()
    if 'No updates to propagate' in lines[-1]:
        display_message = False
    else:
        output = [l for l in lines if "Synchronization" in l]

        message = output[-1]
        message += " It took {0}s.".format(delta)

    if display_message:
        os.system('notify-send -i {2} "{0}" "{1}"'.format(program_name, message,
            'error' if error else 'info'))

if __name__ == "__main__":
    sync()

This is probably perfectible, but that does the job.

Last step is to tell you machine to run that frequently. That's what crontab is made for, so let's crontab -e:

$ * */3 * * * . ~/.Xdbus; /usr/bin/python /home/alexis/dev/python/unison-syncer/sync.py

The ~/.Xdbus allows cron to communicate with your X11 session. Here is its content.

#!/bin/bash

# Get the pid of nautilus
nautilus_pid=$(pgrep -u $LOGNAME -n nautilus)

# If nautilus isn't running, just exit silently
if [ -z "$nautilus_pid" ]; then
exit 0
fi

# Grab the DBUS_SESSION_BUS_ADDRESS variable from nautilus's environment
eval $(tr '\0' '\n' < /proc/$nautilus_pid/environ | grep '^DBUS_SESSION_BUS_ADDRESS=')

# Check that we actually found it
if [ -z "$DBUS_SESSION_BUS_ADDRESS" ]; then
echo "Failed to find bus address" >&2
exit 1
fi

# export it so that child processes will inherit it
export DBUS_SESSION_BUS_ADDRESS

And it comes from here.

A sync takes about 20s + the upload time on my machine, which stay acceptable for all of my developments.

Wrap up of the distutils2 paris' sprint

2011-02-08T00:00:00+01:00

Finally, thanks to a bunch of people that helped me to pay my train and bus tickets, I've made it to paris for the distutils2 sprint.

They have been a bit more than 10 people to come during the sprint, and it was very productive. Here's a taste of what we've been working on:

the datafiles, a way to specify and to handle the installation of files which are not python-related (pictures, manpages and so on).
mkgcfg, a tool to help you to create a setup.cfg in minutes (and with funny examples)
converters from setup.py scripts. We do now have a piece of code which reads your current setup.py file and fill in some fields in the setup.cfg for you.
a compatibility layer for distutils1, so it can read the setup.cfg you will wrote for distutils2 :-)
the uninstaller, so it's now possible to uninstall what have been installed by distutils2 (see PEP 376)
the installer, and the setuptools compatibility layer, which will allow you to rely on setuptools' based distributions (and there are plenty of them!)
The compilers, so they are more flexible than they were. Since that's an obscure part of the code for distutils2 commiters (it comes directly from the distutils1 ages), having some guys who understood the problematics here was a must.

Some people have also tried to port their packaging from distutils1 to distutils2. They have spotted a number of bugs and made some improvements to the code, to make it more friendly to use.

I'm really pleased to see how newcomers went trough the code, and started hacking so fast. I must say it wasn't the case when we started to work on distutils1 so that's a very good point: people now can hack the code quicker than they could before.

Some of the features here are not completely finished yet, but are on the tubes, and will be ready for a release (hopefully) at the end of the week.

Big thanks to logilab for hosting (and sponsoring my train ticket) and providing us food, and to bearstech for providing some money for breakfast and bears^Wbeers.

Again, a big thanks to all the people who gave me money to pay the transport, I really wasn't expecting such thing to happen :-)

PyPI on CouchDB

2011-01-20T00:00:00+01:00

By now, there are two ways to retrieve data from PyPI (the Python Package Index). You can both rely on xml/rpc or on the "simple" API. The simple API is not so simple to use as the name suggest, and have several existing drawbacks.

Basically, if you want to use informations coming from the simple API, you will have to parse web pages manually, to extract informations using some black vodoo magic. Badly, magic have a price, and it's sometimes impossible to get exactly the informations you want to get from this index. That's the technique currently being used by distutils2, setuptools and pip.

On the other side, while XML/RPC is working fine, it's requiring extra work to the python servers each time you request something, which can lead to some outages from time to time. Also, it's important to point out that, even if PyPI have a mirroring infrastructure, it's only for the so-called simple API, and not for the XML/RPC.

CouchDB

Here comes CouchDB. CouchDB is a document oriented database, that knows how to speak REST and JSON. It's easy to use, and provides out of the box a replication mechanism.

So, what ?

Hmm, I'm sure you got it. I've wrote a piece of software to link informations from PyPI to a CouchDB instance. Then you can replicate all the PyPI index with only one HTTP request on the CouchDB server. You can also access the informations from the index directly using a REST API, speaking json. Handy.

So PyPIonCouch is using the PyPI XML/RPC API to get data from PyPI, and generate records in the CouchDB instance.

The final goal is to avoid to rely on this "simple" API, and rely on a REST insterface instead. I have set up a couchdb server on my server, which is available at http://couchdb.notmyidea.org/_utils/database.html?pypi.

There is not a lot to see there for now, but I've done the first import from PyPI yesterday and all went fine: it's possible to access the metadata of all PyPI projects via a REST interface. Next step is to write a client for this REST interface in distutils2.

Example

For now, you can use pypioncouch via the command line, or via the python API.

Using the command line

You can do something like that for a full import. This will take long, because it's fetching all the projects at pypi and importing their metadata:

$ pypioncouch --fullimport http://your.couchdb.instance/

If you already have the data on your couchdb instance, you can just update it with the last informations from pypi. However, I recommend to just replicate the principal node, hosted at http://couchdb.notmyidea.org/pypi/, to avoid the duplication of nodes:

$ pypioncouch --update http://your.couchdb.instance/

The principal node is updated once a day by now, I'll try to see if it's enough, and ajust with the time.

Using the python API

You can also use the python API to interact with pypioncouch:

>>> from pypioncouch import XmlRpcImporter, import_all, update
>>> full_import()
>>> update()

What's next ?

I want to make a couchapp, in order to navigate PyPI easily. Here are some of the features I want to propose:

List all the available projects
List all the projects, filtered by specifiers
List all the projects by author/maintainer
List all the projects by keywords
Page for each project.
Provide a PyPI "Simple" API equivalent, even if I want to replace it, I do think it will be really easy to setup mirrors that way, with the out of the box couchdb replication

I also still need to polish the import mechanism, so I can directly store in couchdb:

The OPML files for each project
The upload_time as couchdb friendly format (list of int)
The tags as lists (currently it's only a string separated by spaces

The work I've done by now is available on https://bitbucket.org/ametaireau/pypioncouch/. Keep in mind that it's still a work in progress, and everything can break at any time. However, any feedback will be appreciated !

Help me to go to the distutils2 paris' sprint

2011-01-15T00:00:00+01:00

Edit: Thanks to logilab and some amazing people, I can make it to paris for the sprint. Many thanks to them for the support!

There will be a distutils2 sprint from the 27th to the 30th of january, thanks to logilab which will host the event.

You can find more informations about the sprint on the wiki page of the event (http://wiki.python.org/moin/Distutils/SprintParis).

I really want to go there but I'm unfortunately blocked in UK for money reasons. The cheapest two ways I've found is about £80, which I can't afford. Following some advices on #distutils, I've set up a ChipIn account for that, so if some people want to help me making it to go there, they can give me some money that way.

I'll probably work on the installer (to support old distutils and setuptools distributions) and on the uninstaller (depending on the first task). If I can't make it to paris, I'll hang around on IRC to give some help while needed.

If you want to contribute some money to help me go there, feel free to use this chipin page: http://ametaireau.chipin.com/distutils2-sprint-in-paris

Thanks for your support !

Fork you ! or how the social coding can help you

2010-11-05T00:00:00+01:00

With github and bitbucket coming around, a lot of new usages appears for the developpers: it's now easy to get feedback on your code/modifications, and to get help from others by, for instance, forking repositories.

Eeach time I see people helping others, I'm amazed by how we like to share our knowledge.

I say github, because it seems to be the more mainstream, but I think it's something strongly related to the DVCS principles: the "only" thing github have made is to turn that into a social network, and to reveal the awesomeness of the DVCSes to the masses.

What is really interesting is to see how this platform is addictive: it's automatically updating a webpages with the more accurate informations about the projects you're involved in, and add a bit of magic to that using webhooks, allowing you to update your website each time you push to you repository, for instance.

Quite nothing, indeed, but, I don't know why, I find this fascinating.

I haven't had the privilege to see my projects forked from github by strangers, but I've forked others repository to give an hand some times, when I wanted to, and the main reason is "because it's fun" to do so.

Yeah, you're probably right, you have to be a nerd to find fun to fork others. The good point is that geeks are a kind of nerds, and some geeks are coders :)

New ways to contribute

In addition, it seems that he community, or the communities, are there, on those new social networks for coders. It's really handy to drop an eye on interesting projects, to report bugs, propose new features, and check what new projects this or this person have made.

Well, "it's not new", you may think. That's true, because it's been a while that SVN was there and even CVS before that. But, it was a bit messy to "fork" a project, isn't it ? And I'm not talking about all the hell SVN involved with it (who have not had issues with those messy .svn folders raises an hand !).

It have not been so easy to share code and thoughts about code, to propose changes on existing code, than now. You think it's better to implement this or that in a different way ? Clone it (fork it), make your changes and publish them, and then ask projects owners about it. For sure you'll have answers.

Even if they don't want it, you can easily keep your changes, and keep getting their updates!

Also, lot of fashionables projects tend to move on DVCS. Personally, if I know I can fork on a DVCS instead of from a "simple" VCS, I'll probably be quicker to fork/clone, and to publish changes on my own copy, than if I had to do so on the upstream repository (and I'll likely dont have the rights to push to it), because I will not be afraid to break things.

DVCSes makes the contribution easier.

Release early, release often

Maybe have you read The cathedral and the bazaar, by Eric Steven Raymond ? (If not, consider doing so, it's a really interesting reading)

Among a lot of others interesting things, one hint he gives is release early, release often.

I understand it as: if you want to get contributors, release your code early, even if it's not perfect, and don't be afraid to publish your changes each time it's needed.

Without notifying it, that's basically what I was doing for my own projects. I guess that's because Social coding platforms encourages those practices, partially cause of the possible impact publishing each of your changes can have on your final solution.

If you have considered publishing your projects, code snippets, or whatever (code related) but did not done it, considering them not yet ready, maybe should you think about it twice: you can get feedback and probably start some interesting discussions about it, if you write code that's readable, of course!

A step further for open source softwares

Well, DVCSes are a honking great idea, and they're starting to be really powerful when applied to free softwares. I mean: if you can't see a project, it will be hard to contribute to it. And, I don't think anyone wants to contribute to something closed/proprietary, just for fun. Or maybe am I missing something.

Maybe it's a kind of revolution, about free and open source softwares (FOSS), that is going on. I really like to know I have my word to say about the changes in the tools I use, and to know that I can make them evolve.

Let's take an example. Imagine I'm using a web framework on daily basis, as a part of my job as a web developer. I do like using an open source software because I know how it's working, and because I know that I can interact with the authors of the framework while they're doing the changes on it.

That's communication, nothing more, and of course I can do that with an internal proprietary solution, but it will cost me a lot more time, for a dead-simple reason: a company is not as big and powerful as a community can be: it will cost time to work on this framework, resources to maintain it, fix bugs etc.

Well, I'm starting advocating here about Free and Open Source Softwares use on companies, what is a bit beyond the scope of this article, so let's back to our DVCSes and new social related tools.

If I find a bug in this framework, while working, I have the possibility to go and talk with the creators of the framework, to open a ticket, and even to make a fix for it, because I've access to the source code. If I want to create a new feature, I just have to fork it, hack it, and then publish my code to have feedback of the community.

My fix/work will benefit to all the people (and maybe others companies) working with this framework, and it's a way to prove the community that my company is enough skilled to make code-fixes to the framework, so that's all good !

What's next ?

I hope those social coding platforms are only the begining of a new area. I hope they will make people realize what the power of the community is, and how easily they can becomes part of it.

If you're not using them right now, maybe you should do so: have a look on how the programs you're using are made, consider publishing your experimentations, and share them with others, you will see, it's kind of addictive !

How to reboot your bebox using the CLI

2010-10-21T00:00:00+02:00

I've an internet connection which, for some obscure reasons, tend to be very slow from time to time. After rebooting the box (yes, that's a hard solution), all the things seems to go fine again.

EDIT : Using grep

After a bit of reflexion, that's also really easy to do using directly the command line tools curl, grep and tail (but really harder to read).

curl -X POST -u joel:joel http://bebox.config/cgi/b/info/restart/\?be\=0\&l0\=1\&l1\=0\&tid\=RESTART -d "0=17&2=`curl -u joel:joel http://bebox.config/cgi/b/info/restart/\?be\=0\&l0\=1\&l1\=0\&tid\=RESTART | grep -o "name='2' value='[0-9]\+" | grep -o "[0-9]\+" | tail -n 1`&1"

The Python version

Well, that's not the optimal solution, that's a bit "gruik", but it works.

import urllib2
import urlparse
import re
import argparse

REBOOT_URL = '/b/info/restart/?be=0&l0=1&l1=0&tid=RESTART'
BOX_URL = 'http://bebox.config/cgi'

def open_url(url, username, password):
    passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
    passman.add_password(None, url, username, password)
    authhandler = urllib2.HTTPBasicAuthHandler(passman)

    opener = urllib2.build_opener(authhandler)

    urllib2.install_opener(opener)

    return urllib2.urlopen(url).read()

def reboot(url, username, password):
    data = open_url(url, username, password)
    token = re.findall("name\=\\'2\\' value=\\'([0-9]+)\\'", data)[1]
    urllib2.urlopen(urllib2.Request(url=url, data='0=17&2=%s&1' % token))

if __file__ == '__main__':
    parser = argparse.ArgumentParser(description="""Reboot your bebox !""")

    parser.add_argument(dest='user', help='username')
    parser.add_argument(dest='password', help='password')
    parser.add_argument(boxurl='boxurl', default=BOX_URL, help='Base box url.  Default is %s' % BOX_URL)

    args = parser.parse_args()
    url = urlparse.urljoin(args.boxurl, REBOOT_URL)
    reboot(url, args.username, args.password)

Dynamically change your gnome desktop wallpaper

2010-10-11T00:00:00+02:00

In gnome, you can can use a XML file to have a dynamic wallpaper. It's not so easy, and you can't just tell: use the pictures in this folder to do so.

You can have a look to the git repository if you want: http://github.com/ametaireau/gnome-background-generator

Some time ago, I've made a little python script to ease that, and you can now use it too. It's named "gnome-background-generator", and you can install it via pip for instance.

$ pip install gnome-background-generator

Then, you have just to use it this way:

$ gnome-background-generator -p ~/Images/walls -s
/home/alexis/Images/walls/dynamic-wallpaper.xml generated

Here is a extract of the --help:

$ gnome-background-generator --help
usage: gnome-background-generator [-h] [-p PATH] [-o OUTPUT]
                                  [-t TRANSITION_TIME] [-d DISPLAY_TIME] [-s]
                                  [-b]

A simple command line tool to generate an XML file to use for gnome
wallpapers, to have dynamic walls

optional arguments:
  -h, --help            show this help message and exit
  -p PATH, --path PATH  Path to look for the pictures. If no output is
                        specified, will be used too for outputing the dynamic-
                        wallpaper.xml file. Default value is the current
                        directory (.)
  -o OUTPUT, --output OUTPUT
                        Output filename. If no filename is specified, a
                        dynamic-wallpaper.xml file will be generated in the
                        path containing the pictures. You can also use "-" to
                        display the xml in the stdout.
  -t TRANSITION_TIME, --transition-time TRANSITION_TIME
                        Time (in seconds) transitions must last (default value
                        is 2 seconds)
  -d DISPLAY_TIME, --display-time DISPLAY_TIME
                        Time (in seconds) a picture must be displayed. Default
                        value is 900 (15mn)
  -s, --set-background  '''try to set the background using gnome-appearance-
                        properties
  -b, --debug

How to install NGINX + PHP 5.3 on FreeBSD.

2010-10-10T00:00:00+02:00

I've not managed so far to get completely rid of php, so here's a simple reminder about how to install php on NGINX, for FreeBSD. Nothing hard, but that's worse to have the piece of configuration somewhere !

# update the ports
$ portsnap fetch update

# install php5 port
$ make config-recursive -C /usr/ports/lang/php5-extensions
$ make package-recursive -C /usr/ports/lang/php5-extensions

# install nginx
$ make config-recursive -C /usr/ports/www/nginx-devel
$ make package-recursive -C /usr/ports/www/nginx-devel

Now we have all the dependencies installed, we need to configure a bit the server.

That's a simple thing in fact, but it could be good to have something that will work without effort over time.

Here's a sample of my configuration:

server {
    server_name ndd;
    set $path /path/to/your/files;
    root   $path;

    location / {
        index  index.php;
    }

    location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml)$ {
      access_log        off;
      expires           30d;
    }

    location ~ .php$ {
        fastcgi_param  SCRIPT_FILENAME  $path$fastcgi_script_name;
        fastcgi_pass   backend;
        include fastcgi_params;
    }
}

upstream backend {
        server 127.0.0.1:9000;
}

And that's it !

Pelican, a simple static blog generator in python

2010-10-06T00:00:00+02:00

Those days, I've wrote a little python application to fit my blogging needs. I'm an occasional blogger, a vim lover, I like restructured text and DVCSes, so I've made a little tool that makes good use of all that.

Pelican (for calepin) is just a simple tool to generate your blog as static files, letting you using your editor of choice (vim!). It's easy to extend, and has a template support (via jinja2).

I've made it to fit my needs. I hope it will fit yours, but maybe it wont, and it have not be designed to feet everyone's needs.

Need an example ? You're looking at it ! This weblog is using pelican to be generated, also for the atom feeds.

I've released it under AGPL, since I want all the modifications to be profitable to all the users.

You can find a mercurial repository to fork at http://hg.lolnet.org/pelican/, feel free to hack it !

If you just want to get started, use your installer of choice (pip, easy_install, …) And then have a look to the help (pelican --help)

$ pip install pelican

Usage

Here's a sample usage of pelican

$ pelican .
writing /home/alexis/projets/notmyidea.org/output/index.html
writing /home/alexis/projets/notmyidea.org/output/tags.html
writing /home/alexis/projets/notmyidea.org/output/categories.html
writing /home/alexis/projets/notmyidea.org/output/archives.html
writing /home/alexis/projets/notmyidea.org/output/category/python.html
writing
/home/alexis/projets/notmyidea.org/output/pelican-a-simple-static-blog-generator-in-python.html
Done !

You also can use the --help option for the command line to get more informations

$pelican --help
usage: pelican [-h] [-t TEMPLATES] [-o OUTPUT] [-m MARKUP] [-s SETTINGS] [-b]
               path

A tool to generate a static blog, with restructured text input files.

positional arguments:
  path                  Path where to find the content files (default is
                        "content").

optional arguments:
  -h, --help            show this help message and exit
  -t TEMPLATES, --templates-path TEMPLATES
                        Path where to find the templates. If not specified,
                        will uses the ones included with pelican.
  -o OUTPUT, --output OUTPUT
                        Where to output the generated files. If not specified,
                        a directory will be created, named "output" in the
                        current path.
  -m MARKUP, --markup MARKUP
                        the markup language to use. Currently only
                        ReSTreucturedtext is available.
  -s SETTINGS, --settings SETTINGS
                        the settings of the application. Default to None.
  -b, --debug

Enjoy :)

An amazing summer of code working on distutils2

2010-08-16T00:00:00+02:00

The Google Summer of Code I've spent working on distutils2 is over. It was a really amazing experience, for many reasons.

First of all, we had a very good team, we were 5 students working on distutils2: Zubin, Éric, Josip, Konrad and me. In addition, Mouad have worked on the PyPI testing infrastructure. You could find what each person have done on the wiki page of distutils2.

We were in contact with each others really often, helping us when possible (in #distutils), and were continuously aware of the state of the work of each participant. This, in my opinion, have bring us in a good shape.

Then, I've learned a lot. Python packaging was completely new to me at the time of the GSoC start, and I was pretty unfamiliar with python good practices too, as I've been introducing myself to python in the late 2009.

I've recently looked at some python code I wrote just three months ago, and I was amazed to think about many improvements to made on it. I guess this is a good indicator of the path I've traveled since I wrote it.

This summer was awesome because I've learned about python good practices, now having some strong mercurial knowledge, and I've seen a little how the python community works.

Then, I would like to say a big thanks to all the mentors that have hanged around while needed, on IRC or via mail, and especially my mentor for this summer, Tarek Ziadé.

Thanks a lot for your motivation, your leadership and your cheerfulness, even with a new-born and a new work!

Why ?

I wanted to work on python packaging because, as the time pass, we were having a sort of complex tools in this field. Each one wanted to add features to distutils, but not in a standard way.

Now, we have PEPs that describes some format we agreed on (see PEP 345), and we wanted to have a tool on which users can base their code on, that's distutils2.

My job

I had to provides a way to crawl the PyPI indexes in a simple way, and do some installation / uninstallation scripts.

All the work done is available in my bitbucket repository.

Crawling the PyPI indexes

There are two ways of requesting informations from the indexes: using the "simple" index, that is a kind of REST index, and using XML-RPC.

I've done the two implementations, and a high level API to query those twos. Basically, this supports the mirroring infrastructure defined in PEP 381. So far, the work I've done is gonna be used in pip (they've basically copy/paste the code, but this will change as soon as we get something completely stable for distutils2), and that's a good news, as it was the main reason for what I've done that.

I've tried to have an unified API for the clients, to switch from one to another implementation easily. I'm already thinking of adding others crawlers to this stuff, and it was made to be extensible.

If you want to get more informations about the crawlers/PyPI clients, please refer to the distutils2 documentation, especially the pages about indexes.

You can find the changes I made about this in the distutils2 source code .

Installation / Uninstallation scripts

Next step was to think about an installation script, and an uninstaller. I've not done the uninstaller part, and it's a smart part, as it's basically removing some files from the system, so I'll probably do it in a near future.

distutils2 provides a way to install distributions, and to handle dependencies between releases. For now, this support is only about the last version of the METADATA (1.2) (See, the PEP 345), but I'm working on a compatibility layer for the old metadata, and for the informations provided via PIP requires.txt, for instance.

Extra work

Also, I've done some extra work. this includes:

working on the PEP 345, and having some discussion about it (about the names of some fields).
writing a PyPI server mock, useful for tests. you can find more information about it on the documentation.

Futures plans

As I said, I've enjoyed working on distutils2, and the people I've met here are really pleasant to work with. So I want to continue contributing on python, and especially on python packaging, because there is still a lot of things to do in this scope, to get something really usable.

I'm not plainly satisfied by the work I've done, so I'll probably tweak it a bit: the installer part is not yet completely finished, and I want to add support for a real REST index in the future.

We'll talk again of this in the next months, probably, but we definitely need a real REST API for PyPI, as the "simple" index is an ugly hack, in my opinion. I'll work on a serious proposition about this, maybe involving CouchDB, as it seems to be a good option for what we want here.

Issues

I've encountered some issues during this summer. The main one is that's hard to work remotely, especially being in the same room that we live, with others. I like to just think about a project with other people, a paper and a pencil, no computers. This have been not so possible at the start of the project, as I needed to read a lot of code to understand the codebase, and then to read/write emails.

I've finally managed to work in an office, so good point for home/office separation.

I'd not planned there will be so a high number of emails to read, in order to follow what's up in the python world, and be a part of the community seems to takes some times to read/write emails, especially for those (like me) that arent so confortable with english (but this had brought me some english fu !).

Thanks !

A big thanks to Graine Libre and Makina Corpus, which has offered me to come into their offices from time to time, to share they cheerfulness ! Many thanks too to the Google Summer of Code program for setting up such an initiative. If you're a student, if you're interested about FOSS, dont hesitate any second, it's a really good opportunity to work on interesting projects!

Introducing the distutils2 index crawlers

2010-07-06T00:00:00+02:00

I'm working for about a month for distutils2, even if I was being a bit busy (as I had some class courses and exams to work on)

I'll try do sum-up my general feelings here, and the work I've made so far. You can also find, if you're interested, my weekly summaries in a dedicated wiki page.

General feelings

First, and it's a really important point, the GSoC is going very well, for me as for other students, at least from my perspective. It's a pleasure to work with such enthusiast people, as this make the global atmosphere very pleasant to live.

First of all, I've spent time to read the existing codebase, and to understand what we're going to do, and what's the rationale to do so.

It's really clear for me now: what we're building is the foundations of a packaging infrastructure in python. The fact is that many projects co-exists, and comes all with their good concepts. Distutils2 tries to take the interesting parts of all, and to provide it in the python standard libs, respecting the recently written PEP about packaging.

With distutils2, it will be simpler to make "things" compatible. So if you think about a new way to deal with distributions and packaging in python, you can use the Distutils2 APIs to do so.

Tasks

My main task while working on distutils2 is to provide an installation and an un-installation command, as described in PEP 376. For this, I first need to get informations about the existing distributions (what's their version, name, metadata, dependencies, etc.)

The main index, you probably know and use, is PyPI. You can access it at http://pypi.python.org.

PyPI index crawling

There is two ways to get these informations from PyPI: using the simple API, or via xml-rpc calls.

A goal was to use the version specifiers defined in`PEP 345 <http://www.python.org/dev/peps/pep-0345/>`_ and to provides a way to sort the grabbed distributions depending our needs, to pick the version we want/need.

Using the simple API

The simple API is composed of HTML pages you can access at http://pypi.python.org/simple/.

Distribute and Setuptools already provides a crawler for that, but it deals with their internal mechanisms, and I found that the code was not so clear as I want, that's why I've preferred to pick up the good ideas, and some implementation details, plus re-thinking the global architecture.

The rules are simple: each project have a dedicated page, which allows us to get informations about:

the distribution download locations (for some versions)
homepage links
some other useful informations, as the bugtracker address, for instance.

If you want to find all the distributions of the "EggsAndSpam" project, you could do the following (do not take so attention to the names here, as the API will probably change a bit):

>>> index = SimpleIndex()
>>> index.find("EggsAndSpam")
[EggsAndSpam 1.1, EggsAndSpam 1.2, EggsAndSpam 1.3]

We also could use version specifiers:

>>> index.find("EggsAndSpam (< =1.2)")
[EggsAndSpam 1.1, EggsAndSpam 1.2]

Internally, what's done here is the following:

it process the http://pypi.python.org/simple/FooBar/ page, searching for download URLs.
for each found distribution download URL, it creates an object, containing informations about the project name, the version and the URL where the archive remains.
it sort the found distributions, using version numbers. The default behavior here is to prefer source distributions (over binary ones), and to rely on the last "final" distribution (rather than beta, alpha etc. ones)

So, nothing hard or difficult here.

We provides a bunch of other features, like relying on the new PyPI mirroring infrastructure or filter the found distributions by some criterias. If you're curious, please browse the distutils2 documentation.

Using xml-rpc

We also can make some xmlrpc calls to retreive informations from PyPI. It's a really more reliable way to get informations from from the index (as it's just the index that provides the informations), but cost processes on the PyPI distant server.

For now, this way of querying the xmlrpc client is not available on Distutils2, as I'm working on it. The main pieces are already present (I'll reuse some work I've made from the SimpleIndex querying, and some code already set up), what I need to do is to provide a xml-rpc PyPI mock server, and that's on what I'm actually working on.

Processes

For now, I'm trying to follow the "documentation, then test, then code" path, and that seems to be really needed while working with a community. Code is hard to read/understand, compared to documentation, and it's easier to change.

While writing the simple index crawling work, I must have done this to avoid some changes on the API, and some loss of time.

Also, I've set up a schedule, and the goal is to be sure everything will be ready in time, for the end of the summer. (And now, I need to learn to follow schedules ...)

Sprinting on distutils2 in Tours

2010-07-06T00:00:00+02:00

Yesterday, as I was traveling to Tours, I've took some time to visit Éric, another student who's working on distutils2 this summer, as a part of the GSoC. Basically, it was to take a drink, discuss a bit about distutils2, our respective tasks and general feelings, and to put a face on a pseudonym. I'd really enjoyed this time, because Éric knows a lot of things about mercurial and python good practices, and I'm eager to learn about those. So, we have discussed about things, have not wrote so much code, but have some things to propose so far, about documentation, and I also provides here some bribes of conversations we had.

Documentation

While writing the PyPI simple index crawler documentation, I realized that we miss some structure, or how-to about the documentation. Yep, you read well. We lack documentation on how to make documentation. Heh. We're missing some rules to follow, and this lead to a not-so-structured final documentation. We probably target three type of publics, and we can split the documentation regarding those:

Packagers who want to distribute their softwares.
End users who need to understand how to use end user commands, like the installer/uninstaller
packaging coders who use distutils2, as a base for building a package manager.

We also need to discuss about a pattern to follow while writing documentation. How many parts do we need ? Where to put the API description ? etc. That's maybe seems to be not so important, but I guess the readers would appreciate to have the same structure all along distutils2 documentation.

Mercurial

I'm really not a mercurial power user. I use it on daily basis, but I lack of basic knowledge about it. Big thanks Éric for sharing yours with me, you're of a great help. We have talked about some mercurial extensions that seems to make the life simpler, while used the right way. I've not used them so far, so consider this as a personal note.

hg histedit, to edit the history
hg crecord, to select the changes to commit

We have spent some time to review a merge I made sunday, to re-merge it, and commit the changes as a new changeset. Awesome. These things make me say I need to read the hg book, and will do as soon as I got some spare time: mercurial seems to be simply great. So ... Great. I'm a powerful merger now !

On using tools

Because we also are hackers, we have shared a bit our ways to code, the tools we use, etc. Both of us were using vim, and I've discovered vimdiff and hgtk, which will completely change the way I navigate into the mercurial history. We aren't "power users", so we have learned from each other about vim tips. You can find my dotfiles on github, if it could help. They're not perfect, and not intended to be, because changing all the time, as I learn. Don't hesitate to have a look, and to propose enhancements if you have !

On being pythonic

My background as an old Java user disserves me so far, as the paradigms are not the same while coding in python. Hard to find the more pythonic way to do, and sometimes hard to unlearn my way to think about software engineering. Well, it seems that the only solution is to read code, and to re-read import this from times to times ! Coding like a pythonista seems to be a must-read, so, I know what to do.

Conclusion

It was really great. Next time, we'll need to focus a bit more on distutils2, and to have a bullet list of things to do, but days like this one are opportunities to catch ! We'll probably do another sprint in a few weeks, stay tuned !

Use Restructured Text (ReST) to power your presentations

2010-06-25T00:00:00+02:00

Wednesday, we give a presentation, with some friends, about the CouchDB Database, to the Toulouse local LUG. Thanks a lot to all the presents for being there, it was a pleasure to talk about this topic with you. Too bad the season is over now an I quit Toulouse next year.

During our brainstorming about the topic, we used some paper, and we wanted to make a presentation the simpler way. First thing that come to my mind was using restructured text, so I've wrote a simple file containing our different bullet points. In fact, there is quite nothing to do then, to have a working presentation.

So far, I've used the rst2pdf program, and a simple template, to generate output. It's probably simple to have similar results using latex + beamer, I'll try this next time, but as I'm not familiar with latex syntax, restructured text was a great option.

Here are the final PDF output, Rhe ReST source, the theme used, and the command line to generate the PDF:

rst2pdf couchdb.rst -b1 -s ../slides.style

first week working on distutils2

2010-06-04T00:00:00+02:00

As I've been working on Distutils2 during the past week, taking part of the GSOC program, here is a short summary of what I've done so far.

As my courses are not over yet, I've not worked as much as I wanted, and this will continues until the end of June. My main tasks are about making installation and uninstallation commands, to have a simple way to install distributions via Distutils2.

To do this, we need to rely on informations provided by the Python Package Index (PyPI), and there is at least two ways to retreive informations from here: XML-RPC and the "simple" API.

So, I've been working on porting some Distribute related stuff to Distutils2, cutting off all non distutils' things, as we do not want to depend from Distribute's internals. My main work has been about reading the whole code, writing tests about this and making those tests possible.

In fact, there was a need of a pypi mocked server, and, after reading and introducing myself to the distutils behaviors and code, I've taken some time to improve the work Konrad makes about this mock.

A PyPI Server mock

The mock is embeded in a thread, to make it available during the tests, in a non blocking way. We first used WSGI and wsgiref in order control what to serve, and to log the requests made to the server, but finally realised that wsgiref is not python 2.4 compatible (and we need to be python 2.4 compatible in Distutils2).

So, we switched to BaseHTTPServer and SimpleHTTPServer, and updated our tests accordingly. It's been an opportunity to realize that WSGI has been a great step forward for making HTTP servers, and expose a really simplest way to discuss with HTTP !

You can find the modifications I made, and the related docs about this on my bitbucket distutils2 clone.

The PyPI Simple API

So, back to the main problematic: make a python library to access and request information stored on PyPI, via the simple API. As I said, I've just grabbed the work made from Distribute, and played a bit with, in order to view what are the different use cases, and started to write the related tests.

The work to come

So, once all use cases covered with tests, I'll rewrite a bit the grabbed code, and do some software design work (to not expose all things as privates methods, have a clear API, and other things like this), then update the tests accordingly and write a documentation to make this clear.

Next step is to a little client, as I've already started here I'll take you updated !

A Distutils2 GSoC

2010-05-01T00:00:00+02:00

WOW. I've been accepted to be a part of the Google Summer Of Code program, and will work on python distutils2, with a lot of (intersting!) people.

So, it's about building the successor of Distutils2, ie. "the python package manager". Today, there is too many ways to package a python application (pip, setuptools, distribute, distutils, etc.) so there is a huge effort to make in order to make all this packaging stuff interoperable, as pointed out by the PEP 376.

In more details, I'm going to work on the Installer / Uninstaller features of Distutils2, and on a PyPI XML-RPC client for distutils2. Here are the already defined tasks:

Implement Distutils2 APIs described in PEP 376.
Add the uninstall command.
think about a basic installer / uninstaller script. (with deps) -- similar to pip/easy_install
in a pypi subpackage;
Integrate a module similar to setuptools' package_index'
PyPI XML-RPC client for distutils 2: http://bugs.python.org/issue8190

As I'm relatively new to python, I'll need some extra work in order to apply all good practice, among other things that can make a developper-life joyful. I'll post here, each week, my advancement, and my tought about python and especialy python packaging world.

Le temps des grâces, courrez-y !

2010-03-28T00:00:00+01:00

Ouf, notre semaine de l'environnement s'est terminée, après un peu de neige et un brin de soleil quand il en fallait.

Ce fut l'occasion de rencontrer beaucoup de gens biens, et de regarder d'un peu plus près toutes ces problématiques qui mériteraient à elles seules, chacune un billet. Après un répis d'une semaine, pour récupérer des folies organisatrices (c'est ça qui fait du bien, ceci dit -- les folies, pas le repos), je me retrouve de nouveau sur ces mêms réflexions, grâce aux journées organisées par France Nature Environnement, qui proposaient ce jeudi soir une projection de "Le temps des grâces", un documentaire sur l'agriculture.

Parce que depuis 60 ans, il est pratiqué le remembrement des parcelles à gogo, parce que nos sols deviennent completement morts à cause des tonnes d'intrants qu'on leur fait absorber, et parce que les exploitations s'agrandissent sans en finir pour tenter de réagir façe à un marché financier toujours plus insaisissable, mené de très loin par la PAC, il fallait un film pour en parler. C'est ce que fait "le temps des grâçes", avec un bon sens et une facilité à faire passer des messages, qu'on ne peut qu'applaudir.

Malheureusement il s'agissait de la dernière diffusion à L'Utopia de Toulouse... Si vous en avez l'occasion, sautez dessus et profitez le temps de ces 2 heures, ou vous pourrez écouter à la fois des discours d'agronomes, d'agriculteurs, de chercheurs et bien d'autres, qui dressent un constat pas si brillant de notre agriculture. On n'est pas sorti de l'auberge, enfin, c'est l'expression.

Je vous laisse avec le synopsis et une bande annonce.

Une enquête documentaire sur le monde agricole français aujourd'hui à travers de nombreux récits : agriculteurs, chercheurs, agronomes, écrivains... Un monde qui parvient à résister aux bouleversements qui le frappent - économiques, scientifiques, sociaux - et qui, bon gré mal gré, continue d'entretenir les liens entre générations. Un monde au centre d'interrogations majeures sur l'avenir.

Ainsi qu'un commentaire que je ne peux m'empécher de relayer, trouvé sur allocine.fr (oui, vous savez, ce site rempli de pubs).

Le temps des grâces c'est je pense le plus grand film traitant de l'écologie en tant que documentaire ou en sujet principal, on y apprend une multitude de choses, on en ressort en colère contre le système, le film propose différents points de vues, ici on a pas de voix off moralisatrice à deux balles avec des gros titres bien surlignés pour que même les beaufs matant TF1 puissent comprendre, ici même si ça reste accessible au citoyen lambda, le film ne fait pas de compromis avec le monde agroalimentaire, il ose dénoncer les filière d'agronomie qui apprennent pas les bonnes choses à leurs étudiants, aux lobbys de cette agro-industrie qui n'en fait qu'à sa tête pour amasser de l'argent, cette tendance à tout uniformiser et détruire… Franchement j'ai trouvé ça génial de bout en bout, captivant, on explore cette campagne française, on comprend les dilemmes des paysants. Le film n'étant pas opposé à la modernité, ni même réactionnaire, il propose juste un constat alarmant du monde agricole français, tout en proposant des alternatives qui pourraient être utilisée, si les pouvoirs publics lâchaient un peu la main des lobbys. Passionnant, vraiment.

Courrez-y, je vous dis.

Semaine de l’environnement: La consommation étudiante

2010-02-24T00:00:00+01:00

Les acteurs associatifs sont bien souvent moteurs des critiques de nos sociétés. Je pense à Framasoft, à la Quadrature du net ou à Ars Industrialis (dans le domaine de la science et de l'informatique), mais aussi aux Amis de la Terre, à ATTAC (dans le domaine de l'environnement entres autres), et a tout un tas d'autres associations que je ne peux pas citer ici exhaustivement... Ce sont eux qui sont porteurs de messages alternatifs, et qui sont les initiateurs de débats publics, qui permettent de faire avancer des thématiques aussi importantes que le logiciel libre ou la protection de l'environnement.

Dans cette optique, depuis près de 3 ans (déjà !), au sein du réseau GRAPPE, on essaye d'aborder des thématiques qui touchent de près ou de loin à l'environnement, parce que c'est un sujet qui nous importe et nous passionne. L'année dernière, c'était l'alimentation étudiante, ce qui à abouti à la publication de la revue "les étudiants se mettent à table". Cette année c'est la consommation étudiante qui est au programme.

La consommation étudiante ?

L'idée principale de cette étude est de tenter de faire une analyse de la "société de consommation", souvent questionnée par les étudiants et par d'autres, et de jeter un œil sur le rapport des étudiants à cette société: Quoi et comment consomment-ils ? Les universités et les écoles ne poussent-elles pas d’une certaine manière les étudiants à la consommation ? Quelles sont les alternatives face aux dérives de surconsommation ?

Analyse des pratiques, réflexions et mobilisations des étudiants en termes de consommation seront réalisés pour comprendre leur modes de vie, mais aussi leurs attentes, leurs propositions sur cette thématique

La semaine de l'environnement !

A travers l'ensemble des villes du réseau, des projections de films et des débats auront donc lieu sur ce thème, lors de la semaine de l'environnement 2010, qui se déroulera d'ailleurs durant le mois de Mars partout en france, et du 6 au 14 Mars sur Toulouse. Jetez un oeil au programme ! Avec pour objectif de sonder un peu le ressenti des étudiants en terme de consommation, nous avons mis en place un questionnaire en ligne, que vous pouvez compléter en une petite 10aine de minutes, n'hésitez pas ! La page sur la consommation étudiante sur le site du GRAPPE

Python ? go !

2009-12-17T00:00:00+01:00

Cela fait maintenant un peu plus d'un mois que je travaille sur un projet en django, et que, nécessairement, je me forme à Python. Je prends un plaisir non dissimulé à découvrir ce langage (et à l'utiliser), qui ne cesse de me surprendre. Les premiers mots qui me viennent à l'esprit à propos de Python, sont "logique" et "simple". Et pourtant puissant pour autant. Je ne manque d'ailleurs pas une occasion pour faire un peu d'évangélisation auprès des quelques personnes qui veulent bien m'écouter.

The Zen of Python

Avant toute autre chose, je pense utile de citer Tim Peters, et le PEP20, qui constituent une très bonne introduction au langage, qui prends la forme d'un easter egg présent dans python

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

J'ai la vague impression que c'est ce que j'ai toujours cherché à faire en PHP, et particulièrement dans le framework Spiral, mais en ajoutant ces concepts dans une sur-couche au langage. Ici, c'est directement de l'esprit de python qu'il s'agit, ce qui signifie que la plupart des bibliothèques python suivent ces concepts. Elle est pas belle la vie ?

Comment commencer, et par ou ?

Pour ma part, j'ai commencé par la lecture de quelques livres et articles intéressants, qui constituent une bonne entrée en matière sur le sujet (La liste n'est bien évidemment pas exhaustive et vos commentaires sont les bienvenus) :

Dive into python
A byte of python
Python: petit guide à l'usage du développeur agile de Tarek Ziadé
La documentation officielle python, bien sûr !
Les vidéos du pyconfr 2009!
Un peu de temps, et une console python ouverte :)

J'essaye par ailleurs de partager au maximum les ressources que je trouve de temps à autres, que ce soit via twitter ou via mon compte delicious. Allez jeter un œil au tag python sur mon profil, peut être que vous trouverez des choses intéressantes, qui sait!

Un python sexy

Quelques fonctionnalités qui devraient vous mettre l'eau à la bouche:

Le chaînage des opérateurs de comparaison est possible (a<b <c dans une condition)
Assignation de valeurs multiples (il est possible de faire a,b,c = 1,2,3 par exemple)
Les listes sont simples à manipuler !
Les list comprehension, ou comment faire des opérations complexes sur les listes, de manière simple.
Les doctests: ou comment faire des tests directement dans la documentation de vos classes, tout en la documentant avec de vrais exemples.
Les métaclasses, ou comment contrôler la manière dont les classes se construisent
Python est un langage à typage fort dynamique: c'est ce qui m'agaçait avec PHP qui est un langage à typage faible dynamique.

Cous pouvez également aller regarder l'atelier donné par Victor Stinner durant le Pyconfr 09. Have fun !

AMAP + Média = Paniers bio à 5e ?!

2009-11-11T00:00:00+01:00

Le raccourci me semble un peu rapide. Et pourtant, il est emprunté bien trop souvent. La dernière mes-utilisation que j'ai à décrier est celle d'un reportage télé, passé sur France 2 vendredi 23 Octobre (voir la vidéo), ou on parles de l'AMAP étudiante Zest.

Malgré tout le temps passé à expliquer que justement, l'AMAP c'est avant tout, pour nous, une notion de solidarité envers les agriculteurs, malgré le fait que le message soit plus profond que simplement aller acheter du bio (ça, c'est possible aussi en supermarché), malgré le fait qu'il s'agisse en fait d'une remise en cause plus profonde de la société, les journalistes n'ont choisi de prendre que les quelques secondes qui leurs convenaient, celles qui ne faisaient pas tache, celle ou je dis que "l'AMAP permet aux étudiants de réapprendre à cuisiner, plutôt que de manger des nouilles et de se faire des plats réchauffés". Bien sur, je l'ai dit et je le pense d'ailleurs, là n'est pas le problème, mais mon message n'était pas celui là.

Jamais, dans les quelques minutes du reportage, on ne dit ce que AMAP signifie: Association de Maintien d'une Agriculture Paysanne. On ne parles même pas des agriculteurs ! C'est simplement plus facile pour les étudiants de venir récupérer leurs paniers, puisque les points de distribution sont sur les facs, et en plus ça ne coute que 5 euros ! ...

Alors je crois bon de rappeler que L'AMAP n'est pas seulement un service, c'est un engagement. Sur 5 mois, un engagement solidaire, c'est la donnée principale de l'équation, celle sans quoi ça ne peut fonctionner. Par le biais de cette AMAP, on essaye de montrer que d'autres agricultures sont possibles, que la solidarité c'est bien réel, et qu'il est possible de sortir du tout, tout de suite.

Mais bon, apparemment, ça passe pas bien à la télé.