diff --git a/content/incubation/caching-elastic-search.rst b/content/incubation/caching-elastic-search.rst deleted file mode 100644 index fa3c7ec..0000000 --- a/content/incubation/caching-elastic-search.rst +++ /dev/null @@ -1,92 +0,0 @@ -How to cache Elastic Seach Queries? -################################### - -:status: other - -These days, I'm working on Marketplace, the Mozilla's Appstore. Internally, -we're doing Elastic Search to do search, and after some load tests, we -eventually found out that Elastic Search was our bottleneck. - -So we want to reduce the number of requests we do to this server. Currently, -the requests are done trough HTTP. - -The first thing to realize is what do we want to cache exactly? There is a fair -bit of things we might want to cache. Let's start by the most queried pages: -the home and the list of apps per category. - -Different approaches -==================== - -You can put the cache in many different locations. The code powering -Marketplace is kinda fuzzy sometimes. The requests to Elastic Search are done -in a number of different parts of the code. They're done sometimes directly -with HTTP calls, sometimes using the ElasticUtils library, sometimes using some -other lib… - -That's kind of hard to get where and how to add the caching layer here. What -did we do? We started to work on an HTTP caching proxy. This proxy could - -Find a key -========== - - - -Caching things -============== - -Caching is easy, it's something like that, in term of python code:: - - if key in cache: - value = cache.get(key) - else: - value = do_the_request() - cache.set(key, value) - return value - -Back to business logic: I want to cache the request that are done on the -homepage. The code currently looks like this:: - - popular = Webapp.popular(region=region, gaia=request.GAIA)[:10] - -Nothing fancy going on here, we're displaying the list of popular and latest -apps on the marketplace. I can cache the results for each region, and depending -if `request.GAIA` is `True` or not. The key will look like `popular-{region}`, -to which I will eventually append `-gaia` if it makes sense. More generally, -I will turn a number of criterias into a single key, like this:: - - def cache(qs, *keys): - """Cache the given querystring""" - key = '-'.join(keys) - if key in cache: - result = cache.get(key) - else: - result = qs - cache.set(key, result) - return result - -Which I can use like this:: - - keys = [region.slug, ] - if request.GAIA: - keys.append('gaia') - - popular = cache(Webapp.popular(region=region, gaia=request.GAIA)[:10], - 'popular', *keys) - -Invalidation is hard? -===================== - -Right, we got this cached, good. But what happens when the data we cached -changes? Currently, nothing, we return the same data over and over again. - -We need to tell our application when this cache isn't good anymore; we need to -*invalidate* it. - -We can use different strategies for this: - -* Set a timeout for the cache. -* Invalidate manually the cache when something changes, for instance using - signals. - -I started by having a look at how I wanted to invalidate all of this, case by -case. The problem diff --git a/content/incubation/service-description.rst b/content/incubation/service-description.rst deleted file mode 100644 index 6ca5cc5..0000000 --- a/content/incubation/service-description.rst +++ /dev/null @@ -1,58 +0,0 @@ -API description using cornice and colander -########################################## - -:status: draft - -One of the goals we want to accomplish with Cornice is to describe the web -services in a nice and easy way, for the developer. This for different reasons. - -Having web services described in some ways allows you do do interesting things, -such as automatically generating a documentation, or a client, for instance. -REST services are supposed to be discoverable, but in a lot of situations, -that's not the case, and it can be a pain to implement a client for them, even -if most of what is being done there is shared between a lot of web services. - -In cornice, we already generate some documentation, but this one is incomplete in -different ways. For instance, it is currently not possible to get information -about the inputs you are waiting for, in the different locations (body, -headers and query string). - -For instance, if you want to describe that a *POST* on `/foobar` needs the -*foo* and *bar* parameters, and that you eventually could add a *baz* -parameter, you can't, or you have to describe it manually in the documentation -of the web service. - -Cornice, while not tied to `Colander`_, tries to make a good use of it when -possible. So it is possible to do validation of the incoming data (in different -locations) and, at the same time, documentation of the web service. - -To do so, you have to use a particular `schema` keyword in the decorator. Here -is an example of definition - -.. code-block:: python - - from cornice import Service - from colander import MappingSchema, SchemaNode, String - - foobar = Service(name="foobar", path="/foobar") - - - class Foobar(MappingSchema): - foo = SchemaNode(String(), location="body") - bar = SchemaNode(String(), location="headers") - baz = SchemaNode(String(), default=None) - - - @foobar.get(scheme=Foobar) - def foobar_get(request): - # do something here - -.. _Colander: http://docs.pylonsproject.org/projects/colander/en/latest/ - -The output documentation currently looks like this: - -.. image:: images/cornice-exampledoc-validation.png - -The plans for the future are to be able to put direct curl calls in the -documentation to demonstrate how the service behaves for instance, and why not -creating a tiny python client able to consume the cornice APIs. diff --git a/content/incubation/vote-electronique-a-distance.rst b/content/incubation/vote-electronique-a-distance.rst deleted file mode 100644 index b6b778c..0000000 --- a/content/incubation/vote-electronique-a-distance.rst +++ /dev/null @@ -1,29 +0,0 @@ -L'heure pour un vote electronique à distance ? -############################################## - -Ce matin, je me fais reveiller par le reveil. La radio. Je suis chez des amis, -à Paris, et le Front National à fait des scores incroyables. Du jamais vu. - -En prenant le RER, je croise le regard de ces gens, probablements immigrés, qui -semblent abrutis par la situation. Je suis le fils d'une émigrée et je partage -leur rage, leur haine. Derrière ces beaux discours, le Front National souhaite -fermer les frontières, controler qui rentre pour les laisser chez eux, ces -personnes qui ont eu l'infortune de naitre ailleurs, la ou la terre est moins -propice. - -Je me sens coupable, ce matin. Coupable de ne pas avoir été voter hier. Dans ma -bulle j'étais, absent. Cela fait plusieurs années que je ne vote plus, par -manque de foi en la politique qui est menée actuellement, et par dépis. J'attends, -mollement, le second tour pour exprimer mon suffrage. Pour ce qu'il vaut. Je ne -crois pas en la representativité sous sa forme actuelle. - -Mais ce matin, je m'en veux quand même. Même sans croire à ce système de -representativité, j'aurais pu éviter que ces idées racistes et haineuses -n'étendent leur pouvoir à d'autres régions. - -Il est trop tard pour changer la donne, le mal est fait. Mais j'aurais pu aller -voter. J'aurais du même. - -La geule encore pateuse, au petit déjeuner, je me questionne sur mon -attachement à un territoir, et aux consequences de ce non-attachement sur mon -envie de voter. diff --git a/content/incubation/walint.rst b/content/incubation/walint.rst deleted file mode 100644 index 19e973e..0000000 --- a/content/incubation/walint.rst +++ /dev/null @@ -1,83 +0,0 @@ -walint - tilt -############# - -:status: draft - -Things start to fold together. I still need some time to be up to -speed (new job / new city / new people), but the projects I'm working on are -pretty exciting and I like what I'm doing. We've been working with Tarek on a -tool about web services testing. - -At Mozilla, we produce and consume web services. As an organisation defending -web standards, providing services that match these specifications is the least -we can do. - -Web App Lint (WALint) is a simple tool to help you to test your web services -against the HTTP specification. The goal is to detect potential problems a -service could be subject to, and to provide a simple-to-configure tool to test -your WS against HTTP as speced. The long term goal is to be exhaustive on the -controllers we provide, but if you have special needs, you should be able to -define tests in a simple way. - -As everything we do at Mozilla, the tool is an opensource project. Our goal is -to have something useful for web service consumers and providers. If you think -you can be of any help, don't hesitate to drop me an email or fork on github. - -Testing the HTTP specification -============================== - -A lot of web services are written on top of HTTP. The protocol is fully -described in a specification and introduce some really interesting stuff we -should care about when implementing our services. This tool wants to check this -up and provide a ways to ensure that you are defining your web services the -right way. It's not a replacement for manual service testing, thus: you still -have to test that your web service is doing what you think it does. WALint is -checking the stuff we almost forgot all of the time. - -For instance, the HTTP specification specifies that if the client sends a -request with a Content-Type not handled by the server, it should answer with a -"406 Not Acceptable" and should return the list of accepted headers (so the -client is able to send again the request with the right accept header). We also -check that sending weird data (broken json objects or authentication headers) -does not break the tested service. - -Obviously, we don't have all the scenarios in mind, and they aren't all -implemented. That's where being an open source project makes sense. You, as -service providers and consumers, know what are the kind of mistake you are used -to do / to deal with. It should make "tilt": fork the project and write some -lines checking the particular behavior that's disturbing you, everyone will -enjoy it. If you prefer, just open a ticket and we'll care about the -implementation. - -The documentation defines a list of all the implemented controllers: -http://packages.python.org/walint/controllers.html - -What does it looks like? -======================== - -Here is an example of a configuration file: you describe the paths and -signatures of your web services, the controllers you want to use and the test -scenarios you want to run, that's it, just run "walint config.cfg" and you're -good. - -Here's a capture of the output of the command line invocation: you get all the -paths tested with the different listed methods, and if needed what's not -working. In the future, why not putting some documentation online with -information about how to fix common mistakes? - -We also provide a wizard so it's possible to describe your web service in a -simple and easy way. Just invoke it with "walint --create config.cfg". - -What's next ? -============= - -We plan to do some fun stuff with WALint. Here are some thoughts: - -* service discovery: just provide an API root URI and we'll try to discover - what's in there and how to generate configuration files - -* unittest integration, so it's possible to have an unified way to describe web - services with existing tools. (heh, we can do Web Services CI the easy way!) -* integration with cornice description language and other web services description - languages (still to be defined) -* your ideas, so please fill issues and provide us feedback, it's useful! diff --git a/pelican.conf.py b/pelican.conf.py index aea8fad..06f7585 100644 --- a/pelican.conf.py +++ b/pelican.conf.py @@ -23,7 +23,7 @@ LOCALE = "fr_FR.utf8" DEFAULT_DATE_FORMAT = ('%d %B %Y') LINKS = [ ('Brasserie du Vieux Singe', 'https://www.vieuxsinge.com'), - ('A propos', 'http://blog.notmyidea.org/pages/about.html') + ('A propos', 'http://blog.notmyidea.org/pages/about.html'), ('Messages courts', 'https://twitter.com/ametaireau'), ('Code', 'https://github.com/almet'), ]