mirror of
https://github.com/almet/notmyidea.git
synced 2025-04-28 19:42:37 +02:00
227 lines
No EOL
18 KiB
HTML
227 lines
No EOL
18 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
|
|
<head>
|
|
<title>Refactoring Cornice - Alexis Métaireau</title>
|
|
<meta charset="utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<link rel="stylesheet" href="https://blog.notmyidea.org/theme/css/main.css?v2" type="text/css" />
|
|
<link href="https://blog.notmyidea.org/feeds/all.atom.xml" type="application/atom+xml" rel="alternate"
|
|
title="Alexis Métaireau ATOM Feed" />
|
|
</head>
|
|
|
|
<body>
|
|
<div id="content">
|
|
<section id="links">
|
|
<ul>
|
|
<li><a class="main" href="/">Alexis Métaireau</a></li>
|
|
<li><a class=""
|
|
href="https://blog.notmyidea.org/journal/index.html">Journal</a></li>
|
|
<li><a class="selected"
|
|
href="https://blog.notmyidea.org/code/">Code, etc.</a></li>
|
|
<li><a class=""
|
|
href="https://blog.notmyidea.org/weeknotes/">Notes hebdo</a></li>
|
|
<li><a class=""
|
|
href="https://blog.notmyidea.org/lectures/">Lectures</a></li>
|
|
</ul>
|
|
</section>
|
|
|
|
<header>
|
|
<h1 class="post-title">Refactoring Cornice</h1>
|
|
<time datetime="2012-05-01T00:00:00+02:00">01 mai 2012</time>
|
|
|
|
|
|
</header>
|
|
<article>
|
|
|
|
<p>After working for a while with <a href="http://cornice.readthedocs.com">Cornice</a>
|
|
to define our APIs at <a href="http://docs.services.mozilla.com">Services</a>, it
|
|
turned out that the current implementation wasn’t flexible enough to
|
|
allow us to do what we wanted to do.</p>
|
|
<p>Cornice started as a toolkit on top of the
|
|
<a href="http://docs.pylonsproject.org/en/latest/docs/pyramid.html">pyramid</a>
|
|
routing system, allowing to register services in a simpler way. Then we
|
|
added some niceties such as the ability to automatically generate the
|
|
services documentation or returning the correct <span class="caps">HTTP</span> headers <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html">as defined
|
|
by the <span class="caps">HTTP</span>
|
|
specification</a>
|
|
without the need from the developer to deal with them nor to know them.</p>
|
|
<p>If you’re not familiar with Cornice, here is how you define a simple
|
|
service with it:</p>
|
|
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">cornice.service</span> <span class="kn">import</span> <span class="n">Service</span>
|
|
<span class="n">bar</span> <span class="o">=</span> <span class="n">Service</span><span class="p">(</span><span class="n">path</span><span class="o">=</span><span class="s2">"/bar"</span><span class="p">)</span>
|
|
|
|
<span class="nd">@bar</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">validators</span><span class="o">=</span><span class="n">validators</span><span class="p">,</span> <span class="n">accept</span><span class="o">=</span><span class="s1">'application/json'</span><span class="p">)</span>
|
|
<span class="k">def</span> <span class="nf">get_drink</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
|
|
<span class="c1"># do something with the request (with moderation).</span>
|
|
</code></pre></div>
|
|
|
|
<p>This external <span class="caps">API</span> is quite cool, as it allows to do a bunch of things
|
|
quite easily. For instance, we’ve written our
|
|
<a href="https://github.com/mozilla-services/tokenserver">token-server</a> code on
|
|
top of this in a blast.</p>
|
|
<h2 id="the-burden">The burden</h2>
|
|
<p>The problem with this was that we were mixing internally the service
|
|
description logic with the route registration one. The way we were doing
|
|
this was via an extensive use of decorators internally.</p>
|
|
<p>The <span class="caps">API</span> of the cornice.service.Service class was as following
|
|
(simplified so you can get the gist of it).</p>
|
|
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Service</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|
|
|
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">service_kwargs</span><span class="p">):</span>
|
|
<span class="c1"># some information, such as the colander schemas (for validation),</span>
|
|
<span class="c1"># the defined methods that had been registered for this service and</span>
|
|
<span class="c1"># some other things were registered as instance variables.</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">schemas</span> <span class="o">=</span> <span class="n">service_kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">schema</span><span class="s1">', None)</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span> <span class="o">=</span> <span class="p">[]</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span> <span class="o">=</span> <span class="p">[]</span>
|
|
|
|
<span class="k">def</span> <span class="nf">api</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">view_kwargs</span><span class="p">):</span>
|
|
<span class="w"> </span><span class="sd">"""This method is a decorator that is being used by some alias</span>
|
|
<span class="sd"> methods.</span>
|
|
<span class="sd"> """</span>
|
|
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">view</span><span class="p">):</span>
|
|
<span class="c1"># all the logic goes here. And when I mean all the logic, I</span>
|
|
<span class="c1"># mean it.</span>
|
|
<span class="c1"># 1. we are registering a callback to the pyramid routing</span>
|
|
<span class="c1"># system so it gets called whenever the module using the</span>
|
|
<span class="c1"># decorator is used.</span>
|
|
<span class="c1"># 2. we are transforming the passed arguments so they conform</span>
|
|
<span class="c1"># to what is expected by the pyramid routing system.</span>
|
|
<span class="c1"># 3. We are storing some of the passed arguments into the</span>
|
|
<span class="c1"># object so we can retrieve them later on.</span>
|
|
<span class="c1"># 4. Also, we are transforming the passed view before</span>
|
|
<span class="c1"># registering it in the pyramid routing system so that it</span>
|
|
<span class="c1"># can do what Cornice wants it to do (checking some rules,</span>
|
|
<span class="c1"># applying validators and filters etc.</span>
|
|
<span class="k">return</span> <span class="n">wrapper</span>
|
|
|
|
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|
<span class="w"> </span><span class="sd">"""A shortcut of the api decorator"""</span>
|
|
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">api</span><span class="p">(</span><span class="n">request_method</span><span class="o">=</span><span class="s2">"GET"</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|
</code></pre></div>
|
|
|
|
<p>I encourage you to go read <a href="https://github.com/mozilla-services/cornice/blob/4e0392a2ae137b6a11690459bcafd7325e86fa9e/cornice/service.py#L44">the entire
|
|
file</a>.
|
|
on github so you can get a better opinion on how all of this was done.</p>
|
|
<p>A bunch of things are wrong:</p>
|
|
<ul>
|
|
<li>first, we are not separating the description logic from the
|
|
registration one. This causes problems when we need to access the
|
|
parameters passed to the service, because the parameters you get are
|
|
not exactly the ones you passed but the ones that the pyramid
|
|
routing system is expecting. For instance, if you want to get the
|
|
view get_drink, you will instead get a decorator which contains
|
|
this view.</li>
|
|
<li>second, we are using decorators as APIs we expose. Even if
|
|
decorators are good as shortcuts, they shouldn’t be the default way
|
|
to deal with an <span class="caps">API</span>. A good example of this is <a href="https://github.com/mozilla-services/cornice/blob/4e0392a2ae137b6a11690459bcafd7325e86fa9e/cornice/resource.py#L56">how the resource
|
|
module consumes this
|
|
<span class="caps">API</span></a>.
|
|
This is quite hard to follow.</li>
|
|
<li>Third, in the api method, a bunch of things are done regarding
|
|
inheritance of parameters that are passed to the service or to its
|
|
decorator methods. This leaves you with a really hard to follow path
|
|
when it comes to add new parameters to your <span class="caps">API</span>.</li>
|
|
</ul>
|
|
<h2 id="how-do-we-improve-this">How do we improve this?</h2>
|
|
<p>Python is great because it allows you to refactor things in an easy way.
|
|
What I did isn’t breaking our APIs, but make things way simpler to
|
|
hack-on. One example is that it allowed me to add features that we
|
|
wanted to bring to Cornice really quickly (a matter of minutes), without
|
|
touching the <span class="caps">API</span> that much.</p>
|
|
<p>Here is the gist of the new architecture:</p>
|
|
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Service</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
|
|
<span class="c1"># we define class-level variables that will be the default values for</span>
|
|
<span class="c1"># this service. This makes things more extensible than it was before.</span>
|
|
<span class="n">renderer</span> <span class="o">=</span> <span class="s1">'simplejson'</span>
|
|
<span class="n">default_validators</span> <span class="o">=</span> <span class="n">DEFAULT_VALIDATORS</span>
|
|
<span class="n">default_filters</span> <span class="o">=</span> <span class="n">DEFAULT_FILTERS</span>
|
|
|
|
<span class="c1"># we also have some class-level parameters that are useful to know</span>
|
|
<span class="c1"># which parameters are supposed to be lists (and so converted as such)</span>
|
|
<span class="c1"># or which are mandatory.</span>
|
|
<span class="n">mandatory_arguments</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'renderer'</span><span class="p">,)</span>
|
|
<span class="n">list_arguments</span> <span class="o">=</span> <span class="p">(</span><span class="s1">'validators'</span><span class="p">,</span> <span class="s1">'filters'</span><span class="p">)</span>
|
|
|
|
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">path</span><span class="p">,</span> <span class="n">description</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
|
|
<span class="c1"># setup name, path and description as instance variables</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">path</span> <span class="o">=</span> <span class="n">path</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
|
|
|
|
<span class="c1"># convert the arguments passed to something we want to store</span>
|
|
<span class="c1"># and then store them as attributes of the instance (because they</span>
|
|
<span class="c1"># were passed to the constructor</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">arguments</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arguments</span><span class="p">(</span><span class="n">kw</span><span class="p">)</span>
|
|
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arguments</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
|
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
|
|
|
|
<span class="c1"># we keep having the defined_methods tuple and the list of</span>
|
|
<span class="c1"># definitions that are done for this service</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span> <span class="o">=</span> <span class="p">[]</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span> <span class="o">=</span> <span class="p">[]</span>
|
|
|
|
<span class="k">def</span> <span class="nf">get_arguments</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">conf</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
|
<span class="w"> </span><span class="sd">"""Returns a dict of arguments. It does all the conversions for</span>
|
|
<span class="sd"> you, and uses the information that were defined at the instance</span>
|
|
<span class="sd"> level as fallbacks.</span>
|
|
<span class="sd"> """</span>
|
|
|
|
<span class="k">def</span> <span class="nf">add_view</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|
<span class="w"> </span><span class="sd">"""Add a view to this service."""</span>
|
|
<span class="c1"># this is really simple and looks a lot like this</span>
|
|
<span class="n">method</span> <span class="o">=</span> <span class="n">method</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="n">args</span><span class="p">))</span>
|
|
<span class="k">if</span> <span class="n">method</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span><span class="p">:</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">method</span><span class="p">)</span>
|
|
|
|
<span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
|
<span class="w"> </span><span class="sd">"""This is only another interface to the add_view method, exposing a</span>
|
|
<span class="sd"> decorator interface"""</span>
|
|
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">view</span><span class="p">):</span>
|
|
<span class="bp">self</span><span class="o">.</span><span class="n">add_view</span><span class="p">(</span><span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
|
<span class="k">return</span> <span class="n">view</span>
|
|
<span class="k">return</span> <span class="n">wrapper</span>
|
|
</code></pre></div>
|
|
|
|
<p>So, the service is now only storing the information that’s passed to it
|
|
and nothing more. No more route registration logic goes here. Instead, I
|
|
added this as another feature, even in a different module. The function
|
|
is named register_service_views and has the following signature:</p>
|
|
<div class="highlight"><pre><span></span><code><span class="n">register_service_views</span><span class="p">(</span><span class="n">config</span><span class="p">,</span> <span class="n">service</span><span class="p">)</span>
|
|
</code></pre></div>
|
|
|
|
<p>To sum up, here are the changes I made:</p>
|
|
<ol>
|
|
<li>Service description is now separated from the route registration.</li>
|
|
<li>cornice.service.Service now provides a hook_view method, which is
|
|
not a decorator. decorators are still present but they are optional
|
|
(you don’t need to use them if you don’t want to).</li>
|
|
<li>Everything has been decoupled as much as possible, meaning that you
|
|
really can use the Service class as a container of information about
|
|
the services you are describing. This is especially useful when
|
|
generating documentation.</li>
|
|
</ol>
|
|
<p>As a result, it is now possible to use Cornice with other frameworks. It
|
|
means that you can stick with the service description but plug any other
|
|
framework on top of it. cornice.services.Service is now only a
|
|
description tool. To register routes, one would need to read the
|
|
information contained into this service and inject the right parameters
|
|
into their preferred routing system.</p>
|
|
<p>However, no integration with other frameworks is done at the moment even
|
|
if the design allows it.</p>
|
|
<p>The same way, the sphinx description layer is now only a consumer of
|
|
this service description tool: it looks at what’s described and build-up
|
|
the documentation from it.</p>
|
|
<p>The resulting branch is not merged yet. Still, you can <a href="https://github.com/mozilla-services/cornice/tree/refactor-the-world">have a look at
|
|
it</a>.</p>
|
|
<p>Any suggestions are of course welcome :-)</p>
|
|
</article>
|
|
<footer>
|
|
<a id="feed" href="/feeds/all.atom.xml"><img src="/theme/rss.svg" /></a>
|
|
</footer>
|
|
</div>
|
|
</body>
|
|
|
|
</html> |