almet/blog.notmyidea.org
1
0
Fork 0
mirror of https://github.com/almet/notmyidea.git synced 2025-04-28 19:42:37 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
blog.notmyidea.org/refactoring-cornice.html

242 lines
No EOL
19 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang="fr">
<head>
<title>
Refactoring&nbsp;Cornice - Alexis Métaireau </title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet"
href="https://blog.notmyidea.org/theme/css/main.css?v2"
type="text/css" />
<link href="https://blog.notmyidea.org/feeds/all.atom.xml"
type="application/atom+xml"
rel="alternate"
title="Alexis Métaireau ATOM Feed" />
</head>
<body>
<div id="content">
<section id="links">
<ul>
<li>
<a class="main" href="/">Alexis Métaireau</a>
</li>
<li>
<a class=""
href="https://blog.notmyidea.org/journal/index.html">Journal</a>
</li>
<li>
<a class="selected"
href="https://blog.notmyidea.org/code/">Code, etc.</a>
</li>
<li>
<a class=""
href="https://blog.notmyidea.org/weeknotes/">Notes hebdo</a>
</li>
<li>
<a class=""
href="https://blog.notmyidea.org/lectures/">Lectures</a>
</li>
<li>
<a class=""
href="https://blog.notmyidea.org/projets.html">Projets</a>
</li>
</ul>
</section>
<header>
<h1 class="post-title">Refactoring&nbsp;Cornice</h1>
<time datetime="2012-05-01T00:00:00+02:00">01 mai 2012</time>
</header>
<article>
<p>After working for a while with <a href="http://cornice.readthedocs.com">Cornice</a>
to define our APIs at <a href="http://docs.services.mozilla.com">Services</a>, it
turned out that the current implementation wasn&#8217;t flexible enough to
allow us to do what we wanted to&nbsp;do.</p>
<p>Cornice started as a toolkit on top of the
<a href="http://docs.pylonsproject.org/en/latest/docs/pyramid.html">pyramid</a>
routing system, allowing to register services in a simpler way. Then we
added some niceties such as the ability to automatically generate the
services documentation or returning the correct <span class="caps">HTTP</span> headers <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html">as defined
by the <span class="caps">HTTP</span>
specification</a>
without the need from the developer to deal with them nor to know&nbsp;them.</p>
<p>If you&#8217;re not familiar with Cornice, here is how you define a simple
service with&nbsp;it:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">cornice.service</span> <span class="kn">import</span> <span class="n">Service</span>
<span class="n">bar</span> <span class="o">=</span> <span class="n">Service</span><span class="p">(</span><span class="n">path</span><span class="o">=</span><span class="s2">&quot;/bar&quot;</span><span class="p">)</span>
<span class="nd">@bar</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">validators</span><span class="o">=</span><span class="n">validators</span><span class="p">,</span> <span class="n">accept</span><span class="o">=</span><span class="s1">&#39;application/json&#39;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">get_drink</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="c1"># do something with the request (with moderation).</span>
</code></pre></div>
<p>This external <span class="caps">API</span> is quite cool, as it allows to do a bunch of things
quite easily. For instance, we&#8217;ve written our
<a href="https://github.com/mozilla-services/tokenserver">token-server</a> code on
top of this in a&nbsp;blast.</p>
<h2 id="the-burden">The&nbsp;burden</h2>
<p>The problem with this was that we were mixing internally the service
description logic with the route registration one. The way we were doing
this was via an extensive use of decorators&nbsp;internally.</p>
<p>The <span class="caps">API</span> of the cornice.service.Service class was as following
(simplified so you can get the gist of&nbsp;it).</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Service</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">service_kwargs</span><span class="p">):</span>
<span class="c1"># some information, such as the colander schemas (for validation),</span>
<span class="c1"># the defined methods that had been registered for this service and</span>
<span class="c1"># some other things were registered as instance variables.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">schemas</span> <span class="o">=</span> <span class="n">service_kwargs</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">schema</span><span class="s1">&#39;, None)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span> <span class="o">=</span> <span class="p">[]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">api</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">view_kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;This method is a decorator that is being used by some alias</span>
<span class="sd"> methods.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">view</span><span class="p">):</span>
<span class="c1"># all the logic goes here. And when I mean all the logic, I</span>
<span class="c1"># mean it.</span>
<span class="c1"># 1. we are registering a callback to the pyramid routing</span>
<span class="c1"># system so it gets called whenever the module using the</span>
<span class="c1"># decorator is used.</span>
<span class="c1"># 2. we are transforming the passed arguments so they conform</span>
<span class="c1"># to what is expected by the pyramid routing system.</span>
<span class="c1"># 3. We are storing some of the passed arguments into the</span>
<span class="c1"># object so we can retrieve them later on.</span>
<span class="c1"># 4. Also, we are transforming the passed view before</span>
<span class="c1"># registering it in the pyramid routing system so that it</span>
<span class="c1"># can do what Cornice wants it to do (checking some rules,</span>
<span class="c1"># applying validators and filters etc.</span>
<span class="k">return</span> <span class="n">wrapper</span>
<span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;A shortcut of the api decorator&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">api</span><span class="p">(</span><span class="n">request_method</span><span class="o">=</span><span class="s2">&quot;GET&quot;</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
</code></pre></div>
<p>I encourage you to go read <a href="https://github.com/mozilla-services/cornice/blob/4e0392a2ae137b6a11690459bcafd7325e86fa9e/cornice/service.py#L44">the entire
file</a>.
on github so you can get a better opinion on how all of this was&nbsp;done.</p>
<p>A bunch of things are&nbsp;wrong:</p>
<ul>
<li>first, we are not separating the description logic from the
registration one. This causes problems when we need to access the
parameters passed to the service, because the parameters you get are
not exactly the ones you passed but the ones that the pyramid
routing system is expecting. For instance, if you want to get the
view get_drink, you will instead get a decorator which contains
this&nbsp;view.</li>
<li>second, we are using decorators as APIs we expose. Even if
decorators are good as shortcuts, they shouldn&#8217;t be the default way
to deal with an <span class="caps">API</span>. A good example of this is <a href="https://github.com/mozilla-services/cornice/blob/4e0392a2ae137b6a11690459bcafd7325e86fa9e/cornice/resource.py#L56">how the resource
module consumes this
<span class="caps">API</span></a>.
This is quite hard to&nbsp;follow.</li>
<li>Third, in the api method, a bunch of things are done regarding
inheritance of parameters that are passed to the service or to its
decorator methods. This leaves you with a really hard to follow path
when it comes to add new parameters to your <span class="caps">API</span>.</li>
</ul>
<h2 id="how-do-we-improve-this">How do we improve&nbsp;this?</h2>
<p>Python is great because it allows you to refactor things in an easy way.
What I did isn&#8217;t breaking our APIs, but make things way simpler to
hack-on. One example is that it allowed me to add features that we
wanted to bring to Cornice really quickly (a matter of minutes), without
touching the <span class="caps">API</span> that&nbsp;much.</p>
<p>Here is the gist of the new&nbsp;architecture:</p>
<div class="highlight"><pre><span></span><code><span class="k">class</span> <span class="nc">Service</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="c1"># we define class-level variables that will be the default values for</span>
<span class="c1"># this service. This makes things more extensible than it was before.</span>
<span class="n">renderer</span> <span class="o">=</span> <span class="s1">&#39;simplejson&#39;</span>
<span class="n">default_validators</span> <span class="o">=</span> <span class="n">DEFAULT_VALIDATORS</span>
<span class="n">default_filters</span> <span class="o">=</span> <span class="n">DEFAULT_FILTERS</span>
<span class="c1"># we also have some class-level parameters that are useful to know</span>
<span class="c1"># which parameters are supposed to be lists (and so converted as such)</span>
<span class="c1"># or which are mandatory.</span>
<span class="n">mandatory_arguments</span> <span class="o">=</span> <span class="p">(</span><span class="s1">&#39;renderer&#39;</span><span class="p">,)</span>
<span class="n">list_arguments</span> <span class="o">=</span> <span class="p">(</span><span class="s1">&#39;validators&#39;</span><span class="p">,</span> <span class="s1">&#39;filters&#39;</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">path</span><span class="p">,</span> <span class="n">description</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
<span class="c1"># setup name, path and description as instance variables</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">path</span> <span class="o">=</span> <span class="n">path</span>
<span class="bp">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
<span class="c1"># convert the arguments passed to something we want to store</span>
<span class="c1"># and then store them as attributes of the instance (because they</span>
<span class="c1"># were passed to the constructor</span>
<span class="bp">self</span><span class="o">.</span><span class="n">arguments</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_arguments</span><span class="p">(</span><span class="n">kw</span><span class="p">)</span>
<span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">arguments</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="c1"># we keep having the defined_methods tuple and the list of</span>
<span class="c1"># definitions that are done for this service</span>
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span> <span class="o">=</span> <span class="p">[]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">get_arguments</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">conf</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Returns a dict of arguments. It does all the conversions for</span>
<span class="sd"> you, and uses the information that were defined at the instance</span>
<span class="sd"> level as fallbacks.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">add_view</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Add a view to this service.&quot;&quot;&quot;</span>
<span class="c1"># this is really simple and looks a lot like this</span>
<span class="n">method</span> <span class="o">=</span> <span class="n">method</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">definitions</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="n">args</span><span class="p">))</span>
<span class="k">if</span> <span class="n">method</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">defined_methods</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">method</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;This is only another interface to the add_view method, exposing a</span>
<span class="sd"> decorator interface&quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">wrapper</span><span class="p">(</span><span class="n">view</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">add_view</span><span class="p">(</span><span class="n">method</span><span class="p">,</span> <span class="n">view</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">return</span> <span class="n">view</span>
<span class="k">return</span> <span class="n">wrapper</span>
</code></pre></div>
<p>So, the service is now only storing the information that&#8217;s passed to it
and nothing more. No more route registration logic goes here. Instead, I
added this as another feature, even in a different module. The function
is named register_service_views and has the following&nbsp;signature:</p>
<div class="highlight"><pre><span></span><code><span class="n">register_service_views</span><span class="p">(</span><span class="n">config</span><span class="p">,</span> <span class="n">service</span><span class="p">)</span>
</code></pre></div>
<p>To sum up, here are the changes I&nbsp;made:</p>
<ol>
<li>Service description is now separated from the route&nbsp;registration.</li>
<li>cornice.service.Service now provides a hook_view method, which is
not a decorator. decorators are still present but they are optional
(you don&#8217;t need to use them if you don&#8217;t want&nbsp;to).</li>
<li>Everything has been decoupled as much as possible, meaning that you
really can use the Service class as a container of information about
the services you are describing. This is especially useful when
generating&nbsp;documentation.</li>
</ol>
<p>As a result, it is now possible to use Cornice with other frameworks. It
means that you can stick with the service description but plug any other
framework on top of it. cornice.services.Service is now only a
description tool. To register routes, one would need to read the
information contained into this service and inject the right parameters
into their preferred routing&nbsp;system.</p>
<p>However, no integration with other frameworks is done at the moment even
if the design allows&nbsp;it.</p>
<p>The same way, the sphinx description layer is now only a consumer of
this service description tool: it looks at what&#8217;s described and build-up
the documentation from&nbsp;it.</p>
<p>The resulting branch is not merged yet. Still, you can <a href="https://github.com/mozilla-services/cornice/tree/refactor-the-world">have a look at
it</a>.</p>
<p>Any suggestions are of course welcome&nbsp;:-)</p>
</article>
<footer>
<a id="feed" href="/feeds/all.atom.xml">
<img alt="RSS Logo" src="/theme/rss.svg" />
</a>
</footer>
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink