Alexis Métaireau - codehttps://blog.notmyidea.org/2023-09-27T00:00:00+02:00llm command-line tips2023-09-27T00:00:00+02:002023-09-27T00:00:00+02:00tag:blog.notmyidea.org,2023-09-27:/llm-command-line-tips.html<p>I&#8217;m using <a href="https://llm.datasette.io">llm</a> more and more, and today I had to find back prompts I used in the past. Here is a command I&#8217;ve been using, which allows me to filter the results based on what I want. It leverages <a href="https://sqlutils.datasette.io">sql-utils</a>, a cli tool which is able to …</p><p>I&#8217;m using <a href="https://llm.datasette.io">llm</a> more and more, and today I had to find back prompts I used in the past. Here is a command I&#8217;ve been using, which allows me to filter the results based on what I want. It leverages <a href="https://sqlutils.datasette.io">sql-utils</a>, a cli tool which is able to talk to a <span class="caps">SQLITE</span> database and answer in json, and <a href="https://github.com/jqlang/jq">jq</a> a command-line tool capable of doing requests for&nbsp;json.</p> <p>All in all, it&#8217;s pretty satisfying to use. I finally got a simple way to query databases! I&#8217;m also using <a href="https://github.com/charmbracelet/glow">glow</a>, which is capable of transforming markdown into a better version on the&nbsp;terminal.</p> <p><span class="dquo">&#8220;</span>`bash sqlite-utils &#8220;$(llm logs path)&#8221; &#8220;<span class="caps">SELECT</span> * <span class="caps">FROM</span> responses <span class="caps">WHERE</span> prompt <span class="caps">LIKE</span> &#8216;%search%&#8217;&#8221; | jq &#8216;.[].response&#8217; -r | glow&nbsp;&#8220;&#8220;</p> <p>Which got me a colored response&nbsp;:-)</p>Setting up a IRC Bouncer with ZNC2023-09-27T00:00:00+02:002023-09-27T00:00:00+02:00tag:blog.notmyidea.org,2023-09-27:/setting-up-a-irc-bouncer-with-znc.html<p>It&#8217;s been a while since I&#8217;ve used <span class="caps">IRC</span>, but I needed to connect to it today to discuss around <a href="https://docs.peewee-orm.com">Peewee</a>.</p> <p>The main issue with <span class="caps">IRC</span> is that you need to be connected to see the answer, and to get the context of the conversation. Unless&#8230; you set up …</p><p>It&#8217;s been a while since I&#8217;ve used <span class="caps">IRC</span>, but I needed to connect to it today to discuss around <a href="https://docs.peewee-orm.com">Peewee</a>.</p> <p>The main issue with <span class="caps">IRC</span> is that you need to be connected to see the answer, and to get the context of the conversation. Unless&#8230; you set up a&nbsp;bouncer.</p> <p>The bouncer is named <a href="https://znc.in"><span class="caps">ZNC</span></a>, and the <span class="caps">IRC</span> client I use is <a href="https://weechat.org">Weechat</a>.</p> <p>So, that&#8217;s what I&nbsp;did:</p> <h2 id="installation-of-znc">Installation of <span class="caps">ZNC</span></h2> <div class="highlight"><pre><span></span><code>apt<span class="w"> </span>install<span class="w"> </span>znc sudo<span class="w"> </span>-u<span class="w"> </span>_znc<span class="w"> </span>/usr/bin/znc<span class="w"> </span>--datadir<span class="o">=</span>/var/lib/znc<span class="w"> </span>--makeconf sudo<span class="w"> </span>systemctl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>znc </code></pre></div> <p>You can answer the questions asked by <code>--makeconf</code>, it will generate you a configuration file like this (stored in <code>/var/lib/znc/configurations/znc.conf</code>):</p> <div class="highlight"><pre><span></span><code>AnonIPLimit<span class="w"> </span>=<span class="w"> </span>10 AuthOnlyViaModule<span class="w"> </span>=<span class="w"> </span>false ConfigWriteDelay<span class="w"> </span>=<span class="w"> </span>0 ConnectDelay<span class="w"> </span>=<span class="w"> </span>5 HideVersion<span class="w"> </span>=<span class="w"> </span>false LoadModule<span class="w"> </span>=<span class="w"> </span>webadmin MaxBufferSize<span class="w"> </span>=<span class="w"> </span>500 ProtectWebSessions<span class="w"> </span>=<span class="w"> </span>true SSLCertFile<span class="w"> </span>=<span class="w"> </span>/var/lib/znc/znc.pem SSLDHParamFile<span class="w"> </span>=<span class="w"> </span>/var/lib/znc/znc.pem SSLKeyFile<span class="w"> </span>=<span class="w"> </span>/var/lib/znc/znc.pem ServerThrottle<span class="w"> </span>=<span class="w"> </span>30 Version<span class="w"> </span>=<span class="w"> </span>1.8.2 <span class="nt">&lt;Listener</span><span class="w"> </span><span class="err">listener0</span><span class="nt">&gt;</span> <span class="w"> </span>AllowIRC<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>AllowWeb<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>IPv4<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>IPv6<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>Port<span class="w"> </span>=<span class="w"> </span>6697 <span class="w"> </span>SSL<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>URIPrefix<span class="w"> </span>=<span class="w"> </span>/ <span class="nt">&lt;/Listener&gt;</span> <span class="nt">&lt;User</span><span class="w"> </span><span class="err">alexis</span><span class="nt">&gt;</span> <span class="w"> </span>Admin<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>Allow<span class="w"> </span>=<span class="w"> </span>* <span class="w"> </span>AltNick<span class="w"> </span>=<span class="w"> </span>alexis_ <span class="w"> </span>AppendTimestamp<span class="w"> </span>=<span class="w"> </span>false <span class="w"> </span>AuthOnlyViaModule<span class="w"> </span>=<span class="w"> </span>false <span class="w"> </span>AutoClearChanBuffer<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>AutoClearQueryBuffer<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>BindHost<span class="w"> </span>=<span class="w"> </span>skate.notmyidea.org <span class="w"> </span>ChanBufferSize<span class="w"> </span>=<span class="w"> </span>50 <span class="w"> </span>DenyLoadMod<span class="w"> </span>=<span class="w"> </span>false <span class="w"> </span>DenySetBindHost<span class="w"> </span>=<span class="w"> </span>false <span class="w"> </span>Ident<span class="w"> </span>=<span class="w"> </span>alexis <span class="w"> </span>JoinTries<span class="w"> </span>=<span class="w"> </span>10 <span class="w"> </span>LoadModule<span class="w"> </span>=<span class="w"> </span>chansaver <span class="w"> </span>LoadModule<span class="w"> </span>=<span class="w"> </span>controlpanel <span class="w"> </span>MaxJoins<span class="w"> </span>=<span class="w"> </span>0 <span class="w"> </span>MaxNetworks<span class="w"> </span>=<span class="w"> </span>1 <span class="w"> </span>MaxQueryBuffers<span class="w"> </span>=<span class="w"> </span>50 <span class="w"> </span>MultiClients<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>Nick<span class="w"> </span>=<span class="w"> </span>alexis <span class="w"> </span>NoTrafficTimeout<span class="w"> </span>=<span class="w"> </span>180 <span class="w"> </span>PrependTimestamp<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>QueryBufferSize<span class="w"> </span>=<span class="w"> </span>50 <span class="w"> </span>QuitMsg<span class="w"> </span>=<span class="w"> </span>See<span class="w"> </span>you<span class="w"> </span>:) <span class="w"> </span>RealName<span class="w"> </span>=<span class="w"> </span>N/A <span class="w"> </span>StatusPrefix<span class="w"> </span>=<span class="w"> </span>* <span class="w"> </span>TimestampFormat<span class="w"> </span>=<span class="w"> </span>[%H:%M:%S] <span class="w"> </span><span class="nt">&lt;Network</span><span class="w"> </span><span class="err">liberachat</span><span class="nt">&gt;</span> <span class="w"> </span>FloodBurst<span class="w"> </span>=<span class="w"> </span>9 <span class="w"> </span>FloodRate<span class="w"> </span>=<span class="w"> </span>2.00 <span class="w"> </span>IRCConnectEnabled<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span>JoinDelay<span class="w"> </span>=<span class="w"> </span>0 <span class="w"> </span>LoadModule<span class="w"> </span>=<span class="w"> </span>simple_away <span class="w"> </span>RealName<span class="w"> </span>=<span class="w"> </span>N/A <span class="w"> </span>Server<span class="w"> </span>=<span class="w"> </span>irc.libera.chat<span class="w"> </span>+6697 <span class="w"> </span>TrustAllCerts<span class="w"> </span>=<span class="w"> </span>false <span class="w"> </span>TrustPKI<span class="w"> </span>=<span class="w"> </span>true <span class="w"> </span><span class="nt">&lt;Chan</span><span class="w"> </span><span class="err">#peewee</span><span class="nt">&gt;</span> <span class="w"> </span><span class="nt">&lt;/Chan&gt;</span> <span class="w"> </span><span class="nt">&lt;/Network&gt;</span> <span class="w"> </span><span class="nt">&lt;Pass</span><span class="w"> </span><span class="err">password</span><span class="nt">&gt;</span> <span class="w"> </span>Hash<span class="w"> </span>=<span class="w"> </span>REDACTED <span class="w"> </span>Method<span class="w"> </span>=<span class="w"> </span>SHA256 <span class="w"> </span>Salt<span class="w"> </span>=<span class="w"> </span>REDACTED <span class="w"> </span><span class="nt">&lt;/Pass&gt;</span> <span class="nt">&lt;/User&gt;</span> </code></pre></div> <p>You can access a web interface on the exposed port. I had to make a change in my Firefox configuration, in <code>about:config</code>, set <code>network.security.ports.banned.override</code> to <code>6697</code>, otherwise, Firefox prevents you from connecting to these ports (which might actually be a good&nbsp;idea).</p> <h2 id="weechat-configuration">Weechat&nbsp;configuration</h2> <p>Now, to use this in weechat, here are some useful commands. First, get the fingerprint of the <span class="caps">SSL</span> certificate generated on your&nbsp;server:</p> <div class="highlight"><pre><span></span><code>cat<span class="w"> </span>/var/log/znc/znc.pem<span class="w"> </span><span class="p">|</span><span class="w"> </span>openssl<span class="w"> </span>x509<span class="w"> </span>-sha512<span class="w"> </span>-fingerprint<span class="w"> </span>-noout<span class="w"> </span><span class="p">|</span><span class="w"> </span>tr<span class="w"> </span>-d<span class="w"> </span><span class="s1">&#39;:&#39;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>tr<span class="w"> </span><span class="s1">&#39;A-Z&#39;</span><span class="w"> </span><span class="s1">&#39;a-z&#39;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>cut<span class="w"> </span>-d<span class="w"> </span><span class="o">=</span><span class="w"> </span>-f<span class="w"> </span><span class="m">2</span> <span class="sb">````</span> Then,<span class="w"> </span><span class="k">in</span><span class="w"> </span>weechat<span class="w"> </span>: <span class="sb">```</span>weechat /server<span class="w"> </span>add<span class="w"> </span>znc<span class="w"> </span>host/6697<span class="w"> </span>-tls<span class="w"> </span>-username<span class="o">=</span>&lt;username&gt;<span class="w"> </span>-password<span class="o">=</span>&lt;yourpass&gt;<span class="w"> </span>-autoconnect /set<span class="w"> </span>irc.server.znc.tls_fingerprint<span class="w"> </span>&lt;fingerprint-goes-here&gt; /connect<span class="w"> </span>znc </code></pre></div> <p>And you should be all&nbsp;set!</p> <p>Resources : <a href="https://wiki.znc.in/Weechat">The <span class="caps">ZNC</span> Wiki on Weechat</a> and the <a href="https://wiki.debian.org/ZNC">Debian page on <span class="caps">ZNC</span></a></p>How to run the vigogne model locally2023-09-22T00:00:00+02:002023-09-22T00:00:00+02:00tag:blog.notmyidea.org,2023-09-22:/how-to-run-the-vigogne-model-locally.html <p><a href="https://github.com/bofenghuang/vigogne">Vigogne</a> is a <span class="caps">LLM</span> model based on <span class="caps">LLAMA2</span>, but trained with french data. As I&#8217;m working mostly in french, it might be useful. The current models that I can get locally are in&nbsp;english.</p> <p>The information I&#8217;ve found online are scarse and not so easy to follow, so …</p> <p><a href="https://github.com/bofenghuang/vigogne">Vigogne</a> is a <span class="caps">LLM</span> model based on <span class="caps">LLAMA2</span>, but trained with french data. As I&#8217;m working mostly in french, it might be useful. The current models that I can get locally are in&nbsp;english.</p> <p>The information I&#8217;ve found online are scarse and not so easy to follow, so here is a step by step tutorial you can follow. I&#8217;m using <a href="https://pipenv.pypa.io/en/latest/">pipenv</a> almost everywhere now, it&#8217;s so easy&nbsp;:-)</p> <div class="highlight"><pre><span></span><code>llm<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>llm-llama-cpp wget<span class="w"> </span>https://huggingface.co/TheBloke/Vigogne-2-7B-Chat-GGUF/resolve/main/vigogne-2-7b-chat.Q4_K_M.gguf llm<span class="w"> </span>llama-cpp<span class="w"> </span>add-model<span class="w"> </span>vigogne-2-7b-chat.Q4_K_M.gguf<span class="w"> </span>-a<span class="w"> </span>vigogne llm<span class="w"> </span>models<span class="w"> </span>default<span class="w"> </span>vigogne </code></pre></div>Creating a simple command line to post snippets on Gitlab2023-09-18T00:00:00+02:002023-09-18T00:00:00+02:00tag:blog.notmyidea.org,2023-09-18:/creating-a-simple-command-line-to-post-snippets-on-gitlab.html <p>I&#8217;m trying to get away from Github, and one thing that I find useful is the <a href="https://gist.github.com">gist</a> utility they&#8217;re providing. Seems that gitlab provides a similar&nbsp;tool.</p> <p>You can use it using <a href="https://python-gitlab.readthedocs.io/">python-gitlab</a>:</p> <div class="highlight"><pre><span></span><code>pipx<span class="w"> </span>install<span class="w"> </span>python-gitlab </code></pre></div> <p>And then&nbsp;:</p> <div class="highlight"><pre><span></span><code>gitlab<span class="w"> </span>snippet<span class="w"> </span>create<span class="w"> </span>--title<span class="o">=</span><span class="s2">&quot;youpi&quot;</span><span class="w"> </span>--file-name<span class="o">=</span><span class="s2">&quot;snip.py&quot;</span><span class="w"> </span>--content<span class="w"> </span>snip …</code></pre></div> <p>I&#8217;m trying to get away from Github, and one thing that I find useful is the <a href="https://gist.github.com">gist</a> utility they&#8217;re providing. Seems that gitlab provides a similar&nbsp;tool.</p> <p>You can use it using <a href="https://python-gitlab.readthedocs.io/">python-gitlab</a>:</p> <div class="highlight"><pre><span></span><code>pipx<span class="w"> </span>install<span class="w"> </span>python-gitlab </code></pre></div> <p>And then&nbsp;:</p> <div class="highlight"><pre><span></span><code>gitlab<span class="w"> </span>snippet<span class="w"> </span>create<span class="w"> </span>--title<span class="o">=</span><span class="s2">&quot;youpi&quot;</span><span class="w"> </span>--file-name<span class="o">=</span><span class="s2">&quot;snip.py&quot;</span><span class="w"> </span>--content<span class="w"> </span>snip.py<span class="w"> </span>--visibility<span class="o">=</span><span class="s2">&quot;public&quot;</span> </code></pre></div> <p>I now wanted a small bash script which will just get the name of the file and infer all the parameters. I asked <span class="caps">GPT</span>-4, and iterated on its&nbsp;answer.</p> <p>Here&#8217;s the resulting bash&nbsp;script:</p> <div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span> <span class="k">if</span><span class="w"> </span><span class="o">[</span><span class="w"> </span>-z<span class="w"> </span><span class="s2">&quot;</span><span class="nv">$1</span><span class="s2">&quot;</span><span class="w"> </span><span class="o">]</span> <span class="k">then</span> <span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s2">&quot;Please provide a filename&quot;</span> <span class="w"> </span><span class="nb">exit</span><span class="w"> </span><span class="m">1</span> <span class="k">fi</span> <span class="nv">file</span><span class="o">=</span><span class="s2">&quot;</span><span class="nv">$1</span><span class="s2">&quot;</span> <span class="nv">base</span><span class="o">=</span><span class="k">$(</span>basename<span class="w"> </span><span class="s2">&quot;</span><span class="nv">$file</span><span class="s2">&quot;</span><span class="k">)</span> <span class="nv">title</span><span class="o">=</span><span class="s2">&quot;</span><span class="nv">$base</span><span class="s2">&quot;</span> <span class="nv">visibility</span><span class="o">=</span><span class="s2">&quot;public&quot;</span> <span class="c1"># Use `cat` to fetch the content of the file</span> <span class="nv">content</span><span class="o">=</span><span class="k">$(</span>cat<span class="w"> </span><span class="s2">&quot;</span><span class="nv">$file</span><span class="s2">&quot;</span><span class="k">)</span> <span class="nv">result</span><span class="o">=</span><span class="k">$(</span>gitlab<span class="w"> </span>snippet<span class="w"> </span>create<span class="w"> </span>--title<span class="o">=</span><span class="s2">&quot;</span><span class="nv">$title</span><span class="s2">&quot;</span><span class="w"> </span>--file-name<span class="o">=</span><span class="s2">&quot;</span><span class="nv">$title</span><span class="s2">&quot;</span><span class="w"> </span>--content<span class="o">=</span><span class="s2">&quot;</span><span class="nv">$content</span><span class="s2">&quot;</span><span class="w"> </span>--visibility<span class="o">=</span><span class="s2">&quot;</span><span class="nv">$visibility</span><span class="s2">&quot;</span><span class="k">)</span> <span class="nv">id</span><span class="o">=</span><span class="k">$(</span><span class="nb">echo</span><span class="w"> </span><span class="s2">&quot;</span><span class="nv">$result</span><span class="s2">&quot;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span><span class="s1">&#39;/id: / { print $2 }&#39;</span><span class="k">)</span> <span class="nb">echo</span><span class="w"> </span><span class="s2">&quot;https://gitlab.com/-/snippets/</span><span class="nv">$id</span><span class="s2">&quot;</span> </code></pre></div> <p>I can now do <code>snip README.md</code> and that will create the snippet for me&nbsp;:-)</p>Creating an online space to share markdown files2023-09-17T00:00:00+02:002023-09-17T00:00:00+02:00tag:blog.notmyidea.org,2023-09-17:/creating-an-online-space-to-share-markdown-files.html <p>I wanted to create a space on my server where I can upload markdown files and have them rendered directly, for them to be shared with other&nbsp;people.</p> <p>I stumbled on <a href="https://github.com/ukarim/ngx_markdown_filter_module">the markdown module for nginx</a> which does exactly what I want, but seemed to ask for compilation of nginx …</p> <p>I wanted to create a space on my server where I can upload markdown files and have them rendered directly, for them to be shared with other&nbsp;people.</p> <p>I stumbled on <a href="https://github.com/ukarim/ngx_markdown_filter_module">the markdown module for nginx</a> which does exactly what I want, but seemed to ask for compilation of nginx, which wasn&#8217;t exactly what I wanted in terms of maintainability (it would make it complicated to update&nbsp;it)</p> <p>I then thought that the <a href="https://caddyserver.com/">Caddy</a> server does that by default, and so I&#8217;ve tested it out. Turns out it&#8217;s not, but it offers ways to do this thanks to its template&nbsp;mecanism.</p> <p>It also, <a href="https://caddyserver.com/docs/automatic-https">setups automatically and transparently <span class="caps">SSL</span> certificates</a> for you (using Let&#8217;s Encrypt!), so I wanted to have a&nbsp;look.</p> <p>Here is the Caddy configuration file I&#8217;m now using&nbsp;:</p> <div class="highlight"><pre><span></span><code>md.notmyidea.org { root * /home/caddy/md.notmyidea.org rewrite * /index.html file_server templates encode zstd gzip } </code></pre></div> <p>And the&nbsp;template:</p> <div class="highlight"><pre><span></span><code>{{$pathParts := splitList &quot;/&quot; .OriginalReq.URL.Path}} {{$markdownFilename := default &quot;index&quot; (slice $pathParts 1 | join &quot;/&quot;)}} {{if not (fileExists $markdownFilename)}} {{httpError 404}} {{end}} {{$markdownFile := (include $markdownFilename | splitFrontMatter)}} <span class="cp">&lt;!DOCTYPE html&gt;</span> <span class="p">&lt;</span><span class="nt">html</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">head</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">title</span><span class="p">&gt;</span>{{ $markdownFilename }}<span class="p">&lt;/</span><span class="nt">title</span><span class="p">&gt;</span> <span class="p">&lt;/</span><span class="nt">head</span><span class="p">&gt;</span> <span class="p">&lt;</span><span class="nt">body</span><span class="p">&gt;</span> {{ markdown $markdownFile.Body }} <span class="p">&lt;/</span><span class="nt">body</span><span class="p">&gt;</span> <span class="p">&lt;/</span><span class="nt">html</span><span class="p">&gt;</span> </code></pre></div> <p>This is a minimalistic version, but it works&nbsp;:-)</p>Conversion d’un fichier svg en favicon.ico2023-09-13T00:00:00+02:002023-09-13T00:00:00+02:00tag:blog.notmyidea.org,2023-09-13:/conversion-dun-fichier-svg-en-faviconico.html <p>Il y a plusieurs sites qui permettent de faire ça automatiquement, mais j&#8217;aime bien faire les choses depuis mon terminal, voici donc une commande qui permet de faire ça simplement, en utilisant <a href="https://imagemagick.org/">ImageMagick</a>. Merci à <a href="https://gist.github.com/azam/3b6995a29b9f079282f3">ce&nbsp;gist</a></p> <div class="highlight"><pre><span></span><code>convert<span class="w"> </span>-density<span class="w"> </span>256x256<span class="w"> </span>-background<span class="w"> </span>transparent<span class="w"> </span>favicon.svg<span class="w"> </span>-define<span class="w"> </span>icon:auto-resize<span class="w"> </span>-colors<span class="w"> </span><span class="m">256 …</span></code></pre></div> <p>Il y a plusieurs sites qui permettent de faire ça automatiquement, mais j&#8217;aime bien faire les choses depuis mon terminal, voici donc une commande qui permet de faire ça simplement, en utilisant <a href="https://imagemagick.org/">ImageMagick</a>. Merci à <a href="https://gist.github.com/azam/3b6995a29b9f079282f3">ce&nbsp;gist</a></p> <div class="highlight"><pre><span></span><code>convert<span class="w"> </span>-density<span class="w"> </span>256x256<span class="w"> </span>-background<span class="w"> </span>transparent<span class="w"> </span>favicon.svg<span class="w"> </span>-define<span class="w"> </span>icon:auto-resize<span class="w"> </span>-colors<span class="w"> </span><span class="m">256</span><span class="w"> </span>favicon.ico </code></pre></div>Découverte de nouveaux outils pour le développement: LLM, Helix et plus2023-09-12T00:00:00+02:002023-09-12T00:00:00+02:00tag:blog.notmyidea.org,2023-09-12:/decouverte-de-nouveaux-outils-pour-le-developpement-llm-helix-et-plus.html <h2 id="llm"><span class="caps">LLM</span></h2> <ul> <li><a href="https://localai.io/model-compatibility/">LocalAI</a> permet de faire tourner des modèles en local avec la même <span class="caps">API</span> <span class="caps">HTTP</span> que celle&nbsp;d&#8217;OpenAI</li> <li><a href="https://github.com/bofenghuang/vigogne">Le modèle Vigogne</a> est un modèle entrainé (<em>fine-tuned</em>) avec des données en Français. Notamment <a href="https://huggingface.co/bofenghuang/vigogne-2-7b-chat/tree/v1.0">ce modèle</a>qui prends <span class="caps">LLAMA2</span> en&nbsp;entrée.</li> <li><a href="https://python.langchain.com/docs/get_started/introduction.html">LangChain</a> semble être un framework pour travailler avec les différents …</li></ul> <h2 id="llm"><span class="caps">LLM</span></h2> <ul> <li><a href="https://localai.io/model-compatibility/">LocalAI</a> permet de faire tourner des modèles en local avec la même <span class="caps">API</span> <span class="caps">HTTP</span> que celle&nbsp;d&#8217;OpenAI</li> <li><a href="https://github.com/bofenghuang/vigogne">Le modèle Vigogne</a> est un modèle entrainé (<em>fine-tuned</em>) avec des données en Français. Notamment <a href="https://huggingface.co/bofenghuang/vigogne-2-7b-chat/tree/v1.0">ce modèle</a>qui prends <span class="caps">LLAMA2</span> en&nbsp;entrée.</li> <li><a href="https://python.langchain.com/docs/get_started/introduction.html">LangChain</a> semble être un framework pour travailler avec les différents concepts utiles. A&nbsp;voir.</li> </ul> <p>Pour la première fois, j&#8217;ai commencé à utiliser un peu plus l&#8217;outil <a href="https://llm.datasette.io">llm</a> pour m&#8217;aider dans les tâches de&nbsp;programmation. </p> <p>!! warning J&#8217;utilise actuellement par défaut le modèle en ligne d&#8217;OpenAI &#8220;<span class="caps">GTP4</span>&#8221;, à travers leur <span class="caps">API</span>. Cela me pose des problèmes éthiques, mais mon approche est pour le moment de voir le type de résultats que j&#8217;obtiens pour ensuite comparer avec des modèles locaux type <span class="caps">LLAMA2</span>.</p> <p>Deux choses que j&#8217;ai trouvées utiles&nbsp;:</p> <div class="highlight"><pre><span></span><code>git<span class="w"> </span>diff<span class="w"> </span><span class="p">|</span><span class="w"> </span>llm<span class="w"> </span><span class="s2">&quot;write me a commit message&quot;</span> git<span class="w"> </span>diff<span class="w"> </span><span class="p">|</span><span class="w"> </span>llm<span class="w"> </span><span class="s2">&quot;find a branch name&quot;</span> </code></pre></div> <p>Qui peuvent faciliter la vie plutôt que de chercher à décrire des choses manuellement (le &#8220;branch-name&#8221;, je le trouve particulièrement&nbsp;utile)</p> <p>J&#8217;ai aussi trouvé l&#8217;idée intéressante d&#8217;automatiser l&#8217;écrite de tests, avec pytest. Bien sur ça ne fait pas tout, mais ça donne un template de départ qui est intéressant&nbsp;:</p> <div class="highlight"><pre><span></span><code>cat<span class="w"> </span>code/test_file.py<span class="w"> </span><span class="p">|</span><span class="w"> </span>llm<span class="w"> </span><span class="s2">&quot;write tests for this, using pytest&quot;</span> </code></pre></div> <p>J&#8217;ai par exemple demandé à llm de me lister 20 titres pour cet article&nbsp;:</p> <div class="highlight"><pre><span></span><code>cat<span class="w"> </span>~/dev/til.notmyidea.org/content/<span class="k">$(</span>date<span class="w"> </span>+%F<span class="k">)</span>.md<span class="w"> </span><span class="p">|</span><span class="w"> </span>llm<span class="w"> </span><span class="s2">&quot;Liste moi 20 titres pour cet article. Quelque chose de simple&quot;</span> <span class="m">1</span>.<span class="w"> </span><span class="s2">&quot;Découverte de nouveaux outils pour le développement: LLM, Helix et plus&quot;</span> <span class="m">2</span>.<span class="w"> </span><span class="s2">&quot;Explorer les modèles LLM pour la génération de code&quot;</span> <span class="m">3</span>.<span class="w"> </span><span class="s2">&quot;L&#39;amélioration du développement grâce à LLM et Helix&quot;</span> <span class="m">4</span>.<span class="w"> </span><span class="s2">&quot;Introduction à l&#39;éditeur Helix et à LLM&quot;</span> <span class="m">5</span>.<span class="w"> </span><span class="s2">&quot;Boostez votre productivité de développement avec LLM et Helix&quot;</span> <span class="m">6</span>.<span class="w"> </span><span class="s2">&quot;Comment j&#39;intègre LLM et Helix dans mon flux de travail de développement&quot;</span> <span class="m">7</span>.<span class="w"> </span><span class="s2">&quot;Les avantages de l&#39;utilisation de LLM pour la programmation&quot;</span> <span class="m">8</span>.<span class="w"> </span><span class="s2">&quot;Approches innovantes pour le développement de code avec LLM&quot;</span> <span class="m">9</span>.<span class="w"> </span><span class="s2">&quot;L&#39;avenir du développement de code avec LLM et Helix&quot;</span> <span class="m">10</span>.<span class="w"> </span><span class="s2">&quot;Optimisation des tâches de programmation avec LLM&quot;</span> <span class="m">11</span>.<span class="w"> </span><span class="s2">&quot;Ouvrir de nouvelles possibilités de codage avec LLM et Helix&quot;</span> <span class="m">12</span>.<span class="w"> </span><span class="s2">&quot;Utiliser des modèles LLM pour améliorer la qualité du code&quot;</span> </code></pre></div> <h2 id="helix">Helix</h2> <p>Je me mets à tester un peu plus l&#8217;éditeur Helix. Je suis un habitué de Vim pour toutes les petites modifications que je fais depuis mon terminal, et Helix semble&nbsp;intéressant.</p> <ul> <li><code>x</code>pour sélectionner une ligne (<code>xd</code> pour sélectionner et&nbsp;supprimer)</li> <li><code>ma</code> pour sélectionner &#8220;around&#8221; quelque chose. <code>maw</code> pour sélectionner le&nbsp;mot</li> <li><code>c</code> pour effacer la sélection et passer en mode&nbsp;insertion.</li> </ul> <h2 id="divers">Divers</h2> <blockquote> <p>J’ai fait confiance, j’ai appris. — <a href="https://d%C3%A9tour.studio">Thomas</a></p> </blockquote> <p>J&#8217;aime beaucoup ce que ça dit. Faire confiance est peut-être nécessaire, même si on est déçu au final, on aura au moins appris. Ça me&nbsp;touche.</p>Running the Gitlab CI locally2023-08-19T00:00:00+02:002023-08-19T00:00:00+02:00tag:blog.notmyidea.org,2023-08-19:/running-the-gitlab-ci-locally.html <p>Sometimes, I need to change how the continuous integration is setup, and I find myself pushing to a branch to test if my changes are working. Oftentimes, it takes me multiple commits to find the correct configuration, which is…&nbsp;suboptimal.</p> <p>I discovered today <a href="https://github.com/firecow/gitlab-ci-local">Gitlab <span class="caps">CI</span> local</a> which makes it possible …</p> <p>Sometimes, I need to change how the continuous integration is setup, and I find myself pushing to a branch to test if my changes are working. Oftentimes, it takes me multiple commits to find the correct configuration, which is…&nbsp;suboptimal.</p> <p>I discovered today <a href="https://github.com/firecow/gitlab-ci-local">Gitlab <span class="caps">CI</span> local</a> which makes it possible to run the <span class="caps">CI</span> actions locally, without having to push to the remote <span class="caps">CI</span>. The same thing exists for <a href="https://github.com/nektos/act">Microsoft Github</a>.</p> <p>Under the hood, it&#8217;s using Docker, so you need to have it running on your system, but once it&#8217;s done, you just have to issue a simple command to see the results. Very helpful&nbsp;:-)</p> <p>Here is an example&nbsp;:</p> <div class="highlight"><pre><span></span><code>$<span class="w"> </span>gitlab-ci-local<span class="w"> </span><span class="nb">test</span> parsing<span class="w"> </span>and<span class="w"> </span>downloads<span class="w"> </span>finished<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">41</span><span class="w"> </span>ms <span class="nb">test</span><span class="w"> </span>starting<span class="w"> </span>python:3.8-alpine<span class="w"> </span><span class="o">(</span><span class="nb">test</span><span class="o">)</span> <span class="nb">test</span><span class="w"> </span>copied<span class="w"> </span>to<span class="w"> </span>docker<span class="w"> </span>volumes<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">4</span>.05<span class="w"> </span>s <span class="nb">test</span><span class="w"> </span>$<span class="w"> </span>apk<span class="w"> </span>update<span class="w"> </span><span class="o">&amp;&amp;</span><span class="w"> </span>apk<span class="w"> </span>add<span class="w"> </span>make<span class="w"> </span>libsass<span class="w"> </span>gcc<span class="w"> </span>musl-dev<span class="w"> </span>g++ <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>fetch<span class="w"> </span>https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/APKINDEX.tar.gz <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>fetch<span class="w"> </span>https://dl-cdn.alpinelinux.org/alpine/v3.18/community/aarch64/APKINDEX.tar.gz <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>v3.18.3-55-g2ee93b9273a<span class="w"> </span><span class="o">[</span>https://dl-cdn.alpinelinux.org/alpine/v3.18/main<span class="o">]</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>v3.18.3-56-g4a3b0382caa<span class="w"> </span><span class="o">[</span>https://dl-cdn.alpinelinux.org/alpine/v3.18/community<span class="o">]</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>OK:<span class="w"> </span><span class="m">19939</span><span class="w"> </span>distinct<span class="w"> </span>packages<span class="w"> </span>available <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">1</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libgcc<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">2</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libstdc++<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">3</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libstdc++-dev<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">4</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>zstd-libs<span class="w"> </span><span class="o">(</span><span class="m">1</span>.5.5-r4<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">5</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>binutils<span class="w"> </span><span class="o">(</span><span class="m">2</span>.40-r7<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">6</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libgomp<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">7</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libatomic<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">8</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>gmp<span class="w"> </span><span class="o">(</span><span class="m">6</span>.2.1-r3<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">9</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>isl26<span class="w"> </span><span class="o">(</span><span class="m">0</span>.26-r1<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">10</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>mpfr4<span class="w"> </span><span class="o">(</span><span class="m">4</span>.2.0_p12-r0<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">11</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>mpc1<span class="w"> </span><span class="o">(</span><span class="m">1</span>.3.1-r1<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">12</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>gcc<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">13</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>musl-dev<span class="w"> </span><span class="o">(</span><span class="m">1</span>.2.4-r1<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">14</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libc-dev<span class="w"> </span><span class="o">(</span><span class="m">0</span>.7.2-r5<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">15</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>g++<span class="w"> </span><span class="o">(</span><span class="m">12</span>.2.1_git20220924-r10<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">16</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>libsass<span class="w"> </span><span class="o">(</span><span class="m">3</span>.6.5-r0<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">(</span><span class="m">17</span>/17<span class="o">)</span><span class="w"> </span>Installing<span class="w"> </span>make<span class="w"> </span><span class="o">(</span><span class="m">4</span>.4.1-r1<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Executing<span class="w"> </span>busybox-1.36.1-r2.trigger <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>OK:<span class="w"> </span><span class="m">246</span><span class="w"> </span>MiB<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">55</span><span class="w"> </span>packages <span class="nb">test</span><span class="w"> </span>$<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-r<span class="w"> </span>requirements.txt <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pelican <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>pelican-4.8.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">1</span>.4<span class="w"> </span>MB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">1</span>.4/1.4<span class="w"> </span>MB<span class="w"> </span><span class="m">539</span>.9<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>markdown <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>Markdown-3.4.4-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">94</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">94</span>.2/94.2<span class="w"> </span>kB<span class="w"> </span><span class="m">540</span>.1<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>typogrify <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>typogrify-2.0.7.tar.gz<span class="w"> </span><span class="o">(</span><span class="m">12</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Preparing<span class="w"> </span>metadata<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>started <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Preparing<span class="w"> </span>metadata<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>finished<span class="w"> </span>with<span class="w"> </span>status<span class="w"> </span><span class="s1">&#39;done&#39;</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pelican-search <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>pelican_search-1.1.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">6</span>.6<span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pelican-neighbors <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>pelican_neighbors-1.2.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">16</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pelican-webassets <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>pelican_webassets-2.0.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">5</span>.8<span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>libsass <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>libsass-0.22.0.tar.gz<span class="w"> </span><span class="o">(</span><span class="m">316</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">316</span>.3/316.3<span class="w"> </span>kB<span class="w"> </span><span class="m">552</span>.1<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Preparing<span class="w"> </span>metadata<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>started <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Preparing<span class="w"> </span>metadata<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>finished<span class="w"> </span>with<span class="w"> </span>status<span class="w"> </span><span class="s1">&#39;done&#39;</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>docutils&gt;<span class="o">=</span><span class="m">0</span>.16 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>docutils-0.20.1-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">572</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">572</span>.7/572.7<span class="w"> </span>kB<span class="w"> </span><span class="m">549</span>.2<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>rich&gt;<span class="o">=</span><span class="m">10</span>.1 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>rich-13.5.2-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">239</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">239</span>.7/239.7<span class="w"> </span>kB<span class="w"> </span><span class="m">485</span>.3<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>jinja2&gt;<span class="o">=</span><span class="m">2</span>.7 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>Jinja2-3.1.2-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">133</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">133</span>.1/133.1<span class="w"> </span>kB<span class="w"> </span><span class="m">342</span>.6<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pytz&gt;<span class="o">=</span><span class="m">2020</span>.1 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>pytz-2023.3-py2.py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">502</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">502</span>.3/502.3<span class="w"> </span>kB<span class="w"> </span><span class="m">547</span>.3<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>pygments&gt;<span class="o">=</span><span class="m">2</span>.6 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>Pygments-2.16.1-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">1</span>.2<span class="w"> </span>MB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">1</span>.2/1.2<span class="w"> </span>MB<span class="w"> </span><span class="m">551</span>.4<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>unidecode&gt;<span class="o">=</span><span class="m">1</span>.1 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>Unidecode-1.3.6-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">235</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">235</span>.9/235.9<span class="w"> </span>kB<span class="w"> </span><span class="m">554</span>.2<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>blinker&gt;<span class="o">=</span><span class="m">1</span>.4 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>blinker-1.6.2-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">13</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>python-dateutil&gt;<span class="o">=</span><span class="m">2</span>.8 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>python_dateutil-2.8.2-py2.py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">247</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">247</span>.7/247.7<span class="w"> </span>kB<span class="w"> </span><span class="m">235</span>.7<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>feedgenerator&gt;<span class="o">=</span><span class="m">1</span>.9 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>feedgenerator-2.1.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">21</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>importlib-metadata&gt;<span class="o">=</span><span class="m">4</span>.4 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>importlib_metadata-6.8.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">22</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>smartypants&gt;<span class="o">=</span><span class="m">1</span>.8.3 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>smartypants-2.0.1-py2.py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">9</span>.9<span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>rtoml&lt;<span class="m">0</span>.10.0,&gt;<span class="o">=</span><span class="m">0</span>.9.0 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>rtoml-0.9.0-cp38-cp38-musllinux_1_1_aarch64.whl<span class="w"> </span><span class="o">(</span><span class="m">846</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">846</span>.2/846.2<span class="w"> </span>kB<span class="w"> </span><span class="m">503</span>.7<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>webassets&lt;<span class="m">3</span>.0,&gt;<span class="o">=</span><span class="m">2</span>.0 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>webassets-2.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">142</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">142</span>.9/142.9<span class="w"> </span>kB<span class="w"> </span><span class="m">551</span>.8<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>zipp&gt;<span class="o">=</span><span class="m">0</span>.5 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>zipp-3.16.2-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">7</span>.2<span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>MarkupSafe&gt;<span class="o">=</span><span class="m">2</span>.0 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>MarkupSafe-2.1.3-cp38-cp38-musllinux_1_1_aarch64.whl<span class="w"> </span><span class="o">(</span><span class="m">30</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>six&gt;<span class="o">=</span><span class="m">1</span>.5 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>six-1.16.0-py2.py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">11</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>markdown-it-py&gt;<span class="o">=</span><span class="m">2</span>.2.0 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>markdown_it_py-3.0.0-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">87</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<span class="w"> </span><span class="m">87</span>.5/87.5<span class="w"> </span>kB<span class="w"> </span><span class="m">561</span>.7<span class="w"> </span>kB/s<span class="w"> </span>eta<span class="w"> </span><span class="m">0</span>:00:00 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>typing-extensions&lt;<span class="m">5</span>.0,&gt;<span class="o">=</span><span class="m">4</span>.0.0 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>typing_extensions-4.7.1-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">33</span><span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Collecting<span class="w"> </span>mdurl~<span class="o">=</span><span class="m">0</span>.1 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Downloading<span class="w"> </span>mdurl-0.1.2-py3-none-any.whl<span class="w"> </span><span class="o">(</span><span class="m">10</span>.0<span class="w"> </span>kB<span class="o">)</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheels<span class="w"> </span><span class="k">for</span><span class="w"> </span>collected<span class="w"> </span>packages:<span class="w"> </span>typogrify,<span class="w"> </span>libsass <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>typogrify<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>started <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>typogrify<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>finished<span class="w"> </span>with<span class="w"> </span>status<span class="w"> </span><span class="s1">&#39;done&#39;</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Created<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>typogrify:<span class="w"> </span><span class="nv">filename</span><span class="o">=</span>typogrify-2.0.7-py2.py3-none-any.whl<span class="w"> </span><span class="nv">size</span><span class="o">=</span><span class="m">13452</span><span class="w"> </span><span class="nv">sha256</span><span class="o">=</span>4ce329903e807671102eab7fd2bc49765b6efc3a4ae68c82053318b62789083c <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Stored<span class="w"> </span><span class="k">in</span><span class="w"> </span>directory:<span class="w"> </span>/root/.cache/pip/wheels/0b/e9/98/c888501e8dd2166da059e4f8418694de9b50b48a7192712be9 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>libsass<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>started <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>libsass<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>still<span class="w"> </span>running... <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Building<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>libsass<span class="w"> </span><span class="o">(</span>setup.py<span class="o">)</span>:<span class="w"> </span>finished<span class="w"> </span>with<span class="w"> </span>status<span class="w"> </span><span class="s1">&#39;done&#39;</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Created<span class="w"> </span>wheel<span class="w"> </span><span class="k">for</span><span class="w"> </span>libsass:<span class="w"> </span><span class="nv">filename</span><span class="o">=</span>libsass-0.22.0-cp38-abi3-linux_aarch64.whl<span class="w"> </span><span class="nv">size</span><span class="o">=</span><span class="m">13710320</span><span class="w"> </span><span class="nv">sha256</span><span class="o">=</span>3dcb4ce97c1aafc179a6343e0f312c17df88e56c4eb647ab54b09ead5ee00b92 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Stored<span class="w"> </span><span class="k">in</span><span class="w"> </span>directory:<span class="w"> </span>/root/.cache/pip/wheels/95/64/fa/47638d5037df216387cdc168e9871d5d9851fc995d636bd108 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Successfully<span class="w"> </span>built<span class="w"> </span>typogrify<span class="w"> </span>libsass <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Installing<span class="w"> </span>collected<span class="w"> </span>packages:<span class="w"> </span>webassets,<span class="w"> </span>smartypants,<span class="w"> </span>pytz,<span class="w"> </span>zipp,<span class="w"> </span>unidecode,<span class="w"> </span>typogrify,<span class="w"> </span>typing-extensions,<span class="w"> </span>six,<span class="w"> </span>rtoml,<span class="w"> </span>pygments,<span class="w"> </span>mdurl,<span class="w"> </span>MarkupSafe,<span class="w"> </span>libsass,<span class="w"> </span>feedgenerator,<span class="w"> </span>docutils,<span class="w"> </span>blinker,<span class="w"> </span>python-dateutil,<span class="w"> </span>markdown-it-py,<span class="w"> </span>jinja2,<span class="w"> </span>importlib-metadata,<span class="w"> </span>rich,<span class="w"> </span>markdown,<span class="w"> </span>pelican,<span class="w"> </span>pelican-webassets,<span class="w"> </span>pelican-search,<span class="w"> </span>pelican-neighbors <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Successfully<span class="w"> </span>installed<span class="w"> </span>MarkupSafe-2.1.3<span class="w"> </span>blinker-1.6.2<span class="w"> </span>docutils-0.20.1<span class="w"> </span>feedgenerator-2.1.0<span class="w"> </span>importlib-metadata-6.8.0<span class="w"> </span>jinja2-3.1.2<span class="w"> </span>libsass-0.22.0<span class="w"> </span>markdown-3.4.4<span class="w"> </span>markdown-it-py-3.0.0<span class="w"> </span>mdurl-0.1.2<span class="w"> </span>pelican-4.8.0<span class="w"> </span>pelican-neighbors-1.2.0<span class="w"> </span>pelican-search-1.1.0<span class="w"> </span>pelican-webassets-2.0.0<span class="w"> </span>pygments-2.16.1<span class="w"> </span>python-dateutil-2.8.2<span class="w"> </span>pytz-2023.3<span class="w"> </span>rich-13.5.2<span class="w"> </span>rtoml-0.9.0<span class="w"> </span>six-1.16.0<span class="w"> </span>smartypants-2.0.1<span class="w"> </span>typing-extensions-4.7.1<span class="w"> </span>typogrify-2.0.7<span class="w"> </span>unidecode-1.3.6<span class="w"> </span>webassets-2.0<span class="w"> </span>zipp-3.16.2 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>WARNING:<span class="w"> </span>Running<span class="w"> </span>pip<span class="w"> </span>as<span class="w"> </span>the<span class="w"> </span><span class="s1">&#39;root&#39;</span><span class="w"> </span>user<span class="w"> </span>can<span class="w"> </span>result<span class="w"> </span><span class="k">in</span><span class="w"> </span>broken<span class="w"> </span>permissions<span class="w"> </span>and<span class="w"> </span>conflicting<span class="w"> </span>behaviour<span class="w"> </span>with<span class="w"> </span>the<span class="w"> </span>system<span class="w"> </span>package<span class="w"> </span>manager.<span class="w"> </span>It<span class="w"> </span>is<span class="w"> </span>recommended<span class="w"> </span>to<span class="w"> </span>use<span class="w"> </span>a<span class="w"> </span>virtual<span class="w"> </span>environment<span class="w"> </span>instead:<span class="w"> </span>https://pip.pypa.io/warnings/venv <span class="nb">test</span><span class="w"> </span>&gt; <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">[</span>notice<span class="o">]</span><span class="w"> </span>A<span class="w"> </span>new<span class="w"> </span>release<span class="w"> </span>of<span class="w"> </span>pip<span class="w"> </span>is<span class="w"> </span>available:<span class="w"> </span><span class="m">23</span>.0.1<span class="w"> </span>-&gt;<span class="w"> </span><span class="m">23</span>.2.1 <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="o">[</span>notice<span class="o">]</span><span class="w"> </span>To<span class="w"> </span>update,<span class="w"> </span>run:<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>--upgrade<span class="w"> </span>pip <span class="nb">test</span><span class="w"> </span>$<span class="w"> </span>make<span class="w"> </span>publish <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span><span class="s2">&quot;pelican&quot;</span><span class="w"> </span><span class="s2">&quot;/gcl-builds/content&quot;</span><span class="w"> </span>-o<span class="w"> </span><span class="s2">&quot;/gcl-builds/public&quot;</span><span class="w"> </span>-s<span class="w"> </span><span class="s2">&quot;/gcl-builds/publishconf.py&quot;</span> <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>Done:<span class="w"> </span>Processed<span class="w"> </span><span class="m">5</span><span class="w"> </span>articles,<span class="w"> </span><span class="m">0</span><span class="w"> </span>drafts,<span class="w"> </span><span class="m">0</span><span class="w"> </span>hidden<span class="w"> </span>articles,<span class="w"> </span><span class="m">2</span><span class="w"> </span>pages,<span class="w"> </span><span class="m">0</span><span class="w"> </span>hidden<span class="w"> </span>pages <span class="nb">test</span><span class="w"> </span>&gt;<span class="w"> </span>and<span class="w"> </span><span class="m">0</span><span class="w"> </span>draft<span class="w"> </span>pages<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.50<span class="w"> </span>seconds. <span class="nb">test</span><span class="w"> </span>finished<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">6</span><span class="w"> </span>min PASS<span class="w"> </span><span class="nb">test</span> </code></pre></div>ArchLinux et mise à jour du keyring2023-08-18T00:00:00+02:002023-08-18T00:00:00+02:00tag:blog.notmyidea.org,2023-08-18:/archlinux-et-mise-a-jour-du-keyring.html <p>Pour les mises à jour Arch, j&#8217;utilise <a href="https://github.com/Jguer/yay">yay</a>. Je ne fais les mises à jour que de manière semi-régulière, et parfois après une longue période je me retrouve avec des soucis de clé qui ne sont plus à jour ou&nbsp;manquantes.</p> <p>Avec une utilisation fréquente du système, aucun problème …</p> <p>Pour les mises à jour Arch, j&#8217;utilise <a href="https://github.com/Jguer/yay">yay</a>. Je ne fais les mises à jour que de manière semi-régulière, et parfois après une longue période je me retrouve avec des soucis de clé qui ne sont plus à jour ou&nbsp;manquantes.</p> <p>Avec une utilisation fréquente du système, aucun problème ne se pose car un service s&#8217;occupe de faire la mise à jour des clés de manière&nbsp;automatique.</p> <p>Pour résoudre le souci, il suffit de mettre à jour le paquet <code>archlinux-keyring</code>, comme décrit <a href="https://wiki.archlinux.org/title/Pacman/Package_signing">dans la page Wiki qui va bien</a>.</p> <div class="highlight"><pre><span></span><code>sudo pacman -S archlinux-keyring </code></pre></div>Python packaging with Hatch, pipx and Zsh environment variables2023-08-17T00:00:00+02:002023-08-17T00:00:00+02:00tag:blog.notmyidea.org,2023-08-17:/python-packaging-with-hatch-pipx-and-zsh-environment-variables.html <p>It&#8217;s been a while I didn&#8217;t packaged something new. I recently remembered an old package of mine that needed some attention : <a href="https://gitlab.com/almet/debts">debts</a>. It&#8217;s now time to package it, so I discovered <a href="https://hatch.pypa.io/">hatch</a></p> <p>hatch new&nbsp;&#8212;init</p> <p>This does the heavy-lifting for you, actually porting the <code>setup.py</code> files …</p> <p>It&#8217;s been a while I didn&#8217;t packaged something new. I recently remembered an old package of mine that needed some attention : <a href="https://gitlab.com/almet/debts">debts</a>. It&#8217;s now time to package it, so I discovered <a href="https://hatch.pypa.io/">hatch</a></p> <p>hatch new&nbsp;&#8212;init</p> <p>This does the heavy-lifting for you, actually porting the <code>setup.py</code> files to the new way of packaging with python (with a <code>pyproject.toml</code> file)</p> <p>Then <code>hatch shell</code> will create a development environment, install dependencies, check the <code>pyproject.toml</code> file in one command, and give you a shell to test whatever you need to&nbsp;test.</p> <h2 id="isolating-system-packages">Isolating system&nbsp;packages</h2> <p>I discovered that <a href="https://github.com/pypa/pipx">pipx</a> is a convenient way to install user-facing applications on my system. I use multiple virtual environments for my different projects, but not for the install that are used&nbsp;system-wide.</p> <p>pipx seems to solve this, and avoid using <code>sudo pip install x</code>.</p> <h2 id="manipulating-env-variables-with-zsh">Manipulating env variables with&nbsp;Zsh</h2> <p>I use <a href="https://www.zsh.org/">Zsh</a> as my main shell for years, and I just discovered that it&#8217;s possible to manipulate environment variables in an easy&nbsp;way.</p> <p>If you&#8217;re like me, you never remember how to add something to your path. You can actually use <code>+=</code>, like&nbsp;this:</p> <div class="highlight"><pre><span></span><code><span class="nv">path</span><span class="o">+=(</span><span class="s1">&#39;/Users/alexis/.local/bin&#39;</span><span class="o">)</span> <span class="nb">export</span><span class="w"> </span>PATH </code></pre></div>Profiling and speeding up Django and Pytest2023-08-16T00:00:00+02:002023-08-16T00:00:00+02:00tag:blog.notmyidea.org,2023-08-16:/profiling-and-speeding-up-django-and-pytest.html <p><a href="https://yaal.coop/">Éloi</a> made <a href="https://github.com/spiral-project/ihatemoney/issues/1214">a pull request on IHateMoney</a> to speedup the tests, with some great tooling for pytest that I wasn&#8217;t aware&nbsp;of:</p> <ul> <li><a href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a> allows to run tests in parallel, using <code>-n auto</code></li> <li><a href="https://pypi.org/project/pytest-profiling/">pytest-profiling</a> makes it easy to get the call stack and time the function calls that take most …</li></ul> <p><a href="https://yaal.coop/">Éloi</a> made <a href="https://github.com/spiral-project/ihatemoney/issues/1214">a pull request on IHateMoney</a> to speedup the tests, with some great tooling for pytest that I wasn&#8217;t aware&nbsp;of:</p> <ul> <li><a href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a> allows to run tests in parallel, using <code>-n auto</code></li> <li><a href="https://pypi.org/project/pytest-profiling/">pytest-profiling</a> makes it easy to get the call stack and time the function calls that take most of the&nbsp;time.</li> <li>You can them analyse the <code>.prof</code> files with <a href="https://pypi.org/project/snakeviz/">Snakeviz</a></li> </ul> <p>So, I spent some time using these on the tests for <a href="https://chariotte.fr">La Chariotte</a>, because they were&nbsp;slow.</p> <p>I found two things&nbsp;:</p> <ul> <li>Login calls are costly in the test, and it&#8217;s possible to speed things up&nbsp;;</li> <li>On my machine, calls to resolve my hostname were slow, using 5s during the tests for a lookup that wasn&#8217;t even&nbsp;useful.</li> </ul> <h2 id="changing-the-hashing-algorithm-to-speedup-tests">Changing the hashing algorithm to speedup&nbsp;tests</h2> <p>By default, Django uses a slow (but secure !) hashing mechanism for checking the user credentials. In the tests, we don&#8217;t need this security, but we need the&nbsp;speed.</p> <p>Changing them to use <span class="caps">MD5</span> turns out to be a way to greatly speed them up! Here is how to do it with a pytest fixture&nbsp;:</p> <div class="highlight"><pre><span></span><code><span class="nd">@pytest</span><span class="o">.</span><span class="n">fixture</span><span class="p">(</span><span class="n">autouse</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">def</span> <span class="nf">password_hasher_setup</span><span class="p">(</span><span class="n">settings</span><span class="p">):</span> <span class="c1"># Use a weaker password hasher during tests, for speed</span> <span class="n">settings</span><span class="o">.</span><span class="n">PASSWORD_HASHERS</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">&quot;django.contrib.auth.hashers.MD5PasswordHasher&quot;</span><span class="p">,</span> <span class="p">]</span> </code></pre></div> <h2 id="speeding-dns-lookups">Speeding <span class="caps">DNS</span>&nbsp;lookups</h2> <p>I&#8217;m currently using a MacOSX machine, and for for whatever reason, the local lookup was not configured properly on my machine. I don&#8217;t think I did anything specific to get this wrong, so it might be your case too. Calls to resolve the local domain were tooking&nbsp;5s.</p> <p>If the answer to <code>scutil --get LocalHostName</code>, <code>hostname</code> and <code>scutil --get HostName</code> differ, then you might be in this case. Here is the fix&nbsp;:</p> <div class="highlight"><pre><span></span><code>sudo<span class="w"> </span>scutil<span class="w"> </span>--set<span class="w"> </span>HostName<span class="w"> </span>&lt;YourHostName&gt; </code></pre></div>