Update CRDTs article

content/code/2024-02-24-crdts.md

@@ -1,10 +1,23 @@
 ---
-title: A comparison of different JavaScript CRDTs
+title: A comparison of JavaScript CRDTs
 status: draft
 tags: crdts, umap, sync
 display_toc: true
 ---
 
+This is not done yet, TODO :
+
+- [x] Change the table to be more readable, especially on mobile
+- [x] Update the examples with JSON Joy to have a working example
+- [x] Change the layout to be easier to understand.
+- [ ] Enhance graphics to be easier to understand
+- [ ] Part 3: Key takeaways
+---
+
+[TOC]
+
+---
+
 Collaboration is one of the most requested features on uMap since a long time. We've added a way to merge same-layers edit, but ideally we would love to make things easier to understand and more fluid for the users.
 
 For this reason, I got more into CRDTs, with the goal of understanding how they work, what are the different libraries out there, and which one would be a good fit for us, if any.
@@ -16,33 +29,14 @@ So far, [the way I've though about collaboration features on uMap](https://blog.
 - c) applying the changes on the receiving client.
 
 This works well in general, but it doesn't take care of conflicts handling, especially when a disconnect can happen.
-
+## Part 1 - What are CRDTs?
-## Our requirements
-
-We're looking for something that:
-
-- **Stores and retrieves arbitrary key/value pairs**, like the name of the map, the default color of the markers, the type of layers, etc.
-- **Propagates the changes** to the other parties, so they can see the edits in real-time  ;
-- **Handles disconnections**, so it's possible to work offline, or with an intermittent/bad connection.
-
-In terms of API, we want something reactive and simple. I'm thinking about:
-
-```js
-let store = new Store({websocket:"wss://server"})
-store.on((updates) => {
-	// do something with the updates
-})
-store.set('key', 'value')
-```
-
-## What are CRDTs anyway?
 
 Conflict-free Resolution Data Types (CRDTs) are as a specific datatype able to merge its state with other states without generating conflicts. They handle consistency in distributed systems, making them particularly well-suited for collaborative real-time applications.
 
 CRDTs ensure that multiple participants can make changes without strict coordination, and all replicas converge to the same state upon synchronization, without conflicts.
 
 "Append-only sets" are probably one of the most common type of CRDT: if multiple parties add the same element, it will be present only once. It's our old friend `Set`.
-### How can CRDT help in our case?
+### Why using them?
 
 For uMap, CRDTs offer a solution to several challenges:
 
@@ -54,7 +48,7 @@ For uMap, CRDTs offer a solution to several challenges:
 
 4. **Decentralization**: While uMap currently relies on a central server, adopting CRDTs could pave the way for a more decentralized architecture, increasing resilience and scalability.
 
-### How do CRDTs differ from traditional data synchronization methods?
+### Comparing with traditional data synchronization methods
 
 Traditional data synchronization methods typically rely on a central source of truth, such as a server, to manage and resolve conflicts. When changes are made by different users, these traditional systems require a round-trip to the server for conflict resolution and thus can be slow or inadequate for real-time collaboration.
 
@@ -78,25 +72,6 @@ At first, I found CRDTs somewhat confusing, owing to their role in addressing co
 
 Note that we could also use a library such as [rxdb](https://github.com/pubkey/rxdb) — to handle the syncing, offline, etc — because we have a master: we use the server, and we can use it to handle the merge conflicts. But by doing so, we also give more responsibility to the server, whereas when using CRDTs it's possible to do the merge only on the clients.
 
----
-## The libraries
-
-So, with this in mind, I've tried to have a look at the different libraries that are out there, and assess how they would work for us. 
-
-I'll be comparing these CRDTs, in the context of a JS mapping application, built with Leaflet :
-
-- [Y.js](https://yjs.dev/)
-- [Automerge](https://automerge.org/)
-- [JSON Joy](https://jsonjoy.com)
-
-Here are the different areas I'm interested in :
-
-1. **Type**: Is it state-based or operation-based? What is the impact for us?
-2. **Offline Sync**: How does it handle offline edits and their integration upon reconnection?
-3. **Efficiency**: Probe the bandwidth and storage demands. What's being transmitted over the wire? To test that, I've just connected two peers and added 20 points on each, and looked at the network impact.
-4. **Community and Support**: How is the size and activity of the developer community / ecosystem?
-5. **Size** of the JavaScript library
-6. **Browser support** in general.
 ### Different types of CRDTs
 
 While reading the literature, I found that there are two kinds of CRDTs: state-based and operation-based. So, what do we need ?
@@ -114,52 +89,118 @@ API to interact with them as you're changing the state, so **it doesn't really m
 >
 > [Wikipedia on CRDTs](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)
 
-### Offline support
-
-One thing that I would like to clarify is how does these libs work when peers get
-offline, and back online, if they do something at all.
-
-I was expecting something in the lines of:
-
-1. Connection is lost: changes are applied locally (messages are piling up somehow);
-2. Connection is back: the client syncs with other clients.
-
-Which is exactly what's happening: CRDTs don't do any magic here: they send the "sync" messages whenever they need, which brings all the clients in the same state.
-
 ### How the server fits in the picture
 
-The sync protocol [is well defined and documented](https://www.npmjs.com/package/@automerge/automerge-repo-network-websocket). While discussing with the team, I quickly understood that I was expecting the server to pass along the messages to the other parties, and that would be the way the synchronisation would be done. It turns out I was mistaken: in this approach, the clients send updates to the server, which merges everything together and only then sends the updates to the other peers.
+While discussing with the automerge team, I understood that I was expecting the server to pass along the messages to the other parties, and that would be the way the synchronisation would be done. It turns out I was mistaken: in this approach, the clients send updates to the server, which merges everything together and only then sends the updates to the other peers.
 
 In order to have peers working with each other, I would need to change the way the provider works, so we can have the server be "brainless" and just relay the messages.
 
 For automerge, it would mean the provider will "just" handle the websocket connection (disconnect and reconnect) and all the peers would be able to talk with each other. The other solution for us would be to have the merge algorithm working on the server side, which comes with upsides (no need to find when the document should be saved by the client to the server) and downsides (it takes some cpu and memory to run the CRDTs on the server)
 
-### Versionning
+---
+## Part 2: making a demo with different libraries
 
-A quick note about versioning: it's possible to get a snapshot of the document at any point in time, and it's possible to store this information. 
+Now that we're familiar with CRDTs and how they can help us, let's create a map application which syncs marker positions, on different clients.
 
-## Y.js
+We'll be comparing three JavaScript libraries: [Y.js](https://yjs.dev/), [Automerge](https://automerge.org/) and [JSON Joy](https://jsonjoy.com), considering:
+
+1. **Efficiency**: Probe the bandwidth when doing edits. What's being transmitted over the wire? 
+2. **API**: is it easy to use for our use case? What are the challenging parts for us?
+3. **Community and Support**: How is the size and activity of the developer community / ecosystem?
+4. **Size** of the JavaScript library, because we want to limit the impact on our users browsers.
+
+### A leaflet map
+
+All the demos are made agains the same application, which creates markers when the map is clicked, and move the markers on hover.
+
+Here's the whole code for this:
+
+```js
+import L from "leaflet";
+import "leaflet/dist/leaflet.css";
+
+// Create a map with a default tilelayer.
+const map = L.map("map").setView([48.1173, -1.6778], 13);
+L.tileLayer("https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png", {
+	maxZoom: 19,
+	attribution: "© OpenStreetMap contributors",
+}).addTo(map);
+
+// Features contains a reference to the marker objects, mapped by the uuids
+let features = {};
+
+// An upsert function
+function upsertMarker({ latlng, uuid, local = false }) {
+	if (!uuid) uuid = crypto.randomUUID();
+	let marker;	
+	
+	if (Object.keys(features).includes(uuid)) {
+		marker = features[uuid];
+		marker.setLatLng(latlng);
+	} else {
+		marker = new L.marker(latlng, {
+			draggable: true,
+			autoPan: true,
+			uuid: uuid,
+		});
+	
+		features[uuid] = marker;
+		marker.addTo(map);
+		marker.on("dragend", ({ target }) => {
+			console.log("dragend");
+		});
+	}
+
+	if (!local) {
+		console.log()
+	}
+}
+
+// Add new features to the map with a click
+map.on("click", ({ latlng }) => {
+	upsertMarker({ latlng });
+});
+```
+
+Now, let's add synchronization.
+### Y.js
 
 Y.js is the first library I've looked at, because it's the oldest one, and the more commonly referred to.
 
-The API seem to offer what we look for, and provides a way to [observe changes](https://docs.yjs.dev/api/shared-types/y.map#observing-changes-y.mapevent).
+The API seem to offer what we look for, and provides a way to [observe changes](https://docs.yjs.dev/api/shared-types/y.map#observing-changes-y.mapevent). Here's what I did:
 
 ```js
-const doc = new Y.Doc()
+import * as Y from "yjs";
-const map = ydoc.getMap()
+import { WebsocketProvider } from "y-websocket";
-map.set('key', 'value')
-map.observe((event) => {
-  // read the keys that changed
-  event.keysChanged
   
-  // If I need to iterate on the keys, or get the old values, it's possible.
+// Instanciate a document
-  event.changes.keys.forEach((change, key) {
+const doc = new Y.Doc();
-    // here, change.action can be "add", "update" or "delete"
+```
-    // So it's the right place to know what happened, and act respectively
+
-    // You can get the last value of the key with `map.get`
+To make changes to the CRDT:
-    map.get(key)
+
-  })
+```js
-})
+let markers = doc.getMap("markers");
+markers.set(target.options.uuid, target._latlng);
+```
+
+To observe the changes:
+
+```js
+markers.observe((event, transaction) => {
+  if (!transaction.local) {
+    event.changes.keys.forEach((change, key) => {
+      let value = markers.get(key);
+      if (change.action === "add") {
+        upsertMarker({ latlng: value, uuid: key, local: true });
+      } else if (change.action === "update") {
+        upsertMarker({ latlng: value, uuid: key, local: true });
+      } else if (change.action === "delete") {
+        console.log(`Property "${key}" was deleted. ".`);
+      }
+    });
+  }
+});
 ```
 
 It comes with multiple "providers", which make it possible to sync with different protocols (there is even a way to sync over the matrix protocol 😇). More usefully for us, there is [an implemented protocol for websockets](https://github.com/yjs/y-websocket).
@@ -168,7 +209,11 @@ Using a provider is as easy as:
 
 ```js
 // Sync clients with the y-websocket provider
-const provider = new WebsocketProvider("ws://localhost:1234", "leaflet-sync", doc );
+const provider = new WebsocketProvider(
+	"ws://localhost:1234",
+	"leaflet-sync",
+	doc
+);
 ```
 
 It's also possible to send "awareness" information (some state you don't want to persist, like the position of the cursor). It contains some useful meta information, such as the number of connected peers.
@@ -182,14 +227,14 @@ map.on("mousemove", ({ latlng }) => {
 ```
 
 I made [a quick proof of concept with Y.js](https://gitlab.com/almet/leaflet-sync/-/tree/yjs) in a few hours flawlessly. It handles offline and reconnects, and exposes awareness information.
-### Python
+#### Python bindings
 
 Y.js has been rewritten in rust, with [the Y.rs project](https://github.com/y-crdt/y-crdt), which makes it possible to use with Python (see [Y.py](https://github.com/y-crdt/ypy)) if needed. The project has been implemented quite recently and is currently looking for a maintainer.
-### Library size
+#### Library size
 
 Size: Y.js is 4,16 Ko, Y-Websocket is 21,14 Ko
 The library is currently used in production for large projects such as AFFiNE and Evernote.
-### The data being transmitted
+#### The data being transmitted
 
 In the scenario where all clients connect to a central server, which handle the CRDT locally and then transmits back to other parties, I found that adding 20 points on one client, and then 20 points in another generates ~5 ko of data (~16 bytes per edit).
 
@@ -201,56 +246,54 @@ In the scenario where all clients connect to a central server, which handle the
 | There is awareness support                                                                               |                                                                                                                             |
 
 ---
-## Automerge
+### Automerge
 
 [Automerge](https://automerge.org/) is another library to handle CRDTs. Automerge is actually the low level interface, and there is a higher-level interface exposed as [Automerge-repo](https://automerge.org/docs/repositories/). Here is how to use it:
 
 ```js
-let handle = repo.create()
+import { Repo } from "@automerge/automerge-repo";
 
-handle.change(d => { d.key = "value"})
+const repo = new Repo();
+let handle = repo.create() // or repo.find(name)
 ```
 
 When you change the document, you actually call `change` which makes it possible to do the changes in a kind of "transaction function".
 
+```js
+handle.change((doc) => {
+	doc[uuid] = cleanLatLng(target._latlng);
+});
+```
+
+Note that I had to use a `cleanLatLng` function in order to not pass the whole object, otherwise it wouldn't be serialized. It's really just a simple helper taking the properties of interest for us (and letting away all the rest).
+
 You can observe the changes, getting you the whole list of patches:
 
 ```js
 handle.on("change", ({ doc, patches }) => {
-	patches.forEach(({ action, path }) => {
+  console.log(patches, doc);
-		// Here I'm only taking action when a value is inserted
+  patches.forEach(({ action, path }) => {
-		// At the position "uuid".
+	// We have to know the specifics of the patch operations getting passed
-		if (path.length == 2 && action === "insert") {
+	// two items in the path means we're inserting an object 
-		let value = doc[path[0]];
+    if (path.length == 2 && action === "insert") {
-		// do something here with the value
+      let value = doc[path[0]];
-	}
+      upsertMarker({ latlng: value, uuid: path[0], local: true });
+    }
+  });
 });
 ```
 
-There is a high-level API, with support for [Websocket](https://www.npmjs.com/package/@automerge/automerge-repo-network-websocket):
+#### Python bindings
 
-```js
+There is an [automerge.py](https://github.com/automerge/automerge-py) project, but no changes has been made to it since 3 years ago. There are plans to update it, though.
-import { Repo } from "@automerge/automerge-repo";
-import { BrowserWebSocketClientAdapter } from "@automerge/automerge-repo-network-websocket";
-import { IndexedDBStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb";
 
-const repo = new Repo({
+#### Library size
-	network: [new BrowserWebSocketClientAdapter("wss://sync.automerge.org")],
-	storage: new IndexedDBStorageAdapter(),
-});
-```
-
-### Python
-
-There is an [automerge.py](https://github.com/automerge/automerge-py) project, but no changes has been made to it since 3 years ago.
-
-### Library size
 
 Size: 1,64 mb, total is 1,74 mb. It's relying on Web assembly by default.
 
 The large bundle size is something that the team is aware of, and are working on a solution for. For us, it's important to have something as lightweight as possible, considering CRDTs is only one part of what we're doing, and that mapping can be done in context where connection is not that reliable and fast.
 
-### The data being transmitted
+#### The data being transmitted
 
 In the same scenario, I found that adding 20 points on one client, and then 20 points in another generates 90 messages and 24,94 Ko of data transmitted (~12 Ko sent and ~12Ko received), so approximately 75 bytes per edit.
 
@@ -261,37 +304,65 @@ In the same scenario, I found that adding 20 points on one client, and then 20 p
 | The team was responsive and trying to help.                                                                                | There is no way (at the moment) to tell that a transaction is local or remote (but in practice it wasn't a problem) |
 
 ---
-## JSON Joy
+### JSON Joy
 
-[Json Joy](https://jsonjoy.com) is the latest to the party. It takes another stake at this by providing small libraries with a small functional perimeter. It sounds promising, even if still quite new, and would left us with the hands free in order to implement the protocol that would work for us.
+[Json Joy](https://jsonjoy.com) is the latest to the party.
+
+It takes another stake by providing multiple libraries with a small functional perimeter. It sounds promising, even if still quite new, and would left us with the hands free in order to implement the protocol that would work for us.
 
 ```js
 import {Model} from 'json-joy/es2020/json-crdt';
 import { s } from "json-joy/es6/json-crdt-patch";
-import { encode, decode } from "json-joy/es6/json-crdt-patch/codec/verbose";
   
-// Create a new JSON CRDT document.
+// Initiate a model with a custom ID
-const model = Model.withLogicalClock();
-const modelMarkers = 
 
-// Find "obj" object node at path [].
+const rootModel = Model.withLogicalClock(11111111);
-model.api.root({
+
+// populate it with default data
+rootModel.api.root({
 	markers: {},
 });
 
-model.api.obj(["markers", uuid]).set(s.con(target._latlng));
+// Fork it on each client
-model.api.obj("markers").events.onViewChanges.listen((changes) => console.log(changes))
+let userModel = rootModel.fork();
-
 ```
 
-When receiving an update, you could apply it, like this:
+Making changes to the model. Here we are changing a constant for another one, by using `s.con`.
+
+```js
+userModel.api.obj(["markers"]).set({
+	[uuid]: s.con(target._latlng),
+});
+```
+
+Creating a patch, before sending it to the other parties:
+
+```js
+import { encode, decode } from "json-joy/es6/json-crdt-patch/codec/verbose";
+
+let patch = userModel.api.flush();
+let payload = encode(patch);
+```
+
+When receiving a message, decode it and apply it:
 
 ```js
 let patch = decode(payload);
 model.api.apply(patch);
+```
 
-// And see the model with 
+We can observe the changes this way. Here, we're having a look at the operation that happened and acting on it. The names of the operations aren't clearly specified by the spec. It seems a bit sketchy, so I'm not sure it's the way to handle this, but it works.
-model.api.view();
+
+```js
+userModel.api.onPatch.listen((patch) => {
+	patch.ops.forEach((op) => {
+		if (op.name() === "ins_obj") {
+			let key = op.data[0][0];
+			let value = userModel.view().markers[key];
+			upsertMarker({ latlng: value, uuid: key, local: true });
+		}
+	});
+});
 ```
 
 Metrics:
@@ -299,29 +370,47 @@ Metrics:
 - Size: 143 ko
 - Data transmitted for 2 peers and 40 edits: (35 bytes per edit)
 
-| Pros                                                                                                                                                       | Cons                                                                              |
+| Pros                                                                                                                                                       | Cons                                                                                                         |
-| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
+| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
-| It's low level, so you know what you're doing                                                                                                              | It doesn't provide a high level interface for sync<br><br>                        |
+| Low level, so you know what you're doing                                                                                                                   | It doesn't provide a high level interface for sync<br><br>                                                   |
-| It's separated as different small atomic libraries, which makes it easy to switch bits of the code if needed, without throwing everything away.            | Seems to be a one-person project until now                                        |
+| Small atomic libraries, making it easy to use only the parts we need.                                                                                      | It's currently a one-person project, without clear community channels to gather with other interested folks. |
-| The interface [proposes to store different type of data](https://jsonjoy.com/libs/json-joy-js/json-crdt/guide/node-types) (constants, values, arrays, etc) | Quite recent, so probably rough spots are to be found                             |
+| The interface [proposes to store different type of data](https://jsonjoy.com/libs/json-joy-js/json-crdt/guide/node-types) (constants, values, arrays, etc) | Quite recent, so probably rough spots are to be found                                                        |
-| Distributed as different type of JS bundles (modules, wasm, etc)                                                                                           | I didn't find a lot of support, (probably because the project is still quite new) |
+| Distributed as different type of JS bundles (modules, wasm, etc)                                                                                           |                                   
-## Summarizing CRDT Libraries for uMap
 
-Let's summarize the key considerations and how each CRDT library aligns with our objectives for uMap. The goal is to assess their fit in terms of specific collaborative features, efficiency, and ease of integration.
+---
+## Part 3: Key takeaways
 
-| Feature / Library         | Y.js                                                   | Automerge                                                  | JSON Joy                                                        |
+Y.js:
-| ------------------------- | ------------------------------------------------------ | ---------------------------------------------------------- | --------------------------------------------------------------- |
-| Intuitive API             | Provides a natural feel with native JS objects.        | API is transactional, detailed for change tracking.        | Low-level API offers granular control.                          |
-| Synchronization Protocol  | Multiple options with providers, including WebSockets. | Multiple options with providers, including WebSockets.     | Requires custom implementation for sync.                        |
-| Conflict Resolution       | Automatic merging with Y.js's internal mechanisms.     | Detailed conflict detection and resolution API.            | Allows control over data types and operations.                  |
-| Offline Changes Handling  | Inbuilt support for offline edits and synchronization. | Requires a more manual approach to handle offline changes. | Focused on model updates without specifics on network handling. |
-| Versioning and History    | Supports selective versioning through snapshots.       | Designed with robust version history tracking.             | Model is versioned but favors compact storage.                  |
-| Community and Support     | Active community with regular updates.                 | Strong support with a focus on collaboration.              | Smaller community; promising but less established.              |
-| Library Size / Efficiency | Small size with efficient operation.                   | Larger library with dependency on WebAssembly.             | Modular design with compact size.                               |
-| Browser Compatibility     | Broad compatibility, some bundler dependencies.        | Supports modern browsers with potential polyfills.         | Flexible bundles for diverse browser support.                   |
-| Suitability for uMap      | Ready-to-use with good documentation and examples.     | Strong features, may require significant integration.      | Promising, would need robustness as it matures.                 |
 
-## Notes on YATA and RGA
+- **JavaScript objects API**: uses native JS objects, making it easier to integrate with existing applications (but can also transport metadata we don't really want)
+- **WebSockets**: a "connection provider" exists for WebSockets, handling offline changes, reconnections, etc. The currently implemented provider considers the server as a central node, which also maintains a CRDT locally.
+- **Awareness API**: a protocol defined to send awareness information (who is connected, and for instance the cursor position)
+- Versioning and history: Supports selective versioning through snapshots.
+- Community and support: Active community with regular updates.
+- Library Size / Network usage: Small size with efficient default compression of the data.
+
+Suitability for uMap: Ready-to-use with good documentation and examples, might do more than what we need, some bundler dependencies.
+
+Automerge:
+
+- **Transactional API**: API is transactional (edits should happen in a function), making it obvious, but can also be harder to implement.
+- **WebSockets**: Multiple options with providers and storage, including WebSockets. It works similarly to the y.js one, with a copy of the data living on the server. There is currently no detection of offline / retry on disconnect, or awareness API.
+- **Conflict Resolution**: [A conflict detection API](https://automerge.org/docs/documents/conflicts/) exists, which make it easy to get the conflicting values, when it happens.
+- **Offline Changes Handling**: Requires a more manual approach to handle offline changes.
+- Versioning and History: Designed with robust version history tracking.
+- **Community and Support**: Strong support with a focus on collaboration.
+- **Library Size / Efficiency**: Larger library with dependency on WebAssembly.
+
+JSON Joy:
+
+- **Lower-level API**: A low-level API offers granular control, it's more hands-on, but less magic.
+- **WebSockets**: Nothing is provided in this area, and it would need to be implemented by ourselves.
+- **Community and Support**: there is only one maintainer as of now, and the community doesn't exist yet.
+- **Browser Compatibility**: Flexible bundles for diverse browser support, which might be helpful in our case.
+
+## Extra notes
+
+###  YATA and RGA
 
 While researching, I found that the two popular CRDTs implementation out there use different approaches for the virtual counter:
 
@@ -337,7 +426,7 @@ While researching, I found that the two popular CRDTs implementation out there u
 
 ### Resources
 
-- [Bartosz Sypytkowski](https://www.bartoszsypytkowski.com/the-state-of-a-state-based-crdts/) introduction on CRDTs, with practical exemples is very intuitive.
+- [CRDTs: The Hard Parts](https://www.youtube.com/watch?v=x7drE24geUw), a video by Martin Kleppmann where he explains the current state of the art of CRDTs, and why some problems aren't solved yet.
-- [CRDT Implementations](https://jzhao.xyz/thoughts/CRDT-Implementations#replicated-growable-array-rga)
-- [CRDTs: The Hard Parts - YouTube](https://www.youtube.com/watch?v=x7drE24geUw)
 - [An Interactive Intro to CRDTs](https://jakelazaroff.com/words/an-interactive-intro-to-crdts/) gets you trough different steps to understand what are CRDTs, and how to implement a LWW Register.
+- [Bartosz Sypytkowski](https://www.bartoszsypytkowski.com/the-state-of-a-state-based-crdts/) introduction on CRDTs, with practical exemples is very intuitive.
+- [CRDT Implementations](https://jzhao.xyz/thoughts/CRDT-Implementations) contains nice information and vocabulary useful when working with CRDTs.

content/pages/projets.md

@@ -1,67 +1,8 @@
 ---
-save_as: projets.html
+save_as: projets/index.html
 slug: projets
 ---
 
-# Projets
-
-J'ai pu <del>jouer avec</del> travailler sur plusieurs projets au fil des années, et bien souvent j'ai passé la main sur certains projets que j'ai initié, pour pouvoir faire autre chose.
-
-Voici quelques logiciels que j'ai initié, ou bien auxquels j'ai participé. Vous pouvez aussi aller faire un tour sur mes dépots publics ([Github](https://github.com/almet) ou [Gitlab](https://gitlab.com/almet))
-
-[Générateur de motifs jacquard (tricot)](https://bekeko.notmyidea.org/) (2022)
-: Un petit logiciel pour faciliter la vie d'une amie qui fabrique des chaussons à la main. L'idée est de pouvoir choisir des motifs et des couleurs, et de se rendre compte du résultat. ([codé en Elm](https://github.com/almet/bekeko))
-
-[Copanier](https://github.com/spiral-project/copanier) (2019-2024)
-: Un logiciel web créé en 2019, qui permet d'organiser des groupements
-d'achats. J'ai repris et adapté le logiciel pour
-l'usage d'un groupement auquel je participe. ([codé en python](https://github.com/almet/copanier))
-
-[I Hate Money](http://ihatemoney.org) (2011-2023)
-: Un site web qui permet de gérer les dépenses de groupes, [créé fin
-2011](https://blog.notmyidea.org/how-are-you-handling-your-shared-expenses.html).
-Il est possible de rentrer qui à payé quoi, et pour qui, et une balance est
-gérée pour vous. Je maintiens une instance ouverte sur [ihatemoney.org](https://ihatemoney.org). ([codé en python+flask](https://github.com/spiral-project/ihatemoney))
-
-[Kinto](https://github.com/kinto/kinto) (2012-2015)
-: Un backend générique pour des applications Web. J'ai initié ce projet avec
-des collègues [en
-2012](https://blog.notmyidea.org/thoughts-about-a-form-generation-service-gis-enabled.html)
-alors que je travaillais pour Mozilla. Le projet est actuellement utilisé pour
-gérer la synchronisation de certaines données dans Firefox. ([codé en python+pyramid](https://github.com/Kinto/kinto))
-
-[Pelican](http://getpelican.com) (2010-2020)
-: Un générateur de site statique [créé en
-2010](https://blog.notmyidea.org/pelican-a-simple-static-blog-generator-in-python.html).
-Ce site fonctionne grace à ce logiciel, et il est utilisé par des projets comme
-le [Noyau Linux](https://www.kernel.org/pelican.html) et
-[Debian](https://bits.debian.org/pages/about.html). L'idée est de pouvoir transformer des fichiers [au format Markdown](https://fr.wikipedia.org/wiki/Markdown) en un site web qui ne sera pas regénéré à chaque requête, et qui est donc très facile à héberger. ([codé en python](https://github.com/getpelican/pelican))
-
----
-
-## Expériences passées
-
-En sortant de mes études, j'ai d'abord travaillé en tant que developpeur avant de monter une brasserie artisanale.
-
-[Mozilla](https://mozilla.org)
-: J'ai travaillé de 2011 à 2016 pour Mozilla au sein de l'équipe « Services ».
-J'ai pu travailler sur des projets autour des données utilisateur, du
-chiffrement et de la synchronisation, ainsi que pour le passage à l'échelle de
-certaines sites comme addons.mozilla.com.
-
-[Brasserie du Vieux Singe](https://www.vieuxsinge.com/)
-: J'ai co-fondé en 2017 une brasserie artisanale biologique qui défend des
-valeurs de coopération, de partage et de gastronomie. J'en suis parti à l'été
-2023 pour retourner vers le developpement. La Brasserie existe toujours.
-
-[Le Grappe](https://www.reseaugrappe.org/)
-: Durant mes années étudiantes (2007-2012), j'ai participé à la création et à l'animation
-d'un réseau d'associations « porteuses de projets en environnement ». Un bon
-moyen de rencontrer d'autres personnes animées par des valeurs collectivistes,
-et de chercher à s'organiser contre l'artificialisation du monde.
-
----
-
 ## 🌟 Valeurs et intérets
 
 Logiciel Libre
@@ -83,6 +24,72 @@ Bonne humeur
 
 ---
 
+## Projets
+
+J'ai pu <del>jouer avec</del> travailler sur plusieurs projets au fil des années. Bien souvent j'ai passé la main sur ceux que j'ai initié.
+
+Voici quelques logiciels auxquels j'ai participé. Vous pouvez aussi aller faire un tour sur mes dépots publics ([Github](https://github.com/almet) ou [Gitlab](https://gitlab.com/almet))
+
+🗺️ [uMap](https://umap-project.org) (2023-2024)
+: Un logiciel de création de cartes pour le web. J'y travaille sur l'ajout de fonctionalités de collaboration, pour que plusieurs utilisateur·ices puissent voir les éditions des autres, en « temps réel ».
+
+🚨 [Argos](https://framasoft.frama.io/framaspace/argos/) (2023-2024)
+: Un logiciel de supervision de sites web, et un *status board*, realisé pour le compte de l'[association Framasoft](https://framasoft.org). 
+
+🧶 [Générateur de motifs jacquard (tricot)](https://bekeko.notmyidea.org/) (2022)
+: Un petit logiciel pour faciliter la vie d'une amie qui fabrique des chaussons à la main. L'idée est de pouvoir choisir des motifs et des couleurs, et de se rendre compte du résultat. ([codé en Elm](https://github.com/almet/bekeko))
+
+🙌 [Copanier](https://github.com/spiral-project/copanier) (2019-2024)
+: Un logiciel web créé en 2019, qui permet d'organiser des groupements
+d'achats. J'ai repris et adapté le logiciel pour
+l'usage d'un groupement auquel je participe. ([codé en python](https://github.com/almet/copanier))
+
+💸 [I Hate Money](http://ihatemoney.org) (2011-2023)
+: Un site web qui permet de gérer les dépenses de groupes, [que j'ai créé fin
+2011](https://blog.notmyidea.org/how-are-you-handling-your-shared-expenses.html).
+Il est possible de rentrer qui à payé quoi, et pour qui, et une balance est
+gérée pour vous. Je maintiens une instance ouverte sur [ihatemoney.org](https://ihatemoney.org). ([codé en python+flask](https://github.com/spiral-project/ihatemoney))
+
+🔄 [Kinto](https://github.com/kinto/kinto) (2012-2015)
+: Un « backend » générique pour des applications Web. J'ai initié ce projet avec
+des collègues [en
+2012](https://blog.notmyidea.org/thoughts-about-a-form-generation-service-gis-enabled.html)
+alors que je travaillais pour Mozilla. Le projet est actuellement utilisé pour
+gérer la synchronisation de certaines données dans Firefox. ([codé en python+pyramid](https://github.com/Kinto/kinto))
+
+✍️ [Pelican](http://getpelican.com) (2010-2020)
+: Un générateur de site statique [que j'ai créé en
+2010](https://blog.notmyidea.org/pelican-a-simple-static-blog-generator-in-python.html).
+Ce site fonctionne grace à ce logiciel, et il est utilisé par des projets comme
+le [Noyau Linux](https://www.kernel.org/pelican.html) et
+[Debian](https://bits.debian.org/pages/about.html). L'idée est de pouvoir transformer des fichiers [au format Markdown](https://fr.wikipedia.org/wiki/Markdown) en un site web qui ne sera pas regénéré à chaque requête, et qui est donc très facile à héberger. ([codé en python](https://github.com/getpelican/pelican))
+
+---
+
+## Expériences
+
+[Développeur indépendant](https://blog.notmyidea.org) (depuis 2023)
+: Je travaille en tant que développeur au sein de ma propre structure.
+
+[Brasserie du Vieux Singe](https://www.vieuxsinge.com/) (2017 - 2023)
+: J'ai co-fondé une brasserie artisanale biologique qui défend des
+valeurs de coopération, de partage et de gastronomie. J'en suis parti à l'été
+2023 pour retourner vers le developpement. La Brasserie existe toujours.
+
+[Mozilla](https://mozilla.org) (2011 - 2016)
+: J'ai travaillé pour Mozilla au sein de l'équipe « Services ».
+J'ai pu travailler sur des projets autour des données utilisateur, du
+chiffrement et de la synchronisation, ainsi que pour le passage à l'échelle de
+certaines sites comme addons.mozilla.com.
+
+[Le Grappe](https://www.reseaugrappe.org/) (2007 - 2012)
+: Durant mes années étudiantes, j'ai participé à la création et à l'animation
+d'un réseau d'associations « porteuses de projets en environnement ». Un bon
+moyen de rencontrer d'autres personnes animées par des valeurs collectivistes,
+et de chercher à s'organiser contre l'artificialisation du monde.
+
+---
+
 ## 🧑‍🔧Technologie
 
 Python

content/pages/worklog/umap.md

@@ -4,6 +4,15 @@ save_as: umap/index.html
 template: worklog
 total_days: 25
 ---
+
+## Vendredi 15 Mars 2024 (4h, 5/5)
+
+J'ai terminé ([PR](https://github.com/umap-project/umap/pull/1692)) la séparation du rendering avec la mise à jour des données, en suivant la piste commencée mardi, puis j'ai passé un peu de temps à faire marcher json joy, suite aux retours du mainteneur. L'API générale me semble bien pour des gens qui utilisent des composants qui savent se re-rendre, mais quand on a besoin de savoir ce qui a été modifié dans le patch, il faut regarder la dedans à la main ce qui rends toute l'opération un peu plus précaire.
+
+Bon, j'ai réussi à faire marcher le tout c'est l'important ! J'en ai profité pour bouger le code de [leaflet-sync](https://gitlab.com/umap-project/leaflet-sync) dans l'organisation umap-project dans gitlab.
+
+Normalement, tout est en place pour qu'on puisse commencer à ajouter du websocket dans le mix, surement dans le courant de la semaine prochaine !
+
 ## Mardi 12 Mars 2024 (7h, 5/5)
 
 Une bonne journée, passée majoritairement en pair prog avec Yohan. On a d'abord fait un point de synchro sur l'avancement général de la synchro, durant lequel on a plus ou moins décidé d'aller dans un premier temps vers la version simple de la synchronisation, en faisant en sorte que les clients écrasent les données des autres clients, avec le serveur qui s'occupe de faire le passe-plat.

mnmlist/static/css/main.css

@@ -1,5 +1,5 @@
 /*
-    Theme Name: mnmlist modified by almet
+    Theme Name: mnmlist modified by Alexis Métaireau
     Theme URI: http://mathieu.agopian.info/mnmlist/theme.html
     Description: Theme adapted from http://mnmlist.com/theme/ from Leo Babauta, a very clean minimalist theme, without comments, search, archives or other features.
     Author: Mathieu Agopian, Mathieu Leplatre, Alexis Métaireau
@@ -77,7 +77,6 @@ h1 {
   font-size: 2em;
   font-weight: normal;
   margin-bottom: 0;
-  font-weight: bold;
   line-height: 1.2em;
 }
 
@@ -224,10 +223,6 @@ a.tag:hover {
   color: var(--highlight-hover);
 }
 
-ul {
-  list-style-type: none;
-}
-
 ul li,
 ol li {
   line-height: 30px;
@@ -468,10 +463,13 @@ a[data-size='5'] {
   font-size: 2.2em;
 }
 
+.footer #feed {
+  top: 5px;
+}
+
 #feed {
   text-decoration: none;
   position: relative;
-  top: 5px;
 }
 #feed img {
   height: 25px;
@@ -532,22 +530,22 @@ dd {
   text-align: center;
 }
 
-article {
-  text-align: justify;
-}
-
 nav {
   padding-top: 2em;
 }
 
 .navigation {
   list-style-type: none;
-  display: inline-block;
   padding-left: 0px;
+  width: 100%;
+  display: flex;
+  flex-wrap: wrap;
+  align-items: center;
+  justify-content: center;
 }
 
 .navigation > li {
-  display: inline;
+  display: inline-flex;
   padding: 0px 0.5em;
 }
 

mnmlist/templates/article.html

@@ -3,56 +3,45 @@
 {% block content %}
     <header>
         {% if article.category == "Lectures" %}
-            <div class="book-container">
+            <h1 class="post-title">
-                <div class="book">
+                {{ article.title }}<small>
-                    <h1 class="post-title">
+                <br />
-                        <a href="{{ SITEURL }}/{{ article.url }}">{{ article.title }}</a><small>
+            de {{ article.author }}</small>
-                            <br />
+        </h1>
-                        par {{ article.author }}</small>
+        <time datetime="{{ article.date.isoformat() }}">Lu en {{ article.date | strftime("%B %Y") }}</time>
-                    </h1>
+    {% else %}
-                    {% if article.headline %}
+        <h1 class="post-title">{{ article.title }}</h1>
-                        <p>
+        <time datetime="{{ article.date.isoformat() }}">{{ article.locale_date }}</time>
-                            <em>{{ article.headline }}</em>
+    {% endif %}
-                        </p>
+    <nav>
-                    {% endif %}
+        <ul class="navigation">
-                    <time datetime="{{ article.date.isoformat() }}">Lu en {{ article.date | strftime("%B %Y") }}</time>
+            {% if article.prev_article %}
-                </div>
+                <li>
-                {% if article.category == "Lectures" and article.isbn_cover %}
+                    <a href="{{ SITEURL }}/{{ article.prev_article.url }}"
-                    <div class="book-cover">
+                       title="{{ article.prev_article.title }}">← Précédent</a>
-                        <img src="{{ SITEURL }}/{{ article.isbn_cover }}" />
+                </li>
-                    </div>
-                {% endif %}
-            </div>
-        {% else %}
-            <h1 class="post-title">{{ article.title }}</h1>
-            <time datetime="{{ article.date.isoformat() }}">{{ article.locale_date }}</time>
-            <nav>
-                <ul class="navigation">
-                    {% if article.prev_article %}
-                        <li>
-                            <a href="{{ SITEURL }}/{{ article.prev_article.url }}"
-                               title="{{ article.prev_article.title }}">← Précédent</a>
-                        </li>
-                    {% endif %}
-                    <li>
-                        <a href="{{ SITEURL }}">Accueil</a>
-                    </li>
-                    {% if article.next_article %}
-                        <li>
-                            <a href="{{ SITEURL }}/{{ article.next_article.url }}"
-                               title="{{ article.next_article.title }}">Suivant →</a>
-                        </li>
-                    </ul>
-                </nav>
             {% endif %}
-        </p>
+            <li>
-    </nav>
+                <a href="{{ SITEURL }}">Accueil</a>
+            </li>
+            {% if article.next_article %}
+                <li>
+                    <a href="{{ SITEURL }}/{{ article.next_article.url }}"
+                       title="{{ article.next_article.title }}">Suivant →</a>
+                </li>
+            </ul>
+        </nav>
+    {% endif %}
     {% if article.headline and article.category != "weeknotes" %}
         <p>
             <em>{{ article.headline }}</em>
         </p>
     {% endif %}
-{% endif %}
+    {% if article.isbn_cover %}
+        <div class="book-cover">
+            <img src="{{ SITEURL }}/{{ article.isbn_cover }}" />
+        </div>
+    {% endif %}
 </header>
 <article>
     {{ article.content }}
@@ -65,5 +54,5 @@
             - Posté dans la catégorie <a href="{{ SITEURL }}/{{ article.category.url }}">{{ article.category }}</a>
         </p>
     {% endif %}
-</article>
+    </article>
 {% endblock %}

mnmlist/templates/base.html

@@ -7,7 +7,7 @@
         <meta charset="utf-8" />
         <meta name="viewport" content="width=device-width, initial-scale=1">
         <link rel="stylesheet"
-              href="{{ SITEURL }}/theme/css/{{ CSS_FILE }}?v3"
+              href="{{ SITEURL }}/theme/css/{{ CSS_FILE }}?v4"
               type="text/css" />
         <link href="{{ FEED_DOMAIN }}/{{ FEED_ALL_ATOM }}"
               type="application/atom+xml"

mnmlist/templates/index.html

@@ -29,6 +29,5 @@
                 {% endfor %}
             </ul>
             Voir <a href="archives.html">toutes les archives</a>
-            <hr />
         {% endif %}
     {% endblock content %}

pelicanconf.py

@@ -71,7 +71,7 @@ CATEGORIES_DESCRIPTION = {
         "Quelques réfléxions, bien souvent autour du monde du travail ou de la technologie.",
     ),
     "notes": (
-        "Carnet de notes",
+        "Notes",
         "Prises bien souvent en regardant une vidéo ou un article en ligne. Je les mets ici pour pouvoir les retrouver quand le besoin se fait sentir.",
     ),
     "ecriture": (

plugins/simplereader.py

@@ -111,6 +111,7 @@ class SimpleReader(MarkdownReader):
     def __init__(self, *args, **kwargs):
         super(SimpleReader, self).__init__(*args, **kwargs)
         self.settings["MARKDOWN"]["extensions"].append("markdown.extensions.toc")
+        self.settings["MARKDOWN"]["extension_configs"].update({'markdown.extensions.toc': {'toc_depth': 3}})
 
     def read(self, source_path):
         self._source_path = source_path