Commit graph

2777 commits

Author SHA1 Message Date
Yohan Boniface
50ef073d36
Merge pull request #1883 from umap-project/remove-u-keys
chore: remove U.Keys, and refactor global shorcuts
2024-06-07 18:35:49 +02:00
fc2de3fc53 tests(sync): Change the way the websocket server is run in the tests
Using [pytest-xprocess](https://pytest-xprocess.readthedocs.io/) proved
not being as useful as I thought at first, because it was causing
intermitent failures when starting the process.

The code now directly uses `subprocess.popen` calls to start the server.
The tests are grouped together using the following decorator:

`@pytest.mark.xdist_group(name="websockets")`

Tests now need to be run with the `pytest --dist loadgroup` so that all
tests of the same group happen on the same process.

More details on this blogpost:

  https://blog.notmyidea.org/start-a-process-when-using-pytest-xdist.html
2024-06-07 18:21:30 +02:00
98a7a56701 refactor(sync): rename ws.py to websocket_server.py
Hopefully this is clearer :-)
2024-06-07 18:21:30 +02:00
f42498d195 refactor(sync): Sync layers creation with map.createDataLayer utility.
Rather than having it done inside the datalayer itself. This gives us
more control.
2024-06-07 18:21:30 +02:00
149a0c1680 refactor: rename geometrytoFeatures to geoJSONToLeaflet
Because we are dealing with technologies using overlapping vocabulary,
it is easy to get lost. Hopefully this change makes it clear that it
converts geoJSON inputs in Leaflet / uMap objects.
2024-06-07 18:21:30 +02:00
137cc21af2 feat(sync): Add a proxy to the SyncEngine objects.
It is now possible to create proxy objects using `sync_engine.proxy(object)`.
The returned proxy object will automatically inject `metadata` and
`subject` parameters, after looking for them in the `getSyncMetadata`
method (these are only known to the synced objects).

As a result, the calls are now simplified:

```
this.sync.update("key", "value")
```
2024-06-07 18:21:30 +02:00
4e7ac23b53 refactor(sync): remove formbuilder this._redraw() callbacks.
They are now handled in the `render()` call, so there is no more need
for them here.
2024-06-07 18:21:30 +02:00
5247bf77d3 refactor(sync): Remove syncUpdateProperties function.
The function was only used once, so removing it  simplified the whole
flow.
2024-06-07 18:21:30 +02:00
d3730a99ff fix: use array.includes(string) the proper way.
Because this `syncUpdatedProperties` function is only called once, it
didn't trigger any issue in practice (as the check was always returning
true).
2024-06-07 18:21:30 +02:00
d91c86e7fa feat(websockets): run the WS server as a django management command.
This allows to handle the loading of the settings in a consistant way,
and aditionnaly to provide a way to override the `WEBSOCKET_BACK_HOST`
and `WEBSOCKET_BACK_PORT` settings with arg commands `--host` and
`--port`.

Without this change, because of how we are currently loading our
settings, we would require the settings the be exposed by the
`umap.settings.__init__` file.

Previous implementations were exposing these settings, with the
following code:

```python
settings_as_dict = {k: v for k, v in globals().items() if k.isupper()}
  ```
2024-06-07 18:21:30 +02:00
940ae77602 settings: Make the websocket settings clearer.
It is now using `WEBSOCKET_BACK_HOST`, `WEBSOCKET_BACK_PORT` and
`WEBSOCKET_FRONT_URI`.

We need to take in consideration that the "front" WebSocket address
(that clients will connect to) might be different than the "back" ip and
port which are bound in the host.

This happens for instance for reverse proxies, or when running inside
a container.

We considered using a `WEBSOCKET_TLS` setting, to try guessing the
"front" address based on `WEBSOCKET_HOST`, `WEBSOCKET_PORT` and
`WEBSOCKET_TLS`, but as the back and front address can differ, this
would need to introduce a `WEBSOCKET_URI` in any case, so we went with
just using it, and not adding an extra `WEBSOCKET_TLS`.
2024-06-07 18:21:30 +02:00
90780fbf6d fixup: changes after @ybon's review. 2024-06-07 18:21:30 +02:00
50b7188647 fix(sync): sync the reference-version across peers
This allows the merge algorithm to not be lost when receiving changes.
Without this change, the optimistic merge algorithm isn't able to make
the distinction between peers, and features end up duplicated.
2024-06-07 18:21:30 +02:00
516c3d9a0d tests(sync): Add a test ensuring cloned features aren't duplicated
This is currently a bug in the current implementation. Hopefully fixed
in later commits.
2024-06-07 18:21:30 +02:00
b5eee142e7 test(sync): Ensure feature properties are synced 2024-06-07 18:21:30 +02:00
b316bde47c docs(sync): Document WEBSOCKET_* settings 2024-06-07 18:21:30 +02:00
b1a4053b2e chore(sync): remove belongsTo for now
As it requires more discussion, it will happen in a separate
pull-request.
2024-06-07 18:21:30 +02:00
8fc116e7a8 chore(test): remove empty test 2024-06-07 18:21:30 +02:00
dfbdc1f975 chore(test): fix a typo
(but I would really like to see what web socker would look like)
2024-06-07 18:21:30 +02:00
db50c6cdd6 chore(sync): relax some validation logic on the websocket server
Messages are now checked for conformity with the procol we defined, but
stop at the `operation` boundary. Values aren't checked.
2024-06-07 18:21:30 +02:00
bc1dec245b chore(sync): Sync engine now retrieves auth token
It's now it's responsability to get the authentication token from
the http server and pass it to the websocket server, it will make it
possible to redo the roundtrip when getting disconnected.
2024-06-07 18:21:30 +02:00
82ede035d6 chore(sync): use sync=false everywhere to stop propagation
In some cases, you want to stop the propagation of events. The previous
code was using `fromSync=true` and `sync=false` interchangeably. This
makes it use `sync=false` everywhere.
2024-06-07 18:21:30 +02:00
637ab67b93 chore(utils): remove console.log calls 2024-06-07 18:21:30 +02:00
5a565dd2ee fix(schema): dashArray belongs to features as well 2024-06-07 18:21:30 +02:00
10a68a75e4 test(sync): Ensure datalayer properties are synced 2024-06-07 18:21:30 +02:00
d10a7b70f1 test(sync): Ensure map properties are synced
This tests that the name of the map, and that zoom-control visibility is
properly synced over websockets.
2024-06-07 18:21:30 +02:00
6f7dd61023 test(sync): ensure polygon drag-n-drop is synced 2024-06-07 18:21:30 +02:00
ad4d6ac672 test(sync): Ensure polygons and their edits can be synced 2024-06-07 18:21:30 +02:00
5feabd4c14 test(sync): Marker drag-n-drop and delete are synced 2024-06-07 18:21:30 +02:00
80f7efc810 tests(sync): Test that marker sync works properly
This commit handles the start and stop of the websocket server during
the tests, using the xprocess library
2024-06-07 18:21:30 +02:00
d0e2d900f7 chore(tests): Rename "collaboration" tests to "optimistic merge"
As "collaboration" can be mistaken between "websocket" and
"server-merge". This naming makes it explicit.
2024-06-07 18:21:30 +02:00
92676e03ae chore(sync): Add message-dispatcher unit tests 2024-06-07 18:21:30 +02:00
74a1670c9d feat(sync): Add JS unittests for updateObjectValue, and refactor.
The new implementation uses `reduce`, in the hopes of producing a more
readable version than old-style loops and reassignment of object values.
2024-06-07 18:21:30 +02:00
9be613e3ce feat(sync): add tests for the websocket token view 2024-06-07 18:21:30 +02:00
80eaa151de refactor(sync): Remove unnecessary complexity
Because we're relying on the `geoJSONToFeatures` method, we don't need
anymore updaters, the default ones (map, datalayer, feature) are enough.

It also makes the codebase compatible with our eslint configuration.
2024-06-07 18:21:30 +02:00
9e36476abe chore(sync) make the test pass 2024-06-07 18:21:30 +02:00
647f6c3dfe chore(docker) remove ws.py for now 2024-06-07 18:21:30 +02:00
d5bff22081 fix(sync) only send datalayer options 2024-06-07 18:21:30 +02:00
898a3e93b2 fix(sync): do not call getSyncMetadata if it doesn't exist
This can arise when the form isn't bound to a "syncable" object. We
might want to provide an utility to check an object is *syncable*, or a
specific form to not attempt to sync objects which don't need it.
2024-06-07 18:21:30 +02:00
8e5c434988 feat(sync): Allow the sync of datalayer creation 2024-06-07 18:21:30 +02:00
458d68a4bb fix(sync): allow features geometries to be synced
By defining the `geometry` property in the schema.
2024-06-07 18:21:30 +02:00
5b55c80529 chore(docker): run the websocket server as a separate command 2024-06-07 18:21:30 +02:00
1d9b086115 chore(sync): remove useless dependencies from ws.py 2024-06-07 18:21:30 +02:00
def915c561 feat(settings): allow reading socket-related settings from ENV variables 2024-06-07 18:21:30 +02:00
01408254c9 fix(sync): intialize the sync engine before entering edit mode.
Otherwise `this.sync` is not defined.
2024-06-07 18:21:30 +02:00
46207741ff feat(sync): handle syncing of map limitbounds. 2024-06-07 18:21:30 +02:00
9cb87392c0 feat(sync): Avoid syncing remote datalayers' features
They will be synced on their own, and we dont want them to be present
twice on the map.
2024-06-07 18:21:30 +02:00
b99c242648 fix(sync): Initialize tilelayerand remoteData earlier.
It makes it possible to set these values on a remote peer.
2024-06-07 18:21:30 +02:00
059103b66c fix(sync): Allow overlays to sync
`this.options.overlay` is now set during `map.initialize()`, which makes
it possible to be set by incoming websocket operations.
2024-06-07 18:21:30 +02:00
2fafad714e chore(sync): Ensure properties can be updated before doing it.
When receiving a message, this checks the given properties belong to
the "subject" before applying the message.
2024-06-07 18:21:30 +02:00