Inform users that for specific distros and versions, we install some
extra packages (PySide6, conmon), in order to fix some incompatibilities
between Dangerzone and the base system. Provide also a link to the
source / build instructions for the package, as well as any relevant
issues.
Fixes#767
Add a section for our end-users in INSTALL.md, that explains how to
verify that our Dangerzone assets have been signed by our advertised
signing key.
This section explains what are the .asc files that users see next to our
release assets, and how they can verify each asset individually using
GPG. It is heavily inspired by a similar section for OnionShare.
Closes#761
Add a new script called `sign-assets.py`, which produces the hash of all
the Dangerzone assets for a release (Windows/macOS installers, container
image), and signs them individually.
Also update our RELEASE.md document, to incorporate this script into our
release workflow.
Gracefully terminate certain conversion processes that may get stuck
when writing lots of data to stdout. Also, handle a race condition when
a conversion process terminates slightly after the associated container.
Fixes#791
We have recently [1] changed the name of the Dangerzone application to
capital-case "Dangerzone", but this breaks our PDF viewer detection
logic. Adjust our check to exclude Dangerzone from the list.
Fixes#790
[1]: See commit 3d426ed36b
In d632908a44 we improved our
`replace_control_chars()` function, by replacing every control or
invalid Unicode character with a placeholder one. This change, however,
made our debug logs harder to read, since newlines were not preserved.
There are indeed various cases in which replacing newlines is wise
(e.g., in filenames), so we should keep this behavior by default.
However, specifically for reading debug logs, we add an option to keep
newlines to improve readability, at no expense to security.
The `exit()` [1] function is not necessarily present in every Python
environment, as it's added by the `site` module. Also, this function is
"[...] useful for the interactive interpreter shell and should not be
used in programs"
For this reason, we replace all such occurrences with `sys.exit()` [2],
which is the canonical function to exit Python programs.
[1]: https://docs.python.org/3/library/constants.html#exit
[2]: https://docs.python.org/3/library/sys.html#sys.exit
On Windows, if we don't use the `startupinfo=` argument of
subprocess.Popen, then a terminal window will flash while running the
command.
Use `startupinfo=` when killing a container, as we do for every other
command.
On Windows, if we somehow attempt to archive the same document twice
(e.g, because it got archived once, and then we copy it back), we will
get an error, because Windows does not overwrite the target path, if it
already exists.
Fix this issue by always removing the previously archived version, when
performing the next archival action, and update our tests.
Quote from `pytest-mock` docs [1]:
> The purpose of this plugin is to make the use of context managers and
> function decorators for mocking unnecessary, so it will emit a warning
> when used as such.
Thus using it as a context manager currently produces a warning during
test runs in CI which is extra noise that could make new (possibly more
important) warnings harder to spot.
[1]: https://pytest-mock.readthedocs.io/en/latest/usage.html#usage-as-context-manager
On Linux, the `%u` field code results in multiple Dangerzone instances
being launched when opening multiple documents with Dangerzone from
e.g. Nautilus, as `%u` signifies that the application (in this case -
Dangerzone) can only open a single file/URL at once.
This changes the field code to `%F` as Dangerzone (now) supports
converting multiple files at once. We use `%F` (multiple local files)
instead of `%U` (multiple files and/or URLs) since Dangerzone does not
support opening URLs.
See also the Desktop Entry Specification [1] for more information on the
field codes.
Fixes#797
[1]: https://specifications.freedesktop.org/desktop-entry-spec/latest/ar01s07.html
Currently, the app ID of the Dangerzone GUI application when running
under Wayland is `python3`, which is not very useful if one wants to
automate some action related to the Dangerzone application window (e.g.
to always start Dangerzone window in floating mode under Sway WM).
Setting the desktop filename property also sets the app ID of the
application under Wayland. According to Qt documentation[1], the
property value should be the name of the application's .desktop file but
without the extension.
Qt documentation also states:
> This property gives a precise indication of what desktop entry
> represents the application and it is needed by the windowing system to
> retrieve such information without resorting to imprecise heuristics.
Therefore I also think that setting this property is needed to display
the correct application name and icon (taken from the .desktop entry)
when running under certain windowing systems (like Wayland)
(see also #402).
Note that this property is not enough, as we've encountered systems
where setting just the desktop file name does not alter the detected
application name by the window manager. For this reason, we also use
set the application name [2] to `dangerzone`, to remove any ambiguity.
[1]: https://doc.qt.io/qt-6/qguiapplication.html#desktopFileName-prop
[2]: https://doc.qt.io/qt-6/qcoreapplication.html#applicationName-propFixes#402
Bump our poetry.lock file, to get the latest versions of our
dependencies. Note that we are aware that this bump does not bring in
the latest PySide6 version.
Refs #773
Update the description in the pre-release task for PySide6, since a lot
has changed after writing this section. Now that `python3-pyside6` is in
the Fedora Rawhide repo, and will soon get backported in the stable
repos, we no longer check for newer upstream versions, but if Fedora
finally did the backport.
Refs freedomofpress/maint-dangerzone-pyside6#5
On Unix systems a filename can be a sequence of bytes that is not valid
UTF-8. Python uses[1] surrogate escapes to allow to decode such
filenames to Unicode (bytes that cannot be decoded are replaced by a
surrogate; upon encoding the surrogate is converted to the original
byte).
From `click` docs[2]:
> Invalid bytes or surrogate escapes will raise an error when written
> to a stream with `errors="strict"`. This will typically happen with
> `stdout` when the locale is something like `en_GB.UTF-8`.
To fix that, we use `utils.replace_control_chars()` before printing the
filenames to `stdout` so that surrogate escapes are replaced by �.
Fixes#768
The `util.replace_control_chars()` function was overly strict, and
would replace every non-ASCII character with "_". This included both
control characters, as well as normal characters in a non-English
alphabet.
Relax these restrictions by checking each character and deciding if it's
a Unicode control character, using the `unicodedata` Python package.
With this change, emojis and non-English letters are now allowed.
Add a pytest fixture that crafts a filename with Unicode characters that
are not considered common for this use. By default, this fixture uses
an invalid Unicode character as well, but we strip it in case of macOS
(APFS) since filenames must be UTF-8 encoded.
[1]: https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations
Include the test dependencies when linting, especially `pytest`. We need
this because `mypy` cannot understand the `pytest.raises` exception, and
specifically the fact that it catches exceptions. It assumes that the
next code block is unreachable, since it doesn't see any try ... except.
Add termination tests for the Dummy provider, so that we can have
cross-platform coverage in our Windows/macOS CI runners, which can't use
the Container isolation providers.
Add termination-related tests for Qubes. To achieve this, we need
to make a change to the Qubes isolation provider. More specifically,
we need to make the isolation provider yield control to the caller only
when the disposable qube is up and running.
Qubes does not provide us a solid guarantee to do so, but we've found a
hacky workaround, whereby we wait until the `qrexec-client-vm` process
opens a `/dev/xen` character device. This should happen, in theory, once
the disposable qube is ready, and has sent a `MSG_SERVICE_CONNECT` RPC
message to the caller.
Add termination-related tests for containers. To achieve this, we need
to make a change to the container isolation provider. More specifically,
we need to make the isolation provider yield control to the caller only
when the container is up and running. Failure to do so may lead to
lingering processes.
Add some test cases in the isolation provider tests, that check how it
behaves when a process completes successfully, lingers, or cannot
terminate.
These tests cannot run yet, since they must be imported by a concrete
isolation provider test class. In subsequent commits, we will start
enabling them.
Previously, we always assumed that the spawned process would quit
within 3 seconds. This was an arbitrary call, and did not work in
practice.
We can improve our standing here by doing the following:
1. Make `Popen.wait()` calls take a generous amount of time (since they
are usually on the sad path), and handle any timeout errors that they
throw. This way, a slow conversion process cleanup does not take too
much of our users time, nor is it reported as an error.
2. Always make sure that once the conversion of doc to pixels is over,
the corresponding process will finish within a reasonable amount of
time as well.
Fixes#749
Get the exit code of the spawned process for the doc-to-pixels phase,
without timing out. More specifically, if the spawned process has not
finished within a generous amount of time (hardcode to 15 seconds),
return UnexpectedConversionError, with a custom message.
This way, the happy path is not affected, and we still make our best to
learn the underlying cause of the I/O error.
Extend the IsolationProvider class with a
`terminate_doc_to_pixels_proc()` method, which must be implemented by
the Qubes/Container providers and gracefully terminate a process started
for the doc to pixels phase.
Refs #563
Set a unique name for spawned containers, based on the ID of the
provided document. This ID is not globally unique, as it has few bits of
entropy. However, since we only want to avoid collisions within a
single Dangerzone invocation, and since we can't support multiple
containers running in parallel, this ID will suffice.
Pass the Document instance that will be converted to the
`IsolationProvider.start_doc_to_pixels_proc()` method. Concrete classes
can then associate this name with the started process, so that they can
later on kill it.
Extend our CircleCI jobs to run CI tests in Ubuntu Noble.
This commit also adds support for building the Dangerzone .deb package
in Ubuntu Noble, but does not actually enable it. The reason is that
stdeb, which produces our Debian packages, does not work with Python
3.12, which ships with Ubuntu Noble
Refs #773
Our end-user Fedora environments, that we create for testing how
Dangerzone would operate on a clean Fedora system, require PySide6 to be
installed. This package is not available from the official Fedora repos
yet.
We have a way instead to check the poetry.lock file, grab the latest
PySide6 version from there, and install it from a URL. This is no longer
necessary, now that PySide6 6.7.0 will soon be available in all stable
Fedora releases. Since the last release maintained by FPF will be
6.6.3.1, we should pin this version in our env.py script. This way, we
can bump poetry.lock independently, and let Windows/macOS users get
different versions.
Refs freedomofpress/maint-dangerzone-pyside6#5
The `checkout`, `setup-python`, `upload-artifact` and `download-artifact`
actions produce warnings about deprecated Node.js 16.
Update the actions to use Node.js 20.
