> f-strings are a convenient way to format strings, but they are not
> necessary if there are no placeholder expressions to format. In this
> case, a regular string should be used instead, as an f-string without
> placeholders can be confusing for readers, who may expect such a
> placeholder to be present.
>
> — [ruff docs](https://docs.astral.sh/ruff/rules/f-string-missing-placeholders/)
The minimum python version when installing from source is now python
3.9, as Pyside6 6.7.1 dropped support for python 3.8 (see #780 for more
information).
On Debian-derivatives distributions, the minimum Python version is now
set to 3.8. In practice, because Pyside6 is not packaged for Debian, we
use Pyside2 [0], which is why we can relax the python version requirement.
In practice, when installing from source on an environment where
python3.9 is not the default python, poetry will look for it and use it
if available
> For various reasons, this Python version might not be compatible with
> the python range supported by the project. In this case, Poetry will
> try to find one that is and use it.
>
> [Poetry docs](https://python-poetry.org/docs/managing-environments/)
On Ubuntu Focal (20.04) where Python 3.9 is not installed by default,
it is possible to install it using the `python3.9` package.
Additionally, In version 1.24.3, PyMuPDF changed its package name from `fitz`
to `pymupdf` [2], resulting in a breakage on how it is installed in our
container. This is now fixed.
[0] More information on how Pyside6 packaging affects dangerzone on #221
[1] See [the current status of Pyside6 packaging](https://repology.org/
project/python:pyside6/packages)
[2] PyMuPDF changelog: https://pymupdf.readthedocs.io/en/latest/changes.html#change-log
Bump the PySide6 version used in our user environments to 6.7.1, to
mirror the one we ship to our users, and also fix a segfault issue in
our CI tests.
Refs #801
When building the Dangerzone RPMs, we were seeing the following shebang
warnings:
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
mangling shebang in /usr/lib/python3.12/site-packages/dangerzone/conversion/doc_to_pixels.py from /usr/bin/env python3 to #!/usr/bin/python3
mangling shebang in /usr/lib/python3.12/site-packages/dangerzone/conversion/common.py from /usr/bin/env python3 to #!/usr/bin/python3
mangling shebang in /usr/lib/python3.12/site-packages/dangerzone/conversion/pixels_to_pdf.py from /usr/bin/env python3 to #!/usr/bin/python3
mangling shebang in /etc/qubes-rpc/dz.ConvertDev from /usr/bin/env python3 to #!/usr/bin/python3
mangling shebang in /etc/qubes-rpc/dz.Convert from /bin/sh to #!/usr/bin/sh
These warnings are benign in nature, but coupled with #727, they could
lead to incorrect file permissions.
Remove shebangs from the following files, since they are not executed
directly, but are imported instead:
dangerzone/conversion/common.py
dangerzone/conversion/doc_to_pixels.py
dangerzone/conversion/pixels_to_pdf.py
Also, accept the suggestions by Fedora (/bin/sh -> /usr/bin/sh,
/usr/bin/env python3 -> /usr/bin/python3) for the following files:
qubes/dz.Convert
qubes/dz.ConvertDev
Refs #727
Switch build directory for the `rpmbuild` command from
`./install/linux/rpm-build` to `~/rpmbuild`. The main reason for this is
that we want a build directory that will not be mounted in the
container, since we've experienced issues with file permissions.
Regarding the choice of directories, we went with `~/rpmbuild` because
it's outside the Dangerzone source, and also because it's the default
choice in Fedora [1].
[1]: 3ae1eeafee/rpmdev-setuptree (L60)Closes#727
When building the Dangerzone RPM package, detect if the files bundled in
it have any incorrect permissions. We have seen in the past that
building RPMs from the Dangerzone source, mounted to a macOS Docker
container, can lead to files readable only by the root user (600 /
rw-------).
Refs #727
Dangezone has received a security audit in December 2023, and published
on February 2024. It would be nice for people seeing this project to
learn about this audit.
Add some articles about the Dangerzone project that may be useful for
those evaluating this tool. This article list is not complete, and has
been sampled from various links we have encountered in the past.
Alpine Linux 3.20 was released recently [1]. As a result, the
`alpine:latest` image ref, that our Dockerfile uses, switched from the
3.19 to the 3.20 Alpine Linux release. This release has Python 3.12,
meaning that the following line in our Dockerfile now fails:
COPY --from=pymupdf-build /usr/lib/python3.11/site-packages/fitz/ /usr/lib/python3.11/site-packages/fitz
Bump the Python version in the Python system path to 3.12, so that we
can successfully build the container image.
[1]: https://alpinelinux.org/posts/Alpine-3.20.0-released.html
Zsh users that attempt to run the following command in our Ubuntu/Debian
installation instructions:
echo deb [signed-by=/etc/apt/keyrings/fpf-apt-tools-archive-keyring.gpg] \
https://packages.freedom.press/apt-tools-prod ${VERSION_CODENAME?} main \
| sudo tee /etc/apt/sources.list.d/fpf-apt-tools.list
encounter the following error:
zsh: no matches found:
[signed-by=/etc/apt/keyrings/fpf-apt-tools-archive-keyring.gpg]
Quote this command to ensure compatibility with other shells, and update
our CI checks.
Fixes#805
Inform users that for specific distros and versions, we install some
extra packages (PySide6, conmon), in order to fix some incompatibilities
between Dangerzone and the base system. Provide also a link to the
source / build instructions for the package, as well as any relevant
issues.
Fixes#767
Add a section for our end-users in INSTALL.md, that explains how to
verify that our Dangerzone assets have been signed by our advertised
signing key.
This section explains what are the .asc files that users see next to our
release assets, and how they can verify each asset individually using
GPG. It is heavily inspired by a similar section for OnionShare.
Closes#761
Add a new script called `sign-assets.py`, which produces the hash of all
the Dangerzone assets for a release (Windows/macOS installers, container
image), and signs them individually.
Also update our RELEASE.md document, to incorporate this script into our
release workflow.
Gracefully terminate certain conversion processes that may get stuck
when writing lots of data to stdout. Also, handle a race condition when
a conversion process terminates slightly after the associated container.
Fixes#791
We have recently [1] changed the name of the Dangerzone application to
capital-case "Dangerzone", but this breaks our PDF viewer detection
logic. Adjust our check to exclude Dangerzone from the list.
Fixes#790
[1]: See commit 3d426ed36b
In d632908a44 we improved our
`replace_control_chars()` function, by replacing every control or
invalid Unicode character with a placeholder one. This change, however,
made our debug logs harder to read, since newlines were not preserved.
There are indeed various cases in which replacing newlines is wise
(e.g., in filenames), so we should keep this behavior by default.
However, specifically for reading debug logs, we add an option to keep
newlines to improve readability, at no expense to security.
The `exit()` [1] function is not necessarily present in every Python
environment, as it's added by the `site` module. Also, this function is
"[...] useful for the interactive interpreter shell and should not be
used in programs"
For this reason, we replace all such occurrences with `sys.exit()` [2],
which is the canonical function to exit Python programs.
[1]: https://docs.python.org/3/library/constants.html#exit
[2]: https://docs.python.org/3/library/sys.html#sys.exit
On Windows, if we don't use the `startupinfo=` argument of
subprocess.Popen, then a terminal window will flash while running the
command.
Use `startupinfo=` when killing a container, as we do for every other
command.
On Windows, if we somehow attempt to archive the same document twice
(e.g, because it got archived once, and then we copy it back), we will
get an error, because Windows does not overwrite the target path, if it
already exists.
Fix this issue by always removing the previously archived version, when
performing the next archival action, and update our tests.
Quote from `pytest-mock` docs [1]:
> The purpose of this plugin is to make the use of context managers and
> function decorators for mocking unnecessary, so it will emit a warning
> when used as such.
Thus using it as a context manager currently produces a warning during
test runs in CI which is extra noise that could make new (possibly more
important) warnings harder to spot.
[1]: https://pytest-mock.readthedocs.io/en/latest/usage.html#usage-as-context-manager
On Linux, the `%u` field code results in multiple Dangerzone instances
being launched when opening multiple documents with Dangerzone from
e.g. Nautilus, as `%u` signifies that the application (in this case -
Dangerzone) can only open a single file/URL at once.
This changes the field code to `%F` as Dangerzone (now) supports
converting multiple files at once. We use `%F` (multiple local files)
instead of `%U` (multiple files and/or URLs) since Dangerzone does not
support opening URLs.
See also the Desktop Entry Specification [1] for more information on the
field codes.
Fixes#797
[1]: https://specifications.freedesktop.org/desktop-entry-spec/latest/ar01s07.html
Currently, the app ID of the Dangerzone GUI application when running
under Wayland is `python3`, which is not very useful if one wants to
automate some action related to the Dangerzone application window (e.g.
to always start Dangerzone window in floating mode under Sway WM).
Setting the desktop filename property also sets the app ID of the
application under Wayland. According to Qt documentation[1], the
property value should be the name of the application's .desktop file but
without the extension.
Qt documentation also states:
> This property gives a precise indication of what desktop entry
> represents the application and it is needed by the windowing system to
> retrieve such information without resorting to imprecise heuristics.
Therefore I also think that setting this property is needed to display
the correct application name and icon (taken from the .desktop entry)
when running under certain windowing systems (like Wayland)
(see also #402).
Note that this property is not enough, as we've encountered systems
where setting just the desktop file name does not alter the detected
application name by the window manager. For this reason, we also use
set the application name [2] to `dangerzone`, to remove any ambiguity.
[1]: https://doc.qt.io/qt-6/qguiapplication.html#desktopFileName-prop
[2]: https://doc.qt.io/qt-6/qcoreapplication.html#applicationName-propFixes#402
Bump our poetry.lock file, to get the latest versions of our
dependencies. Note that we are aware that this bump does not bring in
the latest PySide6 version.
Refs #773
Update the description in the pre-release task for PySide6, since a lot
has changed after writing this section. Now that `python3-pyside6` is in
the Fedora Rawhide repo, and will soon get backported in the stable
repos, we no longer check for newer upstream versions, but if Fedora
finally did the backport.
Refs freedomofpress/maint-dangerzone-pyside6#5
On Unix systems a filename can be a sequence of bytes that is not valid
UTF-8. Python uses[1] surrogate escapes to allow to decode such
filenames to Unicode (bytes that cannot be decoded are replaced by a
surrogate; upon encoding the surrogate is converted to the original
byte).
From `click` docs[2]:
> Invalid bytes or surrogate escapes will raise an error when written
> to a stream with `errors="strict"`. This will typically happen with
> `stdout` when the locale is something like `en_GB.UTF-8`.
To fix that, we use `utils.replace_control_chars()` before printing the
filenames to `stdout` so that surrogate escapes are replaced by �.
Fixes#768
The `util.replace_control_chars()` function was overly strict, and
would replace every non-ASCII character with "_". This included both
control characters, as well as normal characters in a non-English
alphabet.
Relax these restrictions by checking each character and deciding if it's
a Unicode control character, using the `unicodedata` Python package.
With this change, emojis and non-English letters are now allowed.
Add a pytest fixture that crafts a filename with Unicode characters that
are not considered common for this use. By default, this fixture uses
an invalid Unicode character as well, but we strip it in case of macOS
(APFS) since filenames must be UTF-8 encoded.
[1]: https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations
Include the test dependencies when linting, especially `pytest`. We need
this because `mypy` cannot understand the `pytest.raises` exception, and
specifically the fact that it catches exceptions. It assumes that the
next code block is unreachable, since it doesn't see any try ... except.
