Compare commits

..

18 commits

Author SHA1 Message Date
Alex Pyrgiotis
19fa11410b
Update reference template for Qubes to Fedora 41
Closes #1078
2025-04-08 16:37:28 +03:00
Alex Pyrgiotis
10be85b9f2
container: Add workarounds for Podman Desktop support on Windows
In case we run on Windows and use Podman Desktop (for which we currently
offer experimental support), we must not pass some Podman flags in order
to avoid conversion errors.

Refs #1127
2025-04-08 16:36:08 +03:00
Alexis Métaireau
47d732e603
Document the Makefile targets
It now outputs the following:

```
build-linux                  Build linux packages (.rpm and .deb)
build-macos-arm              Build macOS Apple Silicon package (.dmg)
build-macos-intel            Build macOS intel package (.dmg)
Dockerfile                   Regenerate the Dockerfile from its template
fix                          apply all the suggestions from ruff
help                         Print this message and exit.
lint                         Check the code for linting, formatting, and typing issues with ruff and mypy
regenerate-reference-pdfs    Regenerate the reference PDFs
test                         Run the tests
test-large                   Run large test set
```
2025-04-08 16:34:34 +03:00
Alexis Métaireau
d6451290db
Move multithreading patch up so that it's working in the GUI 2025-04-08 16:34:34 +03:00
Alex Pyrgiotis
f0bb65cb4e
Bypass a cx-freeze issue for fitz._wxcolors
Bypass an issue with `cx-freeze` that fails to include the
`fitz._wxcolors` module in the final Windows artifact.

Refs #1128
2025-04-08 16:34:34 +03:00
Alex Pyrgiotis
0c741359cc
Make our build-image.py script runable on Windows 2025-04-08 16:34:34 +03:00
Alex Pyrgiotis
8c61894e25
Handle the case where Docker is not installed
Refs #1132
2025-04-08 16:33:15 +03:00
Alex Pyrgiotis
57667a96be
Add a way to unset the container runtime
Add a way to set the container runtime that Dangerzone uses back to the
default.
2025-04-07 18:23:13 +03:00
Alex Pyrgiotis
1a644e2506
Do not install poetry-plugin-export
Do not unconditionally install the Poetry plugin for exporting
dependencies as a requirements.txt file, since it's used only when
building a Debian package. Keep it instead in the Linux instructions and
when building a Dangerzone environment.
2025-04-07 18:23:10 +03:00
Alex Pyrgiotis
843e68cdf7
Handle the case of empty tesseract dirs during download 2025-04-07 18:22:52 +03:00
Alex Pyrgiotis
33b2a183ce
docs: Improve doit docs 2025-04-07 18:22:52 +03:00
Alex Pyrgiotis
c7121b69a3
Prefer poetry sync to poetry install --sync
Use `poetry sync` instead of `poetry install --sync`, since the latter
is deprecated and will be removed after June 2025, as seen in the
following warning message:

  The `--sync` option is deprecated and slated for removal in the next
  minor release after June 2025, use the `poetry sync` command instead.
2025-04-07 18:22:50 +03:00
Alex Pyrgiotis
0b3bf89d5b
Implicitly run doit with poetry run
Implicitly run `doit` with `poetry run`, else `poetry env remove --all`
will remove the calling Python interpreter.
2025-04-02 12:01:14 +03:00
Alex Pyrgiotis
e0b10c5e40
doit: Remove tessdata dir from targets
Remove the tesseract data dir from the doit targets, else we encounter
the following error:

  Traceback (most recent call last):
    [...]
    File "[...]/Library/Caches/pypoetry/virtualenvs/dangerzone-52Yr5wv_-py3.11/lib/python3.11/site-packages/doit/dependency.py", line 39, in get_file_md5
      with open(path, 'rb') as file_data:
           ^^^^^^^^^^^^^^^^
  IsADirectoryError: [Errno 21] Is a directory: 'share/tessdata'
2025-04-02 11:46:20 +03:00
Alex Pyrgiotis
092eec55d1
doit: Remove unused 'DEBIAN_VERSIONS' variable 2025-04-02 11:45:47 +03:00
Alex Pyrgiotis
14a480c3a3
doit: Fix typo in Fedora targets
Fix a typo when building a Fedora target. Also, add Fedora 42 support.
2025-04-02 11:44:50 +03:00
Alex Pyrgiotis
9df825db5c
debian: Use abbreviated months in changelog
Use abbreviated months in the Debian changelog, else we'll have warnings
like the following:

  LINE:  -- Freedom of the Press Foundation   <info@freedom.press>  Mon, 31 March 2025 15:57:18 +0300
  dpkg-source: warning: dangerzone/debian/changelog(l5): cannot parse non-conformant date '31 March 20
2025-04-02 11:35:31 +03:00
Alex Pyrgiotis
2ee22a497a
Reinstall deps after doit cleans everything
Make sure to reinstall the project dependencies once `doit clean` runs,
since it also removes itself.
2025-04-02 11:30:31 +03:00
15 changed files with 83 additions and 44 deletions

View file

@ -113,7 +113,7 @@ Install Poetry using `pipx`:
```sh
pipx install poetry
pipx inject poetry poetry-plugin-export
pipx inject poetry
```
Clone this repository:
@ -326,7 +326,7 @@ cd dangerzone
Install Python dependencies:
```sh
python3 -m pip install poetry poetry-plugin-export
python3 -m pip install poetry
poetry install
```
@ -387,7 +387,7 @@ Install Microsoft Visual C++ 14.0 or greater. Get it with ["Microsoft C++ Build
Install [poetry](https://python-poetry.org/). Open PowerShell, and run:
```
python -m pip install poetry poetry-plugin-export
python -m pip install poetry
```
Install git from [here](https://git-scm.com/download/win), open a Windows terminal (`cmd.exe`) and clone this repository:

View file

@ -22,7 +22,7 @@ fix: ## apply all the suggestions from ruff
ruff format
.PHONY: test
test:
test: ## Run the tests
# Make each GUI test run as a separate process, to avoid segfaults due to
# shared state.
# See more in https://github.com/freedomofpress/dangerzone/issues/493
@ -47,27 +47,31 @@ test-large: test-large-init ## Run large test set
python -m pytest --tb=no tests/test_large_set.py::TestLargeSet -v $(JUNIT_FLAGS) --junitxml=$(TEST_LARGE_RESULTS)
python $(TEST_LARGE_RESULTS)/report.py $(TEST_LARGE_RESULTS)
Dockerfile: Dockerfile.env Dockerfile.in
Dockerfile: Dockerfile.env Dockerfile.in ## Regenerate the Dockerfile from its template
poetry run jinja2 Dockerfile.in Dockerfile.env > Dockerfile
.PHONY: poetry-install
poetry-install: ## Install project dependencies
poetry install
.PHONY: build-clean
build-clean:
doit clean
poetry run doit clean
.PHONY: build-macos-intel
build-macos-intel: build-clean
doit -n 8
build-macos-intel: build-clean poetry-install ## Build macOS intel package (.dmg)
poetry run doit -n 8
.PHONY: build-macos-arm
build-macos-arm: build-clean
doit -n 8 macos_build_dmg
build-macos-arm: build-clean poetry-install ## Build macOS Apple Silicon package (.dmg)
poetry run doit -n 8 macos_build_dmg
.PHONY: build-linux
build-linux: build-clean
doit -n 8 fedora_rpm debian_deb
build-linux: build-clean poetry-install ## Build linux packages (.rpm and .deb)
poetry run doit -n 8 fedora_rpm debian_deb
.PHONY: regenerate-reference-pdfs
regenerate-reference-pdfs:
regenerate-reference-pdfs: ## Regenerate the reference PDFs
pytest tests/test_cli.py -k regenerate --generate-reference-pdfs
# Makefile self-help borrowed from the securedrop-client project
# Explaination of the below shell command should it ever break.

View file

@ -123,7 +123,7 @@ Here is what you need to do:
# In case of a new Python installation or minor version upgrade, e.g., from
# 3.11 to 3.12, reinstall Poetry
python3 -m pip install poetry poetry-plugin-export
python3 -m pip install poetry
# You can verify the correct Python version is used
poetry debug info
@ -141,7 +141,7 @@ Here is what you need to do:
poetry env remove --all
# Install the dependencies
poetry install --sync
poetry sync
```
- [ ] Build the container image and the OCR language data
@ -205,7 +205,7 @@ The Windows release is performed in a Windows 11 virtual machine (as opposed to
```bash
# In case of a new Python installation or minor version upgrade, e.g., from
# 3.11 to 3.12, reinstall Poetry
python3 -m pip install poetry poetry-plugin-export
python3 -m pip install poetry
# You can verify the correct Python version is used
poetry debug info
@ -223,7 +223,7 @@ The Windows release is performed in a Windows 11 virtual machine (as opposed to
poetry env remove --all
# Install the dependencies
poetry install --sync
poetry sync
```
- [ ] Copy the container image into the VM

View file

@ -4,6 +4,12 @@ import sys
logger = logging.getLogger(__name__)
# Call freeze_support() to avoid passing unknown options to the subprocess.
# See https://github.com/freedomofpress/dangerzone/issues/873
import multiprocessing
multiprocessing.freeze_support()
try:
from . import vendor # type: ignore [attr-defined]

View file

@ -52,7 +52,11 @@ def print_header(s: str) -> None:
@click.option(
"--set-container-runtime",
required=False,
help="The path to the container runtime you want to set in the settings",
help=(
"The name or full path of the container runtime you want Dangerzone to use."
" You can specify the value 'default' if you want to take back your choice, and"
" let Dangerzone use the default runtime for this OS"
),
)
@click.version_option(version=get_version(), message="%(version)s")
@errors.handle_document_errors
@ -69,10 +73,16 @@ def cli_main(
display_banner()
if set_container_runtime:
settings = Settings()
container_runtime = settings.set_custom_runtime(
set_container_runtime, autosave=True
)
click.echo(f"Set the settings container_runtime to {container_runtime}")
if set_container_runtime == "default":
settings.unset_custom_runtime()
click.echo(
"Instructed Dangerzone to use the default container runtime for this OS"
)
else:
container_runtime = settings.set_custom_runtime(
set_container_runtime, autosave=True
)
click.echo(f"Set the settings container_runtime to {container_runtime}")
sys.exit(0)
elif not filenames:
raise click.UsageError("Missing argument 'FILENAMES...'")

View file

@ -3,7 +3,6 @@ import os
import platform
import tempfile
import typing
from multiprocessing import freeze_support
from multiprocessing.pool import ThreadPool
from pathlib import Path
from typing import List, Optional
@ -228,7 +227,9 @@ class MainWindow(QtWidgets.QMainWindow):
if not is_version_valid:
self.handle_docker_desktop_version_check(is_version_valid, version)
except errors.UnsupportedContainerRuntime as e:
pass # It's catched later in the flow.
pass # It's caught later in the flow.
except errors.NoContainerTechException as e:
pass # It's caught later in the flow.
self.show()
@ -1236,9 +1237,6 @@ class DocumentsListWidget(QtWidgets.QListWidget):
def start_conversion(self) -> None:
if not self.thread_pool_initized:
max_jobs = self.dangerzone.isolation_provider.get_max_parallel_conversions()
# Call freeze_support() to avoid passing unknown options to the subprocess.
# See https://github.com/freedomofpress/dangerzone/issues/873
freeze_support()
self.thread_pool = ThreadPool(max_jobs)
for doc in self.docs_list:

View file

@ -56,7 +56,14 @@ class Container(IsolationProvider):
security_args = ["--log-driver", "none"]
security_args += ["--security-opt", "no-new-privileges"]
if container_utils.get_runtime_version() >= (4, 1):
security_args += ["--userns", "nomap"]
# We perform a platform check to avoid the following Podman Desktop
# error on Windows:
#
# Error: nomap is only supported in rootless mode
#
# See also: https://github.com/freedomofpress/dangerzone/issues/1127
if platform.system() != "Windows":
security_args += ["--userns", "nomap"]
else:
security_args = ["--security-opt=no-new-privileges:true"]
@ -67,7 +74,15 @@ class Container(IsolationProvider):
# [1] https://github.com/freedomofpress/dangerzone/issues/846
# [2] https://github.com/containers/common/blob/d3283f8401eeeb21f3c59a425b5461f069e199a7/pkg/seccomp/seccomp.json
seccomp_json_path = str(get_resource_path("seccomp.gvisor.json"))
security_args += ["--security-opt", f"seccomp={seccomp_json_path}"]
# We perform a platform check to avoid the following Podman Desktop
# error on Windows:
#
# Error: opening seccomp profile failed: open
# C:\[...]\dangerzone\share\seccomp.gvisor.json: no such file or directory
#
# See also: https://github.com/freedomofpress/dangerzone/issues/1127
if runtime.name == "podman" and platform.system() != "Windows":
security_args += ["--security-opt", f"seccomp={seccomp_json_path}"]
security_args += ["--cap-drop", "all"]
security_args += ["--cap-add", "SYS_CHROOT"]

View file

@ -52,6 +52,10 @@ class Settings:
self.save()
return container_runtime
def unset_custom_runtime(self) -> None:
self.settings.pop("container_runtime")
self.save()
def get(self, key: str) -> Any:
return self.settings[key]

4
debian/changelog vendored
View file

@ -2,13 +2,13 @@ dangerzone (0.9.0) unstable; urgency=low
* Released Dangerzone 0.9.0
-- Freedom of the Press Foundation <info@freedom.press> Mon, 31 March 2025 15:57:18 +0300
-- Freedom of the Press Foundation <info@freedom.press> Mon, 31 Mar 2025 15:57:18 +0300
dangerzone (0.8.1) unstable; urgency=low
* Released Dangerzone 0.8.1
-- Freedom of the Press Foundation <info@freedom.press> Tue, 22 December 2024 22:03:28 +0300
-- Freedom of the Press Foundation <info@freedom.press> Tue, 22 Dec 2024 22:03:28 +0300
dangerzone (0.8.0) unstable; urgency=low

View file

@ -331,7 +331,7 @@ Install Poetry using `pipx`:
```sh
pipx install poetry
pipx inject poetry poetry-plugin-export
pipx inject poetry
```
Clone this repository:
@ -397,7 +397,7 @@ Install Microsoft Visual C++ 14.0 or greater. Get it with ["Microsoft C++ Build
Install [poetry](https://python-poetry.org/). Open PowerShell, and run:
```
python -m pip install poetry poetry-plugin-export
python -m pip install poetry
```
Install git from [here](https://git-scm.com/download/win), open a Windows terminal (`cmd.exe`) and clone this repository:
@ -835,8 +835,8 @@ class QAWindows(QABase):
"Install Poetry and the project's dependencies", ref=REF_BUILD, auto=True
)
def install_poetry(self):
self.run("python", "-m", "pip", "install", "poetry", "poetry-plugin-export")
self.run("poetry", "install", "--sync")
self.run("python", "-m", "pip", "install", "poetry")
self.run("poetry", "sync")
@QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True)
def build_image(self):

View file

@ -42,7 +42,8 @@ doit <task>
## Tips and tricks
* You can run `doit list --all -s` to see the full list of tasks, their
dependencies, and whether they are up to date.
dependencies, and whether they are up to date (U) or will run (R). Note that
certain small tasks are always configured to run.
* You can run `doit info <task>` to see which dependencies are missing.
* You can pass the following environment variables to the script, in order to
affect some global parameters:

View file

@ -8,8 +8,7 @@ from doit.action import CmdAction
ARCH = "arm64" if platform.machine() == "arm64" else "i686"
VERSION = open("share/version.txt").read().strip()
FEDORA_VERSIONS = ["40", "41"]
DEBIAN_VERSIONS = ["bullseye", "jammy", "mantic", "noble", "trixie"]
FEDORA_VERSIONS = ["40", "41", "42"]
### Global parameters
@ -44,7 +43,6 @@ def list_language_data():
tessdata_dir = Path("share") / "tessdata"
langs = json.loads(open(tessdata_dir.parent / "ocr-languages.json").read()).values()
targets = [tessdata_dir / f"{lang}.traineddata" for lang in langs]
targets.append(tessdata_dir)
return targets
@ -124,7 +122,7 @@ def build_deb(cwd):
def build_rpm(version, cwd, qubes=False):
"""Build an .rpm package on the requested Fedora distro."""
return build_linux_pkg(distro="Fedora", version=version, cwd=cwd, qubes=qubes)
return build_linux_pkg(distro="fedora", version=version, cwd=cwd, qubes=qubes)
### Tasks
@ -208,7 +206,7 @@ def task_build_image():
def task_poetry_install():
"""Setup the Poetry environment"""
return {"actions": ["poetry install --sync"], "clean": ["poetry env remove --all"]}
return {"actions": ["poetry sync"], "clean": ["poetry env remove --all"]}
def task_macos_build_dmg():

View file

@ -5,7 +5,7 @@ import subprocess
import sys
from pathlib import Path
BUILD_CONTEXT = "dangerzone/"
BUILD_CONTEXT = "dangerzone"
IMAGE_NAME = "dangerzone.rocks/dangerzone"
if platform.system() in ["Darwin", "Windows"]:
CONTAINER_RUNTIME = "docker"
@ -122,7 +122,8 @@ def main():
subprocess.run(
[
"./dev_scripts/repro-build.py",
sys.executable,
str(Path("dev_scripts") / "repro-build.py"),
"build",
"--runtime",
args.runtime,

View file

@ -51,6 +51,8 @@ def main():
if files == expected_files:
logger.info("Skipping tessdata download, language data already exists")
return
elif not files:
logger.info("Tesseract dir is empty, proceeding to download language data")
else:
logger.info(f"Found {tessdata_dir} but contents do not match")
return 1

View file

@ -13,7 +13,7 @@ setup(
description="Dangerzone",
options={
"build_exe": {
"packages": ["dangerzone", "dangerzone.gui"],
"packages": ["dangerzone", "dangerzone.gui", "pymupdf._wxcolors"],
"excludes": ["test", "tkinter"],
"include_files": [("share", "share"), ("LICENSE", "LICENSE")],
"include_msvcr": True,