Merge e6380f793b into c89988654c

There are various place in our release process
(build/installation/release instructions and CI checks) where we make
sure that the FPF-maintained PySide6 package works in Fedora 39. Now
that Fedora 39 is nearing its EOL date, we can remove those.

.github/workflows/build.yml

@@ -47,8 +47,6 @@ jobs:
             version: bookworm
           - distro: debian
             version: trixie
-          - distro: fedora
-            version: "39"
           - distro: fedora
             version: "40"
           - distro: fedora

.github/workflows/ci.yml

@@ -288,7 +288,7 @@ jobs:
     strategy:
       matrix:
         distro: ["fedora"]
-        version: ["39", "40", "41"]
+        version: ["40", "41"]
     steps:
       - name: Checkout
         uses: actions/checkout@v4
@@ -332,7 +332,7 @@ jobs:
         run: |
           ./dev_scripts/env.py --distro ${{ matrix.distro }} \
               --version ${{ matrix.version }} \
-              build --download-pyside6
+              build
 
       - name: Run a test command
         run: |
@@ -367,8 +367,6 @@ jobs:
             version: bookworm
           - distro: debian
             version: trixie
-          - distro: fedora
-            version: "39"
           - distro: fedora
             version: "40"
           - distro: fedora

.github/workflows/scan_released.yml

@@ -11,15 +11,16 @@ jobs:
         include:
           - runs-on: ubuntu-latest
             arch: i686
-          - runs-on: macos-latest
+          # Do not scan Silicon mac for now to avoid masking release scan results for other plaforms.
-            arch: arm64
+          # - runs-on: macos-latest
+          #   arch: arm64
     runs-on: ${{ matrix.runs-on }}
     steps:
       - name: Checkout
         uses: actions/checkout@v4
       - name: Download container image for the latest release and load it
         run: |
-          VERSION=$(curl https://api.github.com/repos/freedomofpress/dangerzone/releases/latest | jq -r '.tag_name')
+          VERSION=$(curl https://api.github.com/repos/freedomofpress/dangerzone/releases/latest | grep "tag_name" | cut -d '"' -f 4)
           CONTAINER_FILENAME=container-${VERSION:1}-${{ matrix.arch }}.tar.gz
           wget https://github.com/freedomofpress/dangerzone/releases/download/${VERSION}/${CONTAINER_FILENAME} -O ${CONTAINER_FILENAME}
           docker load -i ${CONTAINER_FILENAME}

BUILD.md

@@ -260,11 +260,17 @@ The following instructions require typing commands in a terminal in dom0.
 
    ```
    qvm-create --class AppVM --label red --template fedora-40-dz dz
+   qvm-volume resize dz:private $(numfmt --from=auto 20Gi)
    ```
 
    > :bulb: Alternatively, you can use a different app qube for Dangerzone
    > development. In that case, replace `dz` with the qube of your choice in the
    > steps below.
+   >
+   > In the commands above, we also resize the private volume of the `dz` qube
+   > to 20GiB, since you may need some extra storage space when developing on
+   > Dangerzone (e.g., for container images, Tesseract data, and Python
+   > virtualenvs).
 
 4. Add an RPC policy (`/etc/qubes/policy.d/50-dangerzone.policy`) that will
    allow launching a disposable qube (`dz-dvm`) when Dangerzone converts a
@@ -308,18 +314,9 @@ test it.
 1. Install the `.rpm` package you just copied
 
    ```sh
-   sudo dnf install 'dnf-command(config-manager)'
-   sudo dnf-3 config-manager --add-repo=https://packages.freedom.press/yum-tools-prod/dangerzone/dangerzone.repo
    sudo dnf install ~/QubesIncoming/dz/*.rpm
    ```
 
-   In the above steps, we add the Dangerzone repo because it includes the
-   necessary PySide6 RPM in order to make Dangerzone work.
-
-   > [!NOTE]
-   > During the installation, you will be asked to
-   > [verify the Dangerzone GPG key](INSTALL.md#verifying-dangerzone-gpg-key).
-
 2. Shutdown the `fedora-40-dz` template
 
 ### Developing Dangerzone

CHANGELOG.md

@@ -12,6 +12,10 @@ since 0.4.1, and this project adheres to [Semantic Versioning](https://semver.or
 - Disable gVisor's DirectFS feature ([#226](https://github.com/freedomofpress/dangerzone/issues/226)).
   Thanks [EtiennePerot](https://github.com/EtiennePerot) for the contribution.
 
+### Removed
+
+- Platform support: Drop support for Fedora 39, since it's end-of-life ([#999](https://github.com/freedomofpress/dangerzone/pull/999))
+
 ## [0.8.0](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...0.7.1)
 
 ### Added

INSTALL.md

@@ -1,8 +1,21 @@
 ## MacOS
-See instructions in [README.md](README.md#macos).
+
+- Download [Dangerzone 0.8.0 for Mac (Apple Silicon CPU)](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0-arm64.dmg)
+- Download [Dangerzone 0.8.0 for Mac (Intel CPU)](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0-i686.dmg)
+
+You can also install Dangerzone for Mac using [Homebrew](https://brew.sh/): `brew install --cask dangerzone`
+
+> **Note**: you will also need to install [Docker Desktop](https://www.docker.com/products/docker-desktop/).
+> This program needs to run alongside Dangerzone at all times, since it is what allows Dangerzone to
+> create the secure environment.
 
 ## Windows
-See instructions in [README.md](README.md#windows).
+
+- Download [Dangerzone 0.8.0 for Windows](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0.msi)
+
+> **Note**: you will also need to install [Docker Desktop](https://www.docker.com/products/docker-desktop/).
+> This program needs to run alongside Dangerzone at all times, since it is what allows Dangerzone to
+> create the secure environment.
 
 ## Linux
 On Linux, Dangerzone uses [Podman](https://podman.io/) instead of Docker Desktop for creating
@@ -18,7 +31,6 @@ Dangerzone is available for:
 - Debian 11 (bullseye)
 - Fedora 41
 - Fedora 40
-- Fedora 39
 - Tails
 - Qubes OS (beta support)
 
@@ -125,23 +137,6 @@ sudo apt install -y dangerzone
 
 ### Fedora
 
-<table>
-  <tr>
-    <td>
-<details>
-  <summary><i>:information_source: Backport notice for Fedora users regarding the <code>python3-pyside6</code> package</i></summary>
-  </br>
-
-  Fedora 39+ onwards does not provide official Python bindings for Qt. For
-  this reason, we provide our own `python3-pyside6` package (see
-  [build instructions](https://github.com/freedomofpress/maint-dangerzone-pyside6))
-  from our YUM repo. For a deeper dive on this subject, you may read
-  [this issue](https://github.com/freedomofpress/dangerzone/issues/211#issuecomment-1827777122).
-</details>
-    </td>
-  </tr>
-</table>
-
 Type the following commands in a terminal:
 
 ```

README.md

@@ -6,33 +6,21 @@ Take potentially dangerous PDFs, office documents, or images and convert them to
 | ![Settings](./assets/screenshot1.png) | ![Converting](./assets/screenshot2.png)
 |--|--|
 
-Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, in a separate sandbox, Dangerzone takes this pixel data and converts it back into a PDF.
+Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, outside of the sandbox, Dangerzone takes this pixel data and converts it back into a PDF.
 
 _Read more about Dangerzone in the [official site](https://dangerzone.rocks/about/)._
 
 ## Getting started
 
-### MacOS
+Follow the instructions for each platform:
-- Download [Dangerzone 0.8.0 for Mac (Apple Silicon CPU)](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0-arm64.dmg)
-- Download [Dangerzone 0.8.0 for Mac (Intel CPU)](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0-i686.dmg)
 
-You can also install Dangerzone for Mac using [Homebrew](https://brew.sh/): `brew install --cask dangerzone`
+* [macOS](https://github.com/freedomofpress/dangerzone/blob/v0.8.0//INSTALL.md#macos)
-
+* [Windows](https://github.com/freedomofpress/dangerzone/blob/v0.8.0//INSTALL.md#windows)
-> **Note**: you will also need to install [Docker Desktop](https://www.docker.com/products/docker-desktop/).
+* [Ubuntu Linux](https://github.com/freedomofpress/dangerzone/blob/v0.8.0/INSTALL.md#ubuntu-debian)
-> This program needs to run alongside Dangerzone at all times, since it is what allows Dangerzone to
+* [Debian Linux](https://github.com/freedomofpress/dangerzone/blob/v0.8.0/INSTALL.md#ubuntu-debian)
-> create the secure environment.
+* [Fedora Linux](https://github.com/freedomofpress/dangerzone/blob/v0.8.0/INSTALL.md#fedora)
-
+* [Qubes OS (beta)](https://github.com/freedomofpress/dangerzone/blob/v0.8.0/INSTALL.md#qubes-os)
-### Windows
+* [Tails](https://github.com/freedomofpress/dangerzone/blob/v0.8.0/INSTALL.md#tails)
-
-- Download [Dangerzone 0.8.0 for Windows](https://github.com/freedomofpress/dangerzone/releases/download/v0.8.0/Dangerzone-0.8.0.msi)
-
-> **Note**: you will also need to install [Docker Desktop](https://www.docker.com/products/docker-desktop/).
-> This program needs to run alongside Dangerzone at all times, since it is what allows Dangerzone to
-> create the secure environment.
-
-### Linux
-
-See [installing Dangerzone](INSTALL.md#linux) for adding the Linux repositories to your system.
 
 ## Some features
 

RELEASE.md

@@ -9,7 +9,6 @@ Before making a release, all of these should be complete:
 - [ ] Copy the checkboxes from these instructions onto a new issue and call it **QA and Release version \<VERSION\>**
 - [ ] [Add new Linux platforms and remove obsolete ones](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#add-new-platforms-and-remove-obsolete-ones)
 - [ ] Bump the Python dependencies using `poetry lock`
-- [ ] [Check for official PySide6 versions](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#check-for-official-pyside6-versions)
 - [ ] Update `version` in `pyproject.toml`
 - [ ] Update `share/version.txt`
 - [ ] Update the "Version" field in `install/linux/dangerzone.spec`
@@ -44,16 +43,6 @@ In case of an EOL version:
    * Consult the previous paragraph, but also `grep` your way around.
 2. Add a notice in our `CHANGELOG.md` about the version removal.
 
-## Check for official PySide6 versions
-
-PySide6 6.7.0 is available from the Fedora Rawhide repo, and we expect that a
-similar version will be pushed soon to the rest of the stable releases. Prior to
-a release, we should check if this has happened already. Once this happens, we
-should update our CI tests accordingly, and remove this notice.
-
-For more info, read:
-https://github.com/freedomofpress/maint-dangerzone-pyside6/issues/5
-
 ## Large Document Testing
 
 Parallel to the QA process, the release candidate should be put through the large document tests in a dedicated machine to run overnight.

dev_scripts/env.py

@@ -16,42 +16,6 @@ DEFAULT_USER = "user"
 DEFAULT_DRY = False
 DEFAULT_DEV = False
 DEFAULT_SHOW_DOCKERFILE = False
-DEFAULT_DOWNLOAD_PYSIDE6 = False
-
-PYSIDE6_VERSION = "6.7.1"
-PYSIDE6_RPM = "python3-pyside6-{pyside6_version}-1.fc{fedora_version}.x86_64.rpm"
-PYSIDE6_URL = (
-    "https://packages.freedom.press/yum-tools-prod/dangerzone/f{fedora_version}/%s"
-    % PYSIDE6_RPM
-)
-
-PYSIDE6_DL_MESSAGE = """\
-Downloading PySide6 RPM from:
-
-    {pyside6_url}
-
-into the following local path:
-
-    {pyside6_local_path}
-
-The RPM is over 100 MB, so this operation may take a while...
-"""
-
-PYSIDE6_NOT_FOUND_ERROR = """\
-The following package is not present in your system:
-
-    {pyside6_local_path}
-
-You can build it locally and copy it in the expected path, following the instructions
-in:
-
-    https://github.com/freedomofpress/python3-pyside6-rpm
-
-Alternatively, you can rerun the command adding the '--download-pyside6' flag, which
-will download it from:
-
-    {pyside6_url}
-"""
 
 # The Linux distributions that we currently support.
 # FIXME: Add a version mapping to avoid mistakes.
@@ -232,11 +196,6 @@ RUN apt-get update \
     && rm -rf /var/lib/apt/lists/*
 """
 
-DOCKERFILE_BUILD_FEDORA_39_DEPS = r"""
-COPY {pyside6_rpm} /tmp/pyside6.rpm
-RUN dnf install -y /tmp/pyside6.rpm
-"""
-
 DOCKERFILE_BUILD_FEDORA_DEPS = r"""
 RUN dnf install -y mupdf thunar && dnf clean all
 
@@ -390,74 +349,6 @@ def get_files_in(*folders: list[str]) -> list[pathlib.Path]:
     return files
 
 
-class PySide6Manager:
-    """Provision PySide6 RPMs in our Dangerzone environments.
-
-    This class holds all the logic around checking and downloading PySide RPMs. It can
-    check if the required RPM version is present under "/dist", and optionally download
-    it.
-    """
-
-    def __init__(self, distro_name, distro_version):
-        if distro_name != "fedora":
-            raise RuntimeError("Managing PySide6 RPMs is available only in Fedora")
-        self.distro_name = distro_name
-        self.distro_version = distro_version
-
-    @property
-    def version(self):
-        """The version of the PySide6 RPM."""
-        return PYSIDE6_VERSION
-
-    @property
-    def rpm_name(self):
-        """The name of the PySide6 RPM."""
-        return PYSIDE6_RPM.format(
-            pyside6_version=self.version, fedora_version=self.distro_version
-        )
-
-    @property
-    def rpm_url(self):
-        """The URL of the PySide6 RPM, as hosted in FPF's RPM repo."""
-        return PYSIDE6_URL.format(
-            pyside6_version=self.version,
-            fedora_version=self.distro_version,
-        )
-
-    @property
-    def rpm_local_path(self):
-        """The local path where this script will look for the PySide6 RPM."""
-        return git_root() / "dist" / self.rpm_name
-
-    @property
-    def is_rpm_present(self):
-        """Check if PySide6 RPM is present in the user's system."""
-        return self.rpm_local_path.exists()
-
-    def download_rpm(self):
-        """Download PySide6 from FPF's RPM repo."""
-        print(
-            PYSIDE6_DL_MESSAGE.format(
-                pyside6_url=self.rpm_url,
-                pyside6_local_path=self.rpm_local_path,
-            ),
-            file=sys.stderr,
-        )
-        try:
-            with (
-                urllib.request.urlopen(self.rpm_url) as r,
-                open(self.rpm_local_path, "wb") as f,
-            ):
-                shutil.copyfileobj(r, f)
-        except:
-            # NOTE: We purposefully catch all exceptions, since we want to catch Ctrl-C
-            # as well.
-            print("Download interrupted, removing file", file=sys.stderr)
-            self.rpm_local_path.unlink()
-            raise
-        print("PySide6 was downloaded successfully", file=sys.stderr)
-
-
 class Env:
     """A class that implements actions on Dangerzone environments"""
 
@@ -736,7 +627,6 @@ class Env:
     def build(
         self,
         show_dockerfile=DEFAULT_SHOW_DOCKERFILE,
-        download_pyside6=DEFAULT_DOWNLOAD_PYSIDE6,
     ):
         """Build a Linux environment and install Dangerzone in it."""
         build_dir = distro_build(self.distro, self.version)
@@ -749,28 +639,6 @@ class Env:
             package = package_src.name
             package_dst = build_dir / package
             install_cmd = "dnf install -y"
-
-            # NOTE: For Fedora 39, we check if a PySide6 RPM package exists in
-            # the user's system. If not, we either throw an error or download it from
-            # FPF's repo, according to the user's choice.
-            if self.version == "39":
-                pyside6 = PySide6Manager(self.distro, self.version)
-                if not pyside6.is_rpm_present:
-                    if download_pyside6:
-                        pyside6.download_rpm()
-                    else:
-                        print(
-                            PYSIDE6_NOT_FOUND_ERROR.format(
-                                pyside6_local_path=pyside6.rpm_local_path,
-                                pyside6_url=pyside6.rpm_url,
-                            ),
-                            file=sys.stderr,
-                        )
-                        return 1
-                shutil.copy(pyside6.rpm_local_path, build_dir / pyside6.rpm_name)
-                install_deps = (
-                    DOCKERFILE_BUILD_FEDORA_DEPS + DOCKERFILE_BUILD_FEDORA_39_DEPS
-                ).format(pyside6_rpm=pyside6.rpm_name)
         else:
             install_deps = DOCKERFILE_BUILD_DEBIAN_DEPS
             if self.distro == "ubuntu" and self.version in ("20.04", "focal"):
@@ -844,7 +712,6 @@ def env_build(args):
     env = Env.from_args(args)
     return env.build(
         show_dockerfile=args.show_dockerfile,
-        download_pyside6=args.download_pyside6,
     )
 
 
@@ -941,12 +808,6 @@ def parse_args():
         action="store_true",
         help="Do not build, only show the Dockerfile",
     )
-    parser_build.add_argument(
-        "--download-pyside6",
-        default=DEFAULT_DOWNLOAD_PYSIDE6,
-        action="store_true",
-        help="Download PySide6 from FPF's RPM repo",
-    )
 
     return parser.parse_args()
 

dev_scripts/qa.py

@@ -3,14 +3,20 @@
 import abc
 import argparse
 import difflib
+import json
 import logging
 import re
 import selectors
 import subprocess
 import sys
+import urllib.request
+from pathlib import Path
 
 logger = logging.getLogger(__name__)
 
+PYTHON_VERSION = "3.12"
+EOL_PYTHON_URL = "https://endoflife.date/api/python.json"
+
 CONTENT_QA = r"""## QA
 
 To ensure that new releases do not introduce regressions, and support existing
@@ -776,6 +782,10 @@ class QABase(abc.ABC):
                 self.prompt("Does it pass?", choices=["y", "n"])
         logger.info("Successfully completed QA scenarios")
 
+    @task("Download Tesseract data", auto=True)
+    def download_tessdata(self):
+        self.run("python", str(Path("install", "common", "download-tessdata.py")))
+
     @classmethod
     @abc.abstractmethod
     def get_id(cls):
@@ -802,6 +812,40 @@ class QAWindows(QABase):
         while msvcrt.kbhit():
             msvcrt.getch()
 
+    def get_latest_python_release(self):
+        with urllib.request.urlopen(EOL_PYTHON_URL) as f:
+            resp = f.read()
+            releases = json.loads(resp)
+            for release in releases:
+                if release["cycle"] == PYTHON_VERSION:
+                    # Transform the Python version string (e.g., "3.12.7") into a list
+                    # (e.g., [3, 12, 7]), and return it
+                    return [int(num) for num in release["latest"].split(".")]
+
+            raise RuntimeError(
+                f"Could not find a Python release for version {PYTHON_VERSION}"
+            )
+
+    @QABase.task(
+        f"Install the latest version of Python {PYTHON_VERSION}", ref=REF_BUILD
+    )
+    def install_python(self):
+        logger.info("Getting latest Python release")
+        try:
+            latest_version = self.get_latest_python_release()
+        except Exception:
+            logger.error("Could not verify that the latest Python version is installed")
+
+        cur_version = list(sys.version_info[:3])
+        if latest_version > cur_version:
+            self.prompt(
+                f"You need to install the latest Python version ({latest_version})"
+            )
+        elif latest_version == cur_version:
+            logger.info(
+                f"Verified that the latest Python version ({latest_version}) is installed"
+            )
+
     @QABase.task("Install and Run Docker Desktop", ref=REF_BUILD)
     def install_docker(self):
         logger.info("Checking if Docker Desktop is installed and running")
@@ -816,7 +860,7 @@ class QAWindows(QABase):
     )
     def install_poetry(self):
         self.run("python", "-m", "pip", "install", "poetry")
-        self.run("poetry", "install")
+        self.run("poetry", "install", "--sync")
 
     @QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True)
     def build_image(self):
@@ -838,9 +882,11 @@ class QAWindows(QABase):
         return "windows"
 
     def start(self):
+        self.install_python()
         self.install_docker()
         self.install_poetry()
         self.build_image()
+        self.download_tessdata()
         self.run_tests()
         self.build_dangerzone_exe()
 
@@ -915,7 +961,6 @@ class QALinux(QABase):
             "--version",
             self.VERSION,
             "build",
-            "--download-pyside6",
         )
 
     @classmethod
@@ -933,6 +978,7 @@ class QALinux(QABase):
     def start(self):
         self.build_dev_image()
         self.build_container_image()
+        self.download_tessdata()
         self.run_tests()
         self.build_package()
         self.build_qa_image()
@@ -1009,10 +1055,6 @@ class QAFedora40(QAFedora):
     VERSION = "40"
 
 
-class QAFedora39(QAFedora):
-    VERSION = "39"
-
-
 def parse_args():
     parser = argparse.ArgumentParser(
         prog=sys.argv[0],