almet/dangerzone
1
0
Fork 0
mirror of https://github.com/freedomofpress/dangerzone.git synced 2025-05-04 04:31:49 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 7

Merge 3b3f02528a into 96aa56a6dc

Browse source
This commit is contained in:
Alexis Métaireau 2024-11-07 14:56:08 +00:00 committed by GitHub
commit f176bd5cb2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 454 additions and 265 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
BUILD.md
View file

@ -260,11 +260,16 @@ The following instructions require typing commands in a terminal in dom0.
``` ```
qvm-create --class AppVM --label red --template fedora-40-dz dz qvm-create --class AppVM --label red --template fedora-40-dz dz
qvm-volume resize dz:private $(numfmt --from=auto 5Gi)
``` ```
> :bulb: Alternatively, you can use a different app qube for Dangerzone > :bulb: Alternatively, you can use a different app qube for Dangerzone
> development. In that case, replace `dz` with the qube of your choice in the > development. In that case, replace `dz` with the qube of your choice in the
> steps below. > steps below.
>
> In the commands above, we also resize the private volume of the `dz` qube
> to 5GiB, since the Tesseract data that will be downloaded in the next steps
> take a bit of space.
4. Add an RPC policy (`/etc/qubes/policy.d/50-dangerzone.policy`) that will 4. Add an RPC policy (`/etc/qubes/policy.d/50-dangerzone.policy`) that will
allow launching a disposable qube (`dz-dvm`) when Dangerzone converts a allow launching a disposable qube (`dz-dvm`) when Dangerzone converts a

197
QA.md Normal file
View file

@ -0,0 +1,197 @@
## QA
To ensure that new releases do not introduce regressions, and support existing
and newer platforms, we have to test that the produced packages work as expected.
Check the following:
- [ ] Make sure that the tip of the `main` branch passes the CI tests.
- [ ] Make sure that the Apple account has a valid application password and has
agreed to the latest Apple terms (see [macOS release](#macos-release)
section).
Because it is repetitive, we wrote a script to help with the QA.
It can run the tasks for you, pausing when it needs manual intervention.
You can run it with a command like:
```bash
poetry run ./dev_scripts/qa.py {distro}-{version}
```
### The checklist
- [ ] Create a test build in Windows and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests.
- [ ] Build and run the Dangerzone .exe
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in macOS (Intel CPU) and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in macOS (M1/2 CPU) and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Ubuntu LTS platform (Ubuntu 24.04
as of writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests.
- [ ] Create a .deb package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Fedora platform (Fedora 41 as of
writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests.
- [ ] Create an .rpm package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Qubes Fedora template (Fedora 40 as
of writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Run the Dangerzone tests.
- [ ] Create a Qubes .rpm package and install it system-wide.
- [ ] Ensure that the Dangerzone application appears in the "Applications"
tab.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below) and make sure
they spawn disposable qubes.
### Scenarios
#### 1. Dangerzone correctly identifies that Docker/Podman is not installed
_(Only for MacOS / Windows)_
Temporarily hide the Docker/Podman binaries, e.g., rename the `docker` /
`podman` binaries to something else. Then run Dangerzone. Dangerzone should
prompt the user to install Docker/Podman.
#### 2. Dangerzone correctly identifies that Docker is not running
_(Only for MacOS / Windows)_
Stop the Docker Desktop application. Then run Dangerzone. Dangerzone should
prompt the user to start Docker Desktop.
#### 3. Updating Dangerzone handles external state correctly.
_(Applies to Windows/MacOS)_
Install the previous version of Dangerzone, downloaded from the website.
Open the Dangerzone application and enable some non-default settings.
**If there are new settings, make sure to change those as well**.
Close the Dangerzone application and get the container image for that
version. For example:
```
$ docker images dangerzone.rocks/dangerzone:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
dangerzone.rocks/dangerzone latest <image ID> <date> <size>
```
Then run the version under QA and ensure that the settings remain changed.
Afterwards check that new docker image was installed by running the same command
and seeing the following differences:
```
$ docker images dangerzone.rocks/dangerzone:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
dangerzone.rocks/dangerzone latest <different ID> <newer date> <different size>
```
#### 4. Dangerzone successfully installs the container image
_(Only for Linux)_
Remove the Dangerzone container image from Docker/Podman. Then run Dangerzone.
Dangerzone should install the container image successfully.
#### 5. Dangerzone retains the settings of previous runs
Run Dangerzone and make some changes in the settings (e.g., change the OCR
language, toggle whether to open the document after conversion, etc.). Restart
Dangerzone. Dangerzone should show the settings that the user chose.
#### 6. Dangerzone reports failed conversions
Run Dangerzone and convert the `tests/test_docs/sample_bad_pdf.pdf` document.
Dangerzone should fail gracefully, by reporting that the operation failed, and
showing the following error message:
> The document format is not supported
#### 7. Dangerzone succeeds in converting multiple documents
Run Dangerzone against a list of documents, and tick all options. Ensure that:
* Conversions take place sequentially.
* Attempting to close the window while converting asks the user if they want to
abort the conversions.
* Conversions are completed successfully.
* Conversions show individual progress in real-time (double-check for Qubes).
* _(Only for Linux)_ The resulting files open with the PDF viewer of our choice.
* OCR seems to have detected characters in the PDF files.
* The resulting files have been saved with the proper suffix, in the proper
location.
* The original files have been saved in the `unsafe/` directory.
#### 8. Dangerzone is able to handle drag-n-drop
Run Dangerzone against a set of documents that you drag-n-drop. Files should be
added and conversion should run without issue.
> [!TIP]
> On our end-user container environments for Linux, we can start a file manager
> with `thunar &`.
#### 9. Dangerzone CLI succeeds in converting multiple documents
_(Only for Windows and Linux)_
Run Dangerzone CLI against a list of documents. Ensure that conversions happen
sequentially, are completed successfully, and we see their progress.
#### 10. Dangerzone can open a document for conversion via right-click -> "Open With"
_(Only for Windows, MacOS and Qubes)_
Go to a directory with office documents, right-click on one, and click on "Open
With". We should be able to open the file with Dangerzone, and then convert it.
#### 11. Dangerzone shows helpful errors for setup issues on Qubes
_(Only for Qubes)_
Check what errors does Dangerzone throw in the following scenarios. The errors
should point the user to the Qubes notifications in the top-right corner:
1. The `dz-dvm` template does not exist. We can trigger this scenario by
temporarily renaming this template.
2. The Dangerzone RPC policy does not exist. We can trigger this scenario by
temporarily renaming the `dz.Convert` policy.
3. The `dz-dvm` disposable Qube cannot start due to insufficient resources. We
can trigger this scenario by temporarily increasing the minimum required RAM
of the `dz-dvm` template to more than the available amount.

2
README.md
View file

@ -6,7 +6,7 @@ Take potentially dangerous PDFs, office documents, or images and convert them to
| ![Settings](./assets/screenshot1.png) | ![Converting](./assets/screenshot2.png) | ![Settings](./assets/screenshot1.png) | ![Converting](./assets/screenshot2.png)
|--|--| |--|--|
Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, in a separate sandbox, Dangerzone takes this pixel data and converts it back into a PDF. Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, outside of the sandbox, Dangerzone takes this pixel data and converts it back into a PDF.
_Read more about Dangerzone in the [official site](https://dangerzone.rocks/about/)._ _Read more about Dangerzone in the [official site](https://dangerzone.rocks/about/)._

390
RELEASE.md
View file

@ -1,12 +1,13 @@
# Release instructions # Release instructions
This section documents the release process. Unless you're a dangerzone developer making a release, you'll probably never need to follow it. This section documents how we currently release Dangerzone for the different distributions we support.
## Pre-release ## Pre-release
Before making a release, all of these should be complete: Here is a list of tasks that should be done before issuing the release:
- [ ] Copy the checkboxes from these instructions onto a new issue and call it **QA and Release version \<VERSION\>** - [ ] Create a new issue named **QA and Release for version \<VERSION\>**, to track the general progress.
You can generate its content with the the `poetry run ./dev_scripts/generate-release-tasks.py` command.
- [ ] [Add new Linux platforms and remove obsolete ones](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#add-new-platforms-and-remove-obsolete-ones) - [ ] [Add new Linux platforms and remove obsolete ones](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#add-new-platforms-and-remove-obsolete-ones)
- [ ] Bump the Python dependencies using `poetry lock` - [ ] Bump the Python dependencies using `poetry lock`
- [ ] [Check for official PySide6 versions](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#check-for-official-pyside6-versions) - [ ] [Check for official PySide6 versions](https://github.com/freedomofpress/dangerzone/blob/main/RELEASE.md#check-for-official-pyside6-versions)
@ -16,6 +17,8 @@ Before making a release, all of these should be complete:
- [ ] Bump the Debian version by adding a new changelog entry in `debian/changelog` - [ ] Bump the Debian version by adding a new changelog entry in `debian/changelog`
- [ ] Update screenshot in `README.md`, if necessary - [ ] Update screenshot in `README.md`, if necessary
- [ ] CHANGELOG.md should be updated to include a list of all major changes since the last release - [ ] CHANGELOG.md should be updated to include a list of all major changes since the last release
- [ ] A draft release should be created. Copy the release notes text from the template at [`docs/templates/release-notes`](https://github.com/freedomofpress/dangerzone/tree/main/docs/templates/)
- [ ] Do the QA tasks
## Add new Linux platforms and remove obsolete ones ## Add new Linux platforms and remove obsolete ones
@ -38,7 +41,7 @@ In case of a new version (beta, RC, or official release):
`BUILD.md` files where necessary. `BUILD.md` files where necessary.
4. Send a PR with the above changes. 4. Send a PR with the above changes.
In case of an EOL version: In case of the removal of a version:
1. Remove any mention to this version from our repo. 1. Remove any mention to this version from our repo.
* Consult the previous paragraph, but also `grep` your way around. * Consult the previous paragraph, but also `grep` your way around.
@ -62,192 +65,13 @@ Follow the instructions in `docs/developer/TESTING.md` to run the tests.
These tests will identify any regressions or progression in terms of document coverage. These tests will identify any regressions or progression in terms of document coverage.
## QA
To ensure that new releases do not introduce regressions, and support existing
and newer platforms, we have to do the following:
- [ ] Make sure that the tip of the `main` branch passes the CI tests.
- [ ] Make sure that the Apple account has a valid application password and has
agreed to the latest Apple terms (see [macOS release](#macos-release)
section).
- [ ] Create a test build in Windows and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Run the Dangerzone tests.
- [ ] Build and run the Dangerzone .exe
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in macOS (Intel CPU) and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in macOS (M1/2 CPU) and make sure it works:
- [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Ubuntu LTS platform (Ubuntu 24.04
as of writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Run the Dangerzone tests.
- [ ] Create a .deb package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Fedora platform (Fedora 41 as of
writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses
the new image.
- [ ] Run the Dangerzone tests.
- [ ] Create an .rpm package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
- [ ] Create a test build in the most recent Qubes Fedora template (Fedora 40 as
of writing this) and make sure it works:
- [ ] Create a new development environment with Poetry.
- [ ] Run the Dangerzone tests.
- [ ] Create a Qubes .rpm package and install it system-wide.
- [ ] Ensure that the Dangerzone application appears in the "Applications"
tab.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below) and make sure
they spawn disposable qubes.
### Scenarios
#### 1. Dangerzone correctly identifies that Docker/Podman is not installed
_(Only for MacOS / Windows)_
Temporarily hide the Docker/Podman binaries, e.g., rename the `docker` /
`podman` binaries to something else. Then run Dangerzone. Dangerzone should
prompt the user to install Docker/Podman.
#### 2. Dangerzone correctly identifies that Docker is not running
_(Only for MacOS / Windows)_
Stop the Docker Desktop application. Then run Dangerzone. Dangerzone should
prompt the user to start Docker Desktop.
#### 3. Updating Dangerzone handles external state correctly.
_(Applies to Windows/MacOS)_
Install the previous version of Dangerzone, downloaded from the website.
Open the Dangerzone application and enable some non-default settings.
**If there are new settings, make sure to change those as well**.
Close the Dangerzone application and get the container image for that
version. For example:
```
$ docker images dangerzone.rocks/dangerzone:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
dangerzone.rocks/dangerzone latest <image ID> <date> <size>
```
Then run the version under QA and ensure that the settings remain changed.
Afterwards check that new docker image was installed by running the same command
and seeing the following differences:
```
$ docker images dangerzone.rocks/dangerzone:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
dangerzone.rocks/dangerzone latest <different ID> <newer date> <different size>
```
#### 4. Dangerzone successfully installs the container image
_(Only for Linux)_
Remove the Dangerzone container image from Docker/Podman. Then run Dangerzone.
Dangerzone should install the container image successfully.
#### 5. Dangerzone retains the settings of previous runs
Run Dangerzone and make some changes in the settings (e.g., change the OCR
language, toggle whether to open the document after conversion, etc.). Restart
Dangerzone. Dangerzone should show the settings that the user chose.
#### 6. Dangerzone reports failed conversions
Run Dangerzone and convert the `tests/test_docs/sample_bad_pdf.pdf` document.
Dangerzone should fail gracefully, by reporting that the operation failed, and
showing the following error message:
> The document format is not supported
#### 7. Dangerzone succeeds in converting multiple documents
Run Dangerzone against a list of documents, and tick all options. Ensure that:
* Conversions take place sequentially.
* Attempting to close the window while converting asks the user if they want to
abort the conversions.
* Conversions are completed successfully.
* Conversions show individual progress in real-time (double-check for Qubes).
* _(Only for Linux)_ The resulting files open with the PDF viewer of our choice.
* OCR seems to have detected characters in the PDF files.
* The resulting files have been saved with the proper suffix, in the proper
location.
* The original files have been saved in the `unsafe/` directory.
#### 8. Dangerzone is able to handle drag-n-drop
Run Dangerzone against a set of documents that you drag-n-drop. Files should be
added and conversion should run without issue.
> [!TIP]
> On our end-user container environments for Linux, we can start a file manager
> with `thunar &`.
#### 9. Dangerzone CLI succeeds in converting multiple documents
_(Only for Windows and Linux)_
Run Dangerzone CLI against a list of documents. Ensure that conversions happen
sequentially, are completed successfully, and we see their progress.
#### 10. Dangerzone can open a document for conversion via right-click -> "Open With"
_(Only for Windows, MacOS and Qubes)_
Go to a directory with office documents, right-click on one, and click on "Open
With". We should be able to open the file with Dangerzone, and then convert it.
#### 11. Dangerzone shows helpful errors for setup issues on Qubes
_(Only for Qubes)_
Check what errors does Dangerzone throw in the following scenarios. The errors
should point the user to the Qubes notifications in the top-right corner:
1. The `dz-dvm` template does not exist. We can trigger this scenario by
temporarily renaming this template.
2. The Dangerzone RPC policy does not exist. We can trigger this scenario by
temporarily renaming the `dz.Convert` policy.
3. The `dz-dvm` disposable Qube cannot start due to insufficient resources. We
can trigger this scenario by temporarily increasing the minimum required RAM
of the `dz-dvm` template to more than the available amount.
## Release ## Release
Once we are confident that the release will be out shortly, and doesn't need any more changes: Once we are confident that the release will be out shortly, and doesn't need any more changes:
- [ ] Create a PGP-signed git tag for the version, e.g., for dangerzone `v0.1.0`: - [ ] Create a PGP-signed git tag for the version, e.g., for dangerzone `v0.1.0`:
``` ```bash
git tag -s v0.1.0 git tag -s v0.1.0
git push origin v0.1.0 git push origin v0.1.0
``` ```
@ -263,6 +87,8 @@ Once we are confident that the release will be out shortly, and doesn't need any
### macOS Release ### macOS Release
This needs to happen for both Silicon and Intel chipsets.
#### Initial Setup #### Initial Setup
- Build machine must have: - Build machine must have:
@ -277,48 +103,87 @@ Once we are confident that the release will be out shortly, and doesn't need any
#### Releasing and Signing #### Releasing and Signing
Here is what you need to do:
- [ ] Verify and install the latest supported Python version from - [ ] Verify and install the latest supported Python version from
[python.org](https://www.python.org/downloads/macos/) (do not use the one from [python.org](https://www.python.org/downloads/macos/) (do not use the one from
brew as it is known to [cause issues](https://github.com/freedomofpress/dangerzone/issues/471)) brew as it is known to [cause issues](https://github.com/freedomofpress/dangerzone/issues/471))
* In case of a new Python installation or minor version upgrade, e.g., from
3.11 to 3.12 , reinstall Poetry with `python3 -m pip install poetry` - [ ] Checkout the dependencies, and clean your local copy:
* You can verify the correct Python version is used with `poetry debug info`
- [ ] Verify and checkout the git tag for this release ```bash
- [ ] Run `poetry install --sync`
- [ ] On the silicon mac, build the container image: # In case of a new Python installation or minor version upgrade, e.g., from
# 3.11 to 3.12, reinstall Poetry
python3 -m pip install poetry
# You can verify the correct Python version is used
poetry debug info
# Replace with the actual version
export DZ_VERSION=$(cat share/version.txt)
# Verify and checkout the git tag for this release:
git checkout v$VERSION
# Clean the git repository
git clean -df
# Clean up the environment
poetry env remove --all
# Install the dependencies
poetry install --sync
``` ```
python3 ./install/common/build-image.py
- [ ] Only on Silicon Mac, build the container image and the OCR language data
```bash
# ONLY ON SILICON MAC,
# It should already be built on the Intel one.
poetry run ./install/common/build-image.py
poetry run ./install/common/download-tessdata.py
# Copy the container image to the assets folder
cp share/container.tar.gz ~dz/release-assets/$VERSION/dangerzone-$VERSION-arm64.tar.gz
cp share/image-id.txt ~dz/release-assets/$VERSION/.
``` ```
Then copy the `share/container.tar.gz` to the assets folder on `dangerzone-$VERSION-arm64.tar.gz`, along with the `share/image-id.txt` file.
- [ ] Run `poetry run ./install/macos/build-app.py`; this will make `dist/Dangerzone.app` - [ ] Build the app bundle
```bash
poetry run ./install/macos/build-app.py
```
- [ ] Make sure that the build application works with the containerd graph - [ ] Make sure that the build application works with the containerd graph
driver (see [#933](https://github.com/freedomofpress/dangerzone/issues/933)) driver (see [#933](https://github.com/freedomofpress/dangerzone/issues/933))
- [ ] Run `poetry run ./install/macos/build-app.py --only-codesign`; this will make `dist/Dangerzone.dmg` - [ ] Sign the application bundle, and notarize it
* You need to run this command as the account that has access to the code signing certificate
* You must run this command from the MacOS UI, from a terminal application. You need to run this command as the account that has access to the code signing certificate
- [ ] Notarize it: `xcrun notarytool submit --wait --apple-id "<email>" --keychain-profile "dz-notarytool-release-key" dist/Dangerzone.dmg`
* You need to change the `<email>` in the above command with the email This command assumes that you have created, and stored in the Keychain, an
associated with the Apple Developer ID. application password associated with your Apple Developer ID, which will be
* This command assumes that you have created, and stored in the Keychain, an used specifically for `notarytool`.
application password associated with your Apple Developer ID, which will be
used specifically for `notarytool`.
- [ ] Wait for it to get approved:
* If it gets rejected, you should be able to see why with the same command
(or use the `log` option for a more verbose JSON output)
* You will also receive an update in your email.
- [ ] After it's approved, staple the ticket: `xcrun stapler staple dist/Dangerzone.dmg`
This process ends up with the final file: ```bash
# Sign the .App and make it a .dmg
poetry run ./install/macos/build-app.py --only-codesign
``` # Notarize it. You must run this command from the MacOS UI
dist/Dangerzone.dmg # from a terminal application.
``` xcrun notarytool submit ./dist/Dangerzone.dmg --apple-id $APPLE_ID --keychain-profile "dz-notarytool-release-key" --wait && xcrun stapler staple dist/Dangerzone.dmg
Rename `Dangerzone.dmg` to `Dangerzone-$VERSION.dmg`. # Copy the .dmg to the assets folder
ARCH=$(uname -m)
if [ "$ARCH" = "x86_64" ]; then
ARCH="i686"
fi
cp dist/Dangerzone.dmg ~dz/release-assets/$VERSION/Dangerzone-$VERSION-$ARCH.dmg
```
### Windows Release ### Windows Release
The Windows release is performed in a Windows 11 virtual machine as opposed to a physical one. The Windows release is performed in a Windows 11 virtual machine (as opposed to a physical one).
#### Initial Setup #### Initial Setup
@ -332,8 +197,30 @@ The Windows release is performed in a Windows 11 virtual machine as opposed to a
#### Releasing and Signing #### Releasing and Signing
- [ ] Verify and checkout the git tag for this release ```bash
- [ ] Run `poetry install --sync` # In case of a new Python installation or minor version upgrade, e.g., from
# 3.11 to 3.12, reinstall Poetry
python3 -m pip install poetry
# You can verify the correct Python version is used
poetry debug info
# Replace with the actual version
export DZ_VERSION=$(cat share/version.txt)
# Verify and checkout the git tag for this release:
git checkout v$VERSION
# Clean the git repository
git clean -df
# Clean up the environment
poetry env remove --all
# Install the dependencies
poetry install --sync
```
- [ ] Copy the container image into the VM - [ ] Copy the container image into the VM
> [!IMPORTANT] > [!IMPORTANT]
> Instead of running `python .\install\windows\build-image.py` in the VM, run the build image script on the host (making sure to build for `linux/amd64`). Copy `share/container.tar.gz` and `share/image-id.txt` from the host into the `share` folder in the VM. > Instead of running `python .\install\windows\build-image.py` in the VM, run the build image script on the host (making sure to build for `linux/amd64`). Copy `share/container.tar.gz` and `share/image-id.txt` from the host into the `share` folder in the VM.
@ -365,21 +252,15 @@ instructions in our build section](https://github.com/freedomofpress/dangerzone/
or create your own locally with: or create your own locally with:
```sh ```sh
# Create and run debian bookworm development environment
./dev_scripts/env.py --distro debian --version bookworm build-dev ./dev_scripts/env.py --distro debian --version bookworm build-dev
./dev_scripts/env.py --distro debian --version bookworm run --dev bash ./dev_scripts/env.py --distro debian --version bookworm run --dev bash
cd dangerzone
```
Build the latest container: # Build the latest container
./dev_scripts/env.py --distro debian --version bookworm run --dev bash -c "cd dangerzone && poetry run ./install/common/build-image.py"
```sh # Create a .deb
python3 ./install/common/build-image.py ./dev_scripts/env.py --distro debian --version bookworm run --dev bash -c "cd dangerzone && ./install/linux/build-deb.py"
```
Create a .deb:
```sh
./install/linux/build-deb.py
``` ```
Publish the .deb under `./deb_dist` to the Publish the .deb under `./deb_dist` to the
@ -398,22 +279,12 @@ or create your own locally with:
```sh ```sh
./dev_scripts/env.py --distro fedora --version 41 build-dev ./dev_scripts/env.py --distro fedora --version 41 build-dev
./dev_scripts/env.py --distro fedora --version 41 run --dev bash
cd dangerzone
```
Build the latest container: # Build the latest container (skip if already built):
./dev_scripts/env.py --distro fedora --version 41 run --dev bash -c "cd dangerzone && poetry run ./install/common/build-image.py"
```sh # Create a .rpm:
python3 ./install/common/build-image.py ./dev_scripts/env.py --distro fedora --version 41 run --dev bash -c "cd dangerzone && ./install/linux/build-rpm.py"
```
Copy the container image to the assets folder on `dangerzone-$VERSION-i686.tar.gz`.
Create a .rpm:
```sh
./install/linux/build-rpm.py
``` ```
Publish the .rpm under `./dist` to the Publish the .rpm under `./dist` to the
@ -424,7 +295,7 @@ Publish the .rpm under `./dist` to the
Create a .rpm for Qubes: Create a .rpm for Qubes:
```sh ```sh
./install/linux/build-rpm.py --qubes ./dev_scripts/env.py --distro fedora --version 41 run --dev bash -c "cd dangerzone && ./install/linux/build-rpm.py --qubes"
``` ```
and similarly publish it to the [`freedomofpress/yum-tools-prod`](https://github.com/freedomofpress/yum-tools-prod) and similarly publish it to the [`freedomofpress/yum-tools-prod`](https://github.com/freedomofpress/yum-tools-prod)
@ -432,36 +303,37 @@ repo.
## Publishing the Release ## Publishing the Release
To publish the release: To publish the release, you can follow these steps:
- [ ] Create an archive of the Dangerzone source in `tar.gz` format: - [ ] Create an archive of the Dangerzone source in `tar.gz` format:
* You can use the following command: ```bash
export VERSION=$(cat share/version.txt)
``` git archive --format=tar.gz -o dangerzone-${VERSION:?}.tar.gz --prefix=dangerzone/ v${VERSION:?}
export DZ_VERSION=$(cat share/version.txt) ```
git archive --format=tar.gz -o dangerzone-${DZ_VERSION:?}.tar.gz --prefix=dangerzone/ v${DZ_VERSION:?}
```
- [ ] Run container scan on the produced container images (some time may have passed since the artifacts were built) - [ ] Run container scan on the produced container images (some time may have passed since the artifacts were built)
``` ```bash
gunzip --keep -c ./share/container.tar.gz > /tmp/container.tar gunzip --keep -c ./share/container.tar.gz > /tmp/container.tar
docker pull anchore/grype:latest docker pull anchore/grype:latest
docker run --rm -v /tmp/container.tar:/container.tar anchore/grype:latest /container.tar docker run --rm -v /tmp/container.tar:/container.tar anchore/grype:latest /container.tar
``` ```
- [ ] Collect the assets in a single directory, calculate their SHA-256 hashes, and sign them. - [ ] Collect the assets in a single directory, calculate their SHA-256 hashes, and sign them.
* You can use `./dev_scripts/sign-assets.py`, if you want to automate this There is an `./dev_scripts/sign-assets.py` script to automate this task.
task.
- [ ] Create a new **draft** release on GitHub and upload the macOS and Windows installers.
* Copy the release notes text from the template at [`docs/templates/release-notes`](https://github.com/freedomofpress/dangerzone/tree/main/docs/templates/)
* You can use `./dev_scripts/upload-asset.py`, if you want to upload an asset
using an access token.
- [ ] Upload the `container-$VERSION-i686.tar.gz` and `container-$VERSION-arm64.tar.gz` images that were created in the previous step
**Important:** Make sure that it's the same container images as the ones that **Important:** Before running the script, make sure that it's the same container images as
are shipped in other platforms (see our [Pre-release](#Pre-release) section) the ones that are shipped in other platforms (see our [Pre-release](#Pre-release) section)
```bash
# Sign all the assets
./dev_scripts/sign-assets.py ~/release-assets/$VERSION/github --version $VERSION
```
- [ ] Upload all the assets to the draft release on GitHub.
```bash
find ~/release-assets/$VERSION/github | xargs -n1 ./dev_scripts/upload-asset.py --token ~/token --draft
```
- [ ] Upload the detached signatures (.asc) and checksum file.
- [ ] Update the [Dangerzone website](https://github.com/freedomofpress/dangerzone.rocks) to link to the new installers. - [ ] Update the [Dangerzone website](https://github.com/freedomofpress/dangerzone.rocks) to link to the new installers.
- [ ] Update the brew cask release of Dangerzone with a [PR like this one](https://github.com/Homebrew/homebrew-cask/pull/116319) - [ ] Update the brew cask release of Dangerzone with a [PR like this one](https://github.com/Homebrew/homebrew-cask/pull/116319)
- [ ] Update version and download links in `README.md` - [ ] Update version and download links in `README.md`

60
dev_scripts/generate-release-tasks.py Executable file
View file

@ -0,0 +1,60 @@
#!/usr/bin/env python3
import pathlib
import subprocess
RELEASE_FILE = "RELEASE.md"
QA_FILE = "QA.md"
def git_root():
"""Get the root directory of the Git repo."""
# FIXME: Use a Git Python binding for this.
# FIXME: Make this work if called outside the repo.
path = subprocess.run(
["git", "rev-parse", "--show-toplevel"],
check=True,
stdout=subprocess.PIPE,
).stdout.decode().strip("\n")
return pathlib.Path(path)
def extract_checkboxes(filename):
headers = []
result = []
with open(filename, 'r') as f:
lines = f.readlines()
current_level = 0
for line in lines:
line = line.rstrip()
# If it's a header, store it
if line.startswith('#'):
# Count number of # to determine header level
level = len(line) - len(line.lstrip('#'))
if level < current_level or not current_level:
headers.extend(['', line, ''])
current_level = level
elif level > current_level:
continue
else:
headers = ['', line, '']
# If it's a checkbox
elif '- [ ]' in line or '- [x]' in line or '- [X]' in line:
# Print the last header if we haven't already
if headers:
result.extend(headers)
headers = []
current_level = 0
# If this is the "Do the QA tasks" line, recursively get QA tasks
if "Do the QA tasks" in line:
result.append(line)
qa_tasks = extract_checkboxes(git_root() / QA_FILE)
result.append(qa_tasks)
else:
result.append(line)
return '\n'.join(result)
if __name__ == '__main__':
print(extract_checkboxes(git_root() / RELEASE_FILE))

65
dev_scripts/qa.py
View file

@ -3,28 +3,49 @@
import abc import abc
import argparse import argparse
import difflib import difflib
import json
import logging import logging
import re import re
import selectors import selectors
import subprocess import subprocess
import sys import sys
import urllib.request
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
PYTHON_VERSION_STR = "3.12"
PYTHON_VERSION = [int(num) for num in PYTHON_VERSION_STR.split(".")]
EOL_PYTHON_URL = "https://endoflife.date/api/python.json"
CONTENT_QA = r"""## QA CONTENT_QA = r"""## QA
To ensure that new releases do not introduce regressions, and support existing To ensure that new releases do not introduce regressions, and support existing
and newer platforms, we have to do the following: and newer platforms, we have to test that the produced packages work as expected.
Check the following:
- [ ] Make sure that the tip of the `main` branch passes the CI tests. - [ ] Make sure that the tip of the `main` branch passes the CI tests.
- [ ] Make sure that the Apple account has a valid application password and has - [ ] Make sure that the Apple account has a valid application password and has
agreed to the latest Apple terms (see [macOS release](#macos-release) agreed to the latest Apple terms (see [macOS release](#macos-release)
section). section).
Because it is repetitive, we wrote a script to help with the QA.
It can run the tasks for you, pausing when it needs manual intervention.
You can run it with a command like:
```bash
poetry run ./dev_scripts/qa.py {distro}-{version}
```
### The checklist
- [ ] Create a test build in Windows and make sure it works: - [ ] Create a test build in Windows and make sure it works:
- [ ] Check if the suggested Python version is still supported. - [ ] Check if the suggested Python version is still supported.
- [ ] Create a new development environment with Poetry. - [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses - [ ] Build the container image and ensure the development environment uses
the new image. the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests. - [ ] Run the Dangerzone tests.
- [ ] Build and run the Dangerzone .exe - [ ] Build and run the Dangerzone .exe
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below). - [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
@ -33,6 +54,7 @@ and newer platforms, we have to do the following:
- [ ] Create a new development environment with Poetry. - [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses - [ ] Build the container image and ensure the development environment uses
the new image. the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests. - [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle. - [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below). - [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
@ -41,6 +63,7 @@ and newer platforms, we have to do the following:
- [ ] Create a new development environment with Poetry. - [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses - [ ] Build the container image and ensure the development environment uses
the new image. the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests. - [ ] Run the Dangerzone tests.
- [ ] Create and run an app bundle. - [ ] Create and run an app bundle.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below). - [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
@ -49,6 +72,7 @@ and newer platforms, we have to do the following:
- [ ] Create a new development environment with Poetry. - [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses - [ ] Build the container image and ensure the development environment uses
the new image. the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests. - [ ] Run the Dangerzone tests.
- [ ] Create a .deb package and install it system-wide. - [ ] Create a .deb package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below). - [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
@ -57,6 +81,7 @@ and newer platforms, we have to do the following:
- [ ] Create a new development environment with Poetry. - [ ] Create a new development environment with Poetry.
- [ ] Build the container image and ensure the development environment uses - [ ] Build the container image and ensure the development environment uses
the new image. the new image.
- [ ] Download the OCR language data using `./install/common/download-tessdata.py`
- [ ] Run the Dangerzone tests. - [ ] Run the Dangerzone tests.
- [ ] Create an .rpm package and install it system-wide. - [ ] Create an .rpm package and install it system-wide.
- [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below). - [ ] Test some QA scenarios (see [Scenarios](#Scenarios) below).
@ -547,7 +572,7 @@ class Reference:
# Convert spaces to dashes # Convert spaces to dashes
anchor = anchor.replace(" ", "-") anchor = anchor.replace(" ", "-")
# Remove non-alphanumeric (except dash and underscore) # Remove non-alphanumeric (except dash and underscore)
anchor = re.sub("[^a-zA-Z\-_]", "", anchor) anchor = re.sub("[^a-zA-Z-_]", "", anchor)
return anchor return anchor
@ -566,8 +591,8 @@ class QABase(abc.ABC):
platforms = {} platforms = {}
REF_QA = Reference("RELEASE.md", content=CONTENT_QA) REF_QA = Reference("QA.md", content=CONTENT_QA)
REF_QA_SCENARIOS = Reference("RELEASE.md", content=CONTENT_QA_SCENARIOS) REF_QA_SCENARIOS = Reference("QA.md", content=CONTENT_QA_SCENARIOS)
# The following class method is available since Python 3.6. For more details, see: # The following class method is available since Python 3.6. For more details, see:
# https://docs.python.org/3.6/whatsnew/3.6.html#pep-487-simpler-customization-of-class-creation # https://docs.python.org/3.6/whatsnew/3.6.html#pep-487-simpler-customization-of-class-creation
@ -802,6 +827,26 @@ class QAWindows(QABase):
while msvcrt.kbhit(): while msvcrt.kbhit():
msvcrt.getch() msvcrt.getch()
@QABase.task(f"Install the latest version of Python {PYTHON_VERSION_STR}", ref=REF_BUILD)
def install_python(self):
cur_version = list(sys.version_info[:3])
logger.info("Getting latest Python release")
with urllib.request.urlopen(EOL_PYTHON_URL) as f:
resp = f.read()
releases = json.loads(resp)
for release in releases:
if release["cycle"] == PYTHON_VERSION_STR:
latest_version = [int(num) for num in release["latest"].split(".")]
if latest_version > cur_version:
self.prompt(f"You need to install the latest Python version ({release['latest']})")
elif latest_version == cur_version:
logger.info(f"Verified that the latest Python version ({release['latest']}) is installed")
return
logger.error("Could not verify that the latest Python version is installed")
@QABase.task("Install and Run Docker Desktop", ref=REF_BUILD) @QABase.task("Install and Run Docker Desktop", ref=REF_BUILD)
def install_docker(self): def install_docker(self):
logger.info("Checking if Docker Desktop is installed and running") logger.info("Checking if Docker Desktop is installed and running")
@ -816,12 +861,16 @@ class QAWindows(QABase):
) )
def install_poetry(self): def install_poetry(self):
self.run("python", "-m", "pip", "install", "poetry") self.run("python", "-m", "pip", "install", "poetry")
self.run("poetry", "install") self.run("poetry", "install", "--sync")
@QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True) @QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True)
def build_image(self): def build_image(self):
self.run("python", r".\install\common\build-image.py") self.run("python", r".\install\common\build-image.py")
@QABase.task("Download Tesseract data", ref=REF_BUILD, auto=True)
def download_tessdata(self):
self.run("python", r".\install\common\download-tessdata.py")
@QABase.task("Run tests", ref="REF_BUILD", auto=True) @QABase.task("Run tests", ref="REF_BUILD", auto=True)
def run_tests(self): def run_tests(self):
# NOTE: Windows does not have Makefile by default. # NOTE: Windows does not have Makefile by default.
@ -838,9 +887,11 @@ class QAWindows(QABase):
return "windows" return "windows"
def start(self): def start(self):
self.install_python()
self.install_docker() self.install_docker()
self.install_poetry() self.install_poetry()
self.build_image() self.build_image()
self.download_tessdata()
self.run_tests() self.run_tests()
self.build_dangerzone_exe() self.build_dangerzone_exe()
@ -1005,6 +1056,10 @@ class QAFedora(QALinux):
) )
class QAFedora41(QAFedora):
VERSION = "41"
class QAFedora40(QAFedora): class QAFedora40(QAFedora):
VERSION = "40" VERSION = "40"