There are times where we may want to build the container image for
testing, but compression takes too much time. If we don't plan to use
this image for production builds, we can specify instead a compression
level that is so low, that the image will be compressed instantly.
In this commit, we allow the user to specify the Gzip compression level,
and even set it to 0. The default will always be 9, so that we don't
make a mistake during release.
The file container-pip-dependencies.txt was being left a directory when
building the docker image. This meant that it was being packaged when it
wasn't supposed to.
To avoid this, we remove file with the help from a context manager.
The change is minimal and the biggest part of the diff are indentation
changes.
Fixes#739
Avoids downloading the container image 4 times in the multi-stage build
by first pulling the alpine image once and then building without any
pulls.
Implemented following a suggestion of @apyrgio.
Make Poetry include data files only in the source distribution, and not
on our wheels. This mainly makes RPM packaging a bit easier, but does
not solve the problem of how to install files to
`/usr/share/dangerzone`.
Also, include files using globs, which is the way Poetry prefers.
Fixes#678
Refs #677
Add the following functionality to the build image script:
1. Let the user choose the container runtime of their choice. In some
systems, both Docker and Podman may be available, so we need to let
the user choose which runtime they want.
2. Let users choose if they want to save the image. For non-production
builds, we may want to simply build the container image, without
the time penalty of compression.
Ensure that when the container image is installing pymupdf (unavailable
in the repos) with verified hashes. To do so, it has the pymupdf
dependency declared in a "container" group in `pyproject.toml`, which
then gets exported into a requirements.txt, which is then used for
hash-verification when building the container.
Because this required modifying the container image build scripts, they
were all merged to avoid duplicate code. This was an overdue change
anyways.
Adding PyMuPDF essentially make the code much simpler since it can do
everything that we'd need multiple programs for. It also includes
tesseract-OCR integration, which this commit makes use of.
Dangerzone was failing to convert documents in Qubes due to missing
client-side dependencies. In particular poppler-utils, ghostscript and
graphicsmagick.
Fixes#647
This commit fixes 3 small issues with the way we produce our Qubes RPM:
1. The `.exists()` method follows symlinks by default, whereas we want
to check if a symlink exists. This functionality has been added in
Python
3.12.
Instead of checking if a symlink exists and then removing it, simply
remove it and don't throw an error if it doesn't exist in the first
place.
2. The `dz.Convert*` policies were not installed with the executable bit
set, therefore the qube could not start.
3. The `dz.ConvertDev` policy in particular had an ambiguous shebang,
thus we change it to explicitly call Python3
Add Tesseract models for the 10 most spoken languages as package
requirements for Qubes. For containers, this problem is already solved
since we install all Tesseract models.
If a user is not covered by the installed models, they can install
extras on their own. We will add a note for this in subsequent commits.
Refs #431
Replace the deprecated `bdist_rpm` method of creating RPMs for
Dangerzone. Instead, update our `install/linux/build-rpm.py` script, to
build Dangerzone RPMs using our SPEC file under
`install/linux/dangerzone.spec`. The script now essentially creates a
source distribution (sdist) using `poetry build`, and then uses
`rpmbuild` to create binary and source RPMs.
Fixes#298
Add an `rpm-build` directory under `install/linux`, which will be used
for building Dangerzone RPMs. For the time being, it only has a
.gitignore file there, but in the future, invoking
`install/linux/build-rpm.py` will populate it.
Introduce a SPEC file that can be used to create an RPM from a Python
source distribution. Some notable features of this SPEC file follow:
1. We can use this SPEC file to create both regular RPM packages and
ones targeted for Qubes.
2. It has a post installation script that removes stale .egg-info
directories, which previously caused issues to our users.
3. It automatically creates a changelog from our Git logs, which differs
from the actual CHANGELOG.md.
4. It folloes the latest Fedora guidelines (as of writing this) for
packaging Python projects.
Fixes#514
Always pull the base container image (alpine:latest) before building our
own container image. Else, in an environments that we haven't touched
for a while, an older image may be used.
Use the MIME types actually used by the `file` command, which was
recently changed for the detection of the HWPX format [1].
application/hwp+zip -> application/x-hwp+zip
But the HWPX format includes a 'mimetype' file, which contains the
MIME type string "application/hwp+zip", so that was left so because
it may be possible to detect it as "application/hwp+zip".
[1]: ceef7ead3a
Add a new Python module called "updater", which contains the logic for
prompting the user to enable updates, and checking our GitHub releases
for new updates.
This class has some light dependency to Qt functionality, since it needs
to:
* Show a prompt to the user,
* Run update checks asynchronously in a Qt thread,
* Provide the main window with the result of the update check
Refs #189
Python 3.10.12 fixes some CVEs for which Dangerzone does not appear to be
affected, however its binaries are not made available by the python
foundation. Moving to 3.11 should be trivial since this was already
deployed in Fedora 37+.
Allow creating an RPM package that is to be installed specifically on
Qubes. This package has the following extra properties from our regular
RPM packages:
1. Make `python3-magic`, `libreoffice` and `tesseract` requirements
for installing Dangerzone, since the conversion takes place in a
disposable qube that needs these packages.
2. Ignore the container.tar.gz file, if it exists.
3. Add our RPC calls under `/etc/qubes-rpc`
The files in `container/` no longer make sense to have that name since
the "document to pixels" part will run in Qubes OS in its own virtual
machine.
To adapt to this, this PR does the following:
- Moves all the files in `container` to `dangerzone/conversion`
- Splits the old `container/dangerzone.py` into its two components
`dangerzone/conversion/{doc_to_pixels,pixels_to_pdf}.py` with a
`common.py` file for shared functions
- Moves the Dockerfile to the project root and adapts it to the new
container code location
- Updates the CircleCI config to properly cache Docker images.
- Updates our install scripts to properly build Docker images.
- Adds the new conversion module to the container image, so that it can
be imported as a package.
- Adapts the container isolation provider to use the new way of calling
the code.
NOTE: We have made zero changes to the conversion code in this commit,
except for necessary imports in order to factor out some common parts.
Any changes necessary for Qubes integration follow in the subsequent
commits.
Replace our reference to an Apple development certificate with a
Developer ID Application certificate. The former is not accepted during
the code notarization phase, whereas the latter is.
Remove any -rc identifiers (e.g., 0.4.1-rc3) from the Dangerzone
version, if it includes them. If we don't remove them, then building
the MSI for Windows will fail as follows:
error CNDL0108: The Product/@Version attribute's value, '0.4.1-rc3',
is not a valid version. Legal version values should look like
'x.x.x.x' where x is an integer from 0 to 65534.
Update several references to First Look Media in the code, to better
reflect the current status, where Freedom of the Press Foundation has
taken over the stewardship of the project.
Fixes#343
Use the full image tag (dangerzone.rocks/dangerzone:latest) when
building the image. Else, we risk creating a `share/image-id.txt` file
with multiple IDs in it, if we have another
`dangerzone.rocks/dangerzone` image (with a different tag) in our dev
environment.
Building the `.msi` on Windows was failing in the `candle.exe` step due
to some files in the PySide6 library being too long (PySide6/examples)
or having illegal character (`+`) in their file names
(PySide6/qml/QtQuick).
Skipping copying these files to the `.msi` fixes the issue. Skipping
`examples/` should be of no impact since they're just examples and
skipping `qml/QtQuick` shouldn't cause issues because we don't use QML.
Reverts commit `bbbf822` and adapts it from PySide2 to PySide6.
Add an optional --distro argument to build-deb.py, to specify the Debian
version in the package name, which currently is "1". This option may
prove useful when publishing packages to freedomofpress/apt-tools-prod,
where packages from different distros with the same names but different
contents are not accepted.
Skip the creation of the `share/container.tar` file, since it's not used
anywhere. Instead, pipe our `docker/podman save` invocations to `gzip`
directly, which will compress the tarfile on the fly. This saves both
time and disk space.
Introduce a script for installing Podman in Ubuntu Focal, in
environments that may, or may not, have sudo installed.
Also, update our CircleCI configuration to use this script when
installing Podman.
