Commit graph

1530 commits

Author SHA1 Message Date
Alex Pyrgiotis
2b45c5cfa0
Merge cbb7ed902f into c407e2ff84 2025-01-20 16:46:01 +01:00
Alex Pyrgiotis
cbb7ed902f
FIXUP: Use semantic diff for diffoci 2025-01-20 17:45:55 +02:00
Alex Pyrgiotis
eacf1eb2fa
grype: Add Debian CVEs to ignore list
Add some CVEs in our ignore list, which are present in the new Debian
image. These CVEs are marked as "wont-fix" by the Debian Security team.
2025-01-20 17:05:48 +02:00
Alex Pyrgiotis
505db39ca0
fixup! FIXUP: Copy all the Python files from the conversion/ dir 2025-01-20 15:40:14 +02:00
Alex Pyrgiotis
0f0fa49923
FIXUP: determine_tag -> determine_git_tag 2025-01-20 15:27:32 +02:00
Alex Pyrgiotis
8911b72529
fixup! FIXUP: Handle diffoci updates appropriately 2025-01-20 15:26:07 +02:00
Alexis Métaireau
c407e2ff84
doc: update Debian Trixie installation instructions
Some checks are pending
Tests / windows (push) Blocked by required conditions
Tests / macOS (arch64) (push) Blocked by required conditions
Tests / macOS (x86_64) (push) Blocked by required conditions
Tests / build-deb (debian bookworm) (push) Blocked by required conditions
Tests / build-deb (debian bullseye) (push) Blocked by required conditions
Tests / build-deb (debian trixie) (push) Blocked by required conditions
Tests / build-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / install-deb (debian bookworm) (push) Blocked by required conditions
Tests / install-deb (debian bullseye) (push) Blocked by required conditions
Tests / install-deb (debian trixie) (push) Blocked by required conditions
Tests / install-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 40) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 41) (push) Blocked by required conditions
Tests / run tests (debian bookworm) (push) Blocked by required conditions
Tests / run tests (debian bullseye) (push) Blocked by required conditions
Tests / run tests (debian trixie) (push) Blocked by required conditions
Tests / run tests (fedora 40) (push) Blocked by required conditions
Tests / run tests (fedora 41) (push) Blocked by required conditions
Tests / run tests (ubuntu 20.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 22.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.10) (push) Blocked by required conditions
Scan latest app and container / security-scan-container (push) Waiting to run
Scan latest app and container / security-scan-app (push) Waiting to run
Starting with Debian Trixie, `apt secure` relies on `sqv` to do its verification, which doesn't support the GPG keybox database format.

At the same time, using the standard PGP base64 format makes the verification fail for versions of `apt secure` which relies on `gpg`, as the subkey isn't detected there.

Fixes #1055
2025-01-20 14:10:15 +01:00
Alex Pyrgiotis
725ce3b9c7
FIXUP: Add easier method to generate Dockerfile 2025-01-20 15:02:04 +02:00
Alex Pyrgiotis
afc5e8e636
FIXUP: Bump Dockerfile envs 2025-01-20 14:59:42 +02:00
Alex Pyrgiotis
5bb37ef48f
fixup! FIXUP: Rename dangerzone/container to dangerzone/container_helpers 2025-01-20 14:20:44 +02:00
Alex Pyrgiotis
c70d1970dd
FIXUP: Remove unnecessary needs 2025-01-20 14:20:44 +02:00
Alex Pyrgiotis
ec616be2c0
Rename vendor-pymupdf.py to debian-vendor-pymupdf.py
Rename the `vendor-pymupdf.py` script to `debian-vendor-pymupdf.py`,
since it's used only when building Debian packages.
2025-01-20 12:37:02 +02:00
Alex Pyrgiotis
acbc433717
FIXUP: Keep only the necessary instructions for checking reproducibility 2025-01-20 12:35:32 +02:00
Alex Pyrgiotis
685cf431a3
FIXUP: Handle diffoci updates appropriately 2025-01-20 12:28:34 +02:00
Alex Pyrgiotis
2b71e615a8
FIXUP: Add indication of faulty file command 2025-01-20 12:23:56 +02:00
Alex Pyrgiotis
02e63e5a49
FIXUP: Rename dangerzone/container to dangerzone/container_helpers 2025-01-16 19:00:04 +02:00
Alex Pyrgiotis
9daf30154b
FIXUP: Copy all the Python files from the conversion/ dir 2025-01-16 18:56:34 +02:00
Alexis Métaireau
7f418118e6
CI: Drop Fedora 39 from the CI checks
Some checks failed
Scan latest app and container / security-scan-container (push) Has been cancelled
Tests / windows (push) Has been cancelled
Tests / macOS (arch64) (push) Has been cancelled
Tests / build-deb (ubuntu 22.04) (push) Has been cancelled
Tests / macOS (x86_64) (push) Has been cancelled
Tests / build-deb (debian bookworm) (push) Has been cancelled
Scan latest app and container / security-scan-app (push) Has been cancelled
Tests / install-deb (ubuntu 22.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.10) (push) Has been cancelled
Tests / build-deb (debian bullseye) (push) Has been cancelled
Tests / build-deb (debian trixie) (push) Has been cancelled
Tests / build-deb (ubuntu 20.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.10) (push) Has been cancelled
Tests / install-deb (debian bookworm) (push) Has been cancelled
Tests / install-deb (debian bullseye) (push) Has been cancelled
Tests / install-deb (debian trixie) (push) Has been cancelled
Tests / install-deb (ubuntu 20.04) (push) Has been cancelled
Tests / run tests (fedora 40) (push) Has been cancelled
Tests / run tests (fedora 41) (push) Has been cancelled
Tests / run tests (ubuntu 20.04) (push) Has been cancelled
Tests / run tests (ubuntu 22.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.10) (push) Has been cancelled
Tests / build-install-rpm (fedora 40) (push) Has been cancelled
Tests / build-install-rpm (fedora 41) (push) Has been cancelled
Tests / run tests (debian bookworm) (push) Has been cancelled
Tests / run tests (debian bullseye) (push) Has been cancelled
Tests / run tests (debian trixie) (push) Has been cancelled
2025-01-16 11:51:22 +01:00
Alexis Métaireau
02602b072a
Remove intermediate variables for conversion start/end logs
Some checks are pending
Tests / run tests (ubuntu 22.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.10) (push) Blocked by required conditions
Tests / run-lint (push) Waiting to run
Tests / build-container-image (push) Waiting to run
Tests / Download and cache Tesseract data (push) Waiting to run
Tests / windows (push) Blocked by required conditions
Tests / macOS (arch64) (push) Blocked by required conditions
Tests / macOS (x86_64) (push) Blocked by required conditions
Tests / build-deb (debian bookworm) (push) Blocked by required conditions
Tests / build-deb (debian bullseye) (push) Blocked by required conditions
Tests / build-deb (debian trixie) (push) Blocked by required conditions
Tests / build-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / install-deb (debian bookworm) (push) Blocked by required conditions
Tests / install-deb (debian bullseye) (push) Blocked by required conditions
Tests / install-deb (debian trixie) (push) Blocked by required conditions
Tests / install-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 40) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 41) (push) Blocked by required conditions
Tests / run tests (debian bookworm) (push) Blocked by required conditions
Tests / run tests (debian bullseye) (push) Blocked by required conditions
Tests / run tests (debian trixie) (push) Blocked by required conditions
Scan latest app and container / security-scan-container (push) Waiting to run
Scan latest app and container / security-scan-app (push) Waiting to run
Also, state that the logs are incomplete in the header.
2025-01-16 11:35:07 +01:00
Alexis Métaireau
acf20ef700
Add a --debug flag to the CLI to help retrieve more logs
When the flag is set, the `RUNSC_DEBUG=1` environment variable is added
to the outer container, and stderr is captured in a separate thread, before printing its output.
2025-01-16 11:35:06 +01:00
Alexis Métaireau
3499010d8e
docs(install): store GPG keys in the base64 format 2025-01-15 19:48:00 +01:00
Alexis Métaireau
2423fc18c5
CI: Store the signature key using the base64 format
The GPG binary format used until now doesn't seem to please `sqv` which
is now used by default on debian trixie.

Fixes #1052
2025-01-15 19:39:02 +01:00
Alex Pyrgiotis
92d8a4c556
FIXUP: Improve readability 2025-01-14 23:49:43 +02:00
Alex Pyrgiotis
aa710e84c9
FIXUP: Improve the reproducibility check section 2025-01-14 23:47:16 +02:00
Alex Pyrgiotis
c1f25484ff
FIXUP: Invalidate downloaded diffoci helper if checksum differs 2025-01-14 23:43:53 +02:00
Alex Pyrgiotis
6cf4c5cc46
Update docs/developer/reproducibility.md
Co-authored-by: Alexis Métaireau <alexis@freedom.press>
2025-01-14 23:43:46 +02:00
Alex Pyrgiotis
e77580f845
Update docs/developer/reproducibility.md
Co-authored-by: Alexis Métaireau <alexis@freedom.press>
2025-01-14 23:42:49 +02:00
Alex Pyrgiotis
b8755f26ee
FIXUP: Remove stray comment 2025-01-14 23:14:05 +02:00
Alex Pyrgiotis
f019ce05d6
Update RELEASE.md
Co-authored-by: Alexis Métaireau <alexis@freedom.press>
2025-01-14 15:04:31 +02:00
Alex Pyrgiotis
cbeb103067
FIXUP: Separate some Dockerfile commands 2025-01-14 14:53:31 +02:00
Alex Pyrgiotis
96ab442873
FIXUP: Add links for container params 2025-01-14 14:46:32 +02:00
Alex Pyrgiotis
10b8cd48af
FIXUP: Change name of reproduce CI job 2025-01-14 14:21:39 +02:00
Alex Pyrgiotis
b42bd67f6c
FIXUP: Remove stray podmna load command 2025-01-14 14:20:36 +02:00
Alex Pyrgiotis
45f43964a5
fixup! Do not use poetry.lock when building the container image 2025-01-14 14:19:12 +02:00
Alex Pyrgiotis
e02dbfdc79
WIP: Reproduce 2025-01-14 12:29:09 +02:00
Alex Pyrgiotis
d53c4d06b5
fixup! Render the Dockerfile from a template and some params 2025-01-14 12:07:45 +02:00
Alex Pyrgiotis
279322bf43
ci: Add a CI job that enforces image reproducibility
Add a CI job that uses the `reproduce.py` dev script to enforce image
reproducibility, for every PR that we send to the repo.

Fixes #1047
2025-01-14 11:58:22 +02:00
Alex Pyrgiotis
7a59940493
dev_scripts: Add script for enforcing image reproducibility
Add a dev script for Linux platforms that verifies that a source image
can be reproducibly built from the current Git commit. The
reproducibility check is enforced by the `diffoci` tool, which is
downloaded as part of running the script.
2025-01-14 11:58:22 +02:00
Alex Pyrgiotis
375efe5af4
Allow setting a tag for the container image
Allow setting a tag for the container image, when building it with the
`build-image.py` script. This should be used for development purposes
only, since the proper image name should be dictated by the script.
2025-01-14 11:58:22 +02:00
Alex Pyrgiotis
a8436bba98
Render the Dockerfile from a template and some params
Allow updating the Dockerfile from a template and some envs, so that
it's easier to bump the dates in it.
2025-01-14 11:58:22 +02:00
Alex Pyrgiotis
fccfd510b7
Add jinja2-cli package dependency
Add jinja2-cli as a package dependency, since it will be used to create
the Dockerfile from some user parameters and a template.
2025-01-14 11:58:20 +02:00
Alex Pyrgiotis
1ca3ef9796
Allow using the container engine cache when building our image
Remove our suggestions for not using the container cache, which stemmed
from the fact that our Dangerzone image was not reproducible. Now that
we have switched to Debian Stable and the Dockerfile is all we need to
reproducibly build the exact same container image, we can just use the
cache to speed up builds.
2025-01-14 11:58:08 +02:00
Alex Pyrgiotis
460b7a178b
Do not use poetry.lock when building the container image
Remove all the scaffolding in our `build-image.py` script for using the
`poetry.lock` file, now that we install PyMuPDF from the Debian repos.
2025-01-14 11:58:06 +02:00
Alex Pyrgiotis
42646877d7
Switch base image to Debian Stable
Switch base image from Alpine Linux to Debian Stable, in order to reduce
our image footprint, improve our security posture, and build our
container image reproducibly.

Fixes #1046
Refs #1047
2025-01-14 11:57:37 +02:00
Alex Pyrgiotis
5ff1d30278
container: Copy gVisor public key and a helper script
Download and copy the following artifacts that will be used for building
a Debian-based Dangerzone container image in the subsequent commits:
* The APT key for the gVisor repo [1]
* A helper script for building reproducible Debian images [2]

[1] https://gvisor.dev/archive.key
[2] d15cf12b26/repro-sources-list.sh
2025-01-14 10:53:11 +02:00
Alex Pyrgiotis
9c0c880cd3
docs: Add design document for artifact reproducibility
Refs #1047
2025-01-14 10:53:11 +02:00
Alex Pyrgiotis
e554a573e5
Reuse the same rootfs for the inner and outer container
Remove the need to copy the Dangerzone container image (used by the
inner container) within a wrapper gVisor image (used by the outer
container). Instead, use the root of the container filesystem for both
containers. We can do this safely because we don't mount any secrets to
the container, and because gVisor offers a read-only view of the
underlying filesystem

Fixes #1048
2025-01-13 18:14:23 +02:00
Alex Pyrgiotis
d2f483e970
Move container-only build context to dangerzone/container
Move container-only build context - currently just the entrypoint script
- from `dangerzone/gvisor_wrapper` to `dangerzone/container`. Update the
  rest of the scripts to use this location as well.
2025-01-13 18:14:15 +02:00
Alex Pyrgiotis
df3a60edc6
Whitespace fixes 2025-01-13 15:41:52 +02:00
Alexis Métaireau
1298e9c398
build: add build_scripts/env.py to the hashed files
Some checks failed
Scan latest app and container / security-scan-container (push) Has been cancelled
Scan latest app and container / security-scan-app (push) Has been cancelled
Tests / windows (push) Has been cancelled
Tests / macOS (arch64) (push) Has been cancelled
Tests / build-deb (ubuntu 22.04) (push) Has been cancelled
Tests / macOS (x86_64) (push) Has been cancelled
Tests / build-deb (debian bookworm) (push) Has been cancelled
Tests / build-deb (debian bullseye) (push) Has been cancelled
Tests / build-deb (debian trixie) (push) Has been cancelled
Tests / build-deb (ubuntu 20.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.10) (push) Has been cancelled
Tests / install-deb (debian bookworm) (push) Has been cancelled
Tests / install-deb (debian bullseye) (push) Has been cancelled
Tests / install-deb (debian trixie) (push) Has been cancelled
Tests / install-deb (ubuntu 20.04) (push) Has been cancelled
Tests / install-deb (ubuntu 22.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.10) (push) Has been cancelled
Tests / build-install-rpm (fedora 40) (push) Has been cancelled
Tests / build-install-rpm (fedora 41) (push) Has been cancelled
Tests / run tests (debian bookworm) (push) Has been cancelled
Tests / run tests (debian bullseye) (push) Has been cancelled
Tests / run tests (debian trixie) (push) Has been cancelled
Tests / run tests (fedora 40) (push) Has been cancelled
Tests / run tests (fedora 41) (push) Has been cancelled
Tests / run tests (ubuntu 20.04) (push) Has been cancelled
Tests / run tests (ubuntu 22.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.10) (push) Has been cancelled
It contains information that define the build environments, and as such, modifying it should result in a new release of the dev containers.
2025-01-08 06:18:30 +01:00