Compare commits

..

13 commits

Author SHA1 Message Date
Alex Pyrgiotis
1d2a91e8c5
FIXUP: Small fixes
Some checks failed
Tests / windows (push) Has been cancelled
Tests / macOS (arch64) (push) Has been cancelled
Tests / macOS (x86_64) (push) Has been cancelled
Tests / build-deb (debian bookworm) (push) Has been cancelled
Tests / build-deb (debian bullseye) (push) Has been cancelled
Tests / build-deb (debian trixie) (push) Has been cancelled
Tests / build-deb (ubuntu 20.04) (push) Has been cancelled
Tests / build-deb (ubuntu 22.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.10) (push) Has been cancelled
Tests / install-deb (debian bookworm) (push) Has been cancelled
Tests / install-deb (debian bullseye) (push) Has been cancelled
Tests / install-deb (debian trixie) (push) Has been cancelled
Tests / install-deb (ubuntu 20.04) (push) Has been cancelled
Tests / install-deb (ubuntu 22.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.10) (push) Has been cancelled
Tests / build-install-rpm (fedora 39) (push) Has been cancelled
Tests / build-install-rpm (fedora 40) (push) Has been cancelled
Tests / build-install-rpm (fedora 41) (push) Has been cancelled
Tests / run tests (debian bookworm) (push) Has been cancelled
Tests / run tests (debian bullseye) (push) Has been cancelled
Tests / run tests (debian trixie) (push) Has been cancelled
Tests / run tests (fedora 39) (push) Has been cancelled
Tests / run tests (fedora 40) (push) Has been cancelled
Tests / run tests (fedora 41) (push) Has been cancelled
Tests / run tests (ubuntu 20.04) (push) Has been cancelled
Tests / run tests (ubuntu 22.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.10) (push) Has been cancelled
2024-11-21 18:55:33 +02:00
Alex Pyrgiotis
82c29b2098
Make README.md point to INSTALL.md for instructions
Our repo's README.md should point to our INSTALL.md for installation
instructions, and not the other way around. This fixes an issue with
INSTALL.md pointing to a stale README.md version. Updating our README
before tagging is not possible, since the latest version is the one that
our users visit, and it can't point to download links that do not exist.

Fixes #1003
2024-11-21 18:55:33 +02:00
Alex Pyrgiotis
ce5aca4ba1
dev_scripts: Implement two more steps
Implement the following steps from the QA docs:

1. Check if the latest Python version that we support is installed. For
   example, we currently support Python 3.12, so we add code to check
   that the latest Python 3.12.x version is installed.
2. Download the Tesseract data using our script, both on Windows and
   Linux.
2024-11-21 18:29:43 +02:00
Alex Pyrgiotis
13f38cc8a9
Update our description 2024-11-21 18:29:43 +02:00
Alex Pyrgiotis
57df6fdfe5
Increase the size of the dz qube to 5GiB
Increase the size of the `dz` qube in our build instructions. We
increase it from 2GiB (default), to 5GiB (suggested), in order to cater
for some extra space that our build instructions need (e.g., the
download of the Tesseract data).
2024-11-21 18:29:43 +02:00
Alexis Métaireau
20354e7c11
CI: Use grep + cut rather than jq to get the version number
Some checks are pending
Tests / macOS (x86_64) (push) Blocked by required conditions
Tests / build-deb (debian bookworm) (push) Blocked by required conditions
Tests / build-deb (debian bullseye) (push) Blocked by required conditions
Tests / build-deb (debian trixie) (push) Blocked by required conditions
Tests / build-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / install-deb (debian bookworm) (push) Blocked by required conditions
Tests / install-deb (debian bullseye) (push) Blocked by required conditions
Tests / install-deb (debian trixie) (push) Blocked by required conditions
Tests / install-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 39) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 40) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 41) (push) Blocked by required conditions
Tests / run tests (debian bookworm) (push) Blocked by required conditions
Tests / run tests (debian bullseye) (push) Blocked by required conditions
Tests / run tests (debian trixie) (push) Blocked by required conditions
Tests / run tests (fedora 39) (push) Blocked by required conditions
Tests / run tests (fedora 40) (push) Blocked by required conditions
Tests / run tests (fedora 41) (push) Blocked by required conditions
Tests / run tests (ubuntu 20.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 22.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.10) (push) Blocked by required conditions
Scan latest app and container / security-scan-container (push) Waiting to run
Scan latest app and container / security-scan-app (push) Waiting to run
Github macOS runners don't come with `jq` pre-installed.
2024-11-21 12:34:15 +01:00
Alexis Métaireau
d722800a4b
Update Lock file
Some checks are pending
Tests / macOS (x86_64) (push) Blocked by required conditions
Tests / build-deb (debian bookworm) (push) Blocked by required conditions
Tests / build-deb (debian bullseye) (push) Blocked by required conditions
Tests / build-deb (debian trixie) (push) Blocked by required conditions
Tests / build-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / build-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / install-deb (debian bookworm) (push) Blocked by required conditions
Tests / install-deb (debian bullseye) (push) Blocked by required conditions
Tests / install-deb (debian trixie) (push) Blocked by required conditions
Tests / install-deb (ubuntu 20.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 22.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.04) (push) Blocked by required conditions
Tests / install-deb (ubuntu 24.10) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 39) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 40) (push) Blocked by required conditions
Tests / build-install-rpm (fedora 41) (push) Blocked by required conditions
Tests / run tests (debian bookworm) (push) Blocked by required conditions
Tests / run tests (debian bullseye) (push) Blocked by required conditions
Tests / run tests (debian trixie) (push) Blocked by required conditions
Tests / run tests (fedora 39) (push) Blocked by required conditions
Tests / run tests (fedora 40) (push) Blocked by required conditions
Tests / run tests (fedora 41) (push) Blocked by required conditions
Tests / run tests (ubuntu 20.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 22.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.04) (push) Blocked by required conditions
Tests / run tests (ubuntu 24.10) (push) Blocked by required conditions
Scan latest app and container / security-scan-container (push) Waiting to run
Scan latest app and container / security-scan-app (push) Waiting to run
2024-11-20 17:42:59 +01:00
Alexis Métaireau
4cfc633cdb
Add a script to help generate release notes from merged pull requests 2024-11-20 17:42:59 +01:00
Alexis Métaireau
944d58dd8d
CI: Update container scanning to account for the arm64 architecture. 2024-11-20 17:12:20 +01:00
Alexis Métaireau
f3806b96af
Reapply "Disable gVisor's DirectFS feature.""
This reverts commit 68f8338d20.

Fixes #982
2024-11-20 16:41:56 +01:00
Alexis Métaireau
c4bb7c28c8
Unpin gVisor, now that upstream is able to support Linux Yama Mode 2
Fixes #298
2024-11-20 16:41:55 +01:00
Alexis Métaireau
630083bdea
CI: Only run the CI on pull requests, and on the "main" branch
Previously, the actions were duplicated, due to the fact when developing
we often create feature branches and open pull requests.

This new setup requires us to open pull requests to trigger the CI.
2024-11-20 15:56:28 +01:00
Alexis Métaireau
504a9e1df2
tests: mark the hancom office suite tests for rerun on failures
Some checks failed
Tests / run tests (fedora 41) (push) Has been cancelled
Tests / run tests (ubuntu 20.04) (push) Has been cancelled
Tests / run tests (ubuntu 22.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.04) (push) Has been cancelled
Tests / run tests (ubuntu 24.10) (push) Has been cancelled
Tests / windows (push) Has been cancelled
Tests / macOS (arch64) (push) Has been cancelled
Tests / macOS (x86_64) (push) Has been cancelled
Tests / build-deb (debian bookworm) (push) Has been cancelled
Tests / build-deb (debian bullseye) (push) Has been cancelled
Tests / build-deb (debian trixie) (push) Has been cancelled
Tests / build-deb (ubuntu 20.04) (push) Has been cancelled
Tests / build-deb (ubuntu 22.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.04) (push) Has been cancelled
Tests / build-deb (ubuntu 24.10) (push) Has been cancelled
Tests / install-deb (debian bookworm) (push) Has been cancelled
Tests / install-deb (debian bullseye) (push) Has been cancelled
Tests / install-deb (debian trixie) (push) Has been cancelled
Tests / install-deb (ubuntu 20.04) (push) Has been cancelled
Tests / install-deb (ubuntu 22.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.04) (push) Has been cancelled
Tests / install-deb (ubuntu 24.10) (push) Has been cancelled
Tests / build-install-rpm (fedora 39) (push) Has been cancelled
Tests / build-install-rpm (fedora 40) (push) Has been cancelled
Tests / build-install-rpm (fedora 41) (push) Has been cancelled
Tests / run tests (debian bookworm) (push) Has been cancelled
Tests / run tests (debian bullseye) (push) Has been cancelled
Tests / run tests (debian trixie) (push) Has been cancelled
Tests / run tests (fedora 39) (push) Has been cancelled
Tests / run tests (fedora 40) (push) Has been cancelled
It seem that these tests are flaky, and as a result our CI pipeline is
failing from time to time. This will rerun it automatically when there
is an error.

See https://github.com/freedomofpress/dangerzone/issues/968 for more
information
2024-11-19 18:00:47 +01:00
15 changed files with 527 additions and 85 deletions

View file

@ -1,6 +1,10 @@
name: Build dev environments name: Build dev environments
on: on:
pull_request:
push: push:
branches:
- main
- "test/**"
schedule: schedule:
- cron: "0 0 * * *" # Run every day at 00:00 UTC. - cron: "0 0 * * *" # Run every day at 00:00 UTC.

View file

@ -1,6 +1,6 @@
name: Check branch conformity name: Check branch conformity
on: on:
push: pull_request:
jobs: jobs:
prevent-fixup-commits: prevent-fixup-commits:

View file

@ -1,8 +1,10 @@
name: Tests name: Tests
on: on:
push:
pull_request: pull_request:
branches: [main] push:
branches:
- main
- "test/**"
schedule: schedule:
- cron: "2 0 * * *" # Run every day at 02:00 UTC. - cron: "2 0 * * *" # Run every day at 02:00 UTC.
workflow_dispatch: workflow_dispatch:
@ -91,7 +93,8 @@ jobs:
windows: windows:
runs-on: windows-latest runs-on: windows-latest
needs: download-tessdata needs:
- download-tessdata
env: env:
DUMMY_CONVERSION: 1 DUMMY_CONVERSION: 1
steps: steps:
@ -121,7 +124,8 @@ jobs:
macOS: macOS:
name: "macOS (${{ matrix.arch }})" name: "macOS (${{ matrix.arch }})"
runs-on: ${{ matrix.runner }} runs-on: ${{ matrix.runner }}
needs: download-tessdata needs:
- download-tessdata
strategy: strategy:
matrix: matrix:
include: include:
@ -149,9 +153,10 @@ jobs:
run: poetry run make test run: poetry run make test
build-deb: build-deb:
needs:
- build-container-image
name: "build-deb (${{ matrix.distro }} ${{ matrix.version }})" name: "build-deb (${{ matrix.distro }} ${{ matrix.version }})"
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: build-container-image
strategy: strategy:
matrix: matrix:
include: include:
@ -219,7 +224,8 @@ jobs:
install-deb: install-deb:
name: "install-deb (${{ matrix.distro }} ${{ matrix.version }})" name: "install-deb (${{ matrix.distro }} ${{ matrix.version }})"
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: build-deb needs:
- build-deb
strategy: strategy:
matrix: matrix:
include: include:
@ -273,7 +279,8 @@ jobs:
build-install-rpm: build-install-rpm:
name: "build-install-rpm (${{ matrix.distro }} ${{matrix.version}})" name: "build-install-rpm (${{ matrix.distro }} ${{matrix.version}})"
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: build-container-image needs:
- build-container-image
strategy: strategy:
matrix: matrix:
distro: ["fedora"] distro: ["fedora"]

View file

@ -1,8 +1,9 @@
name: Scan latest app and container name: Scan latest app and container
on: on:
push: push:
branches:
- main
pull_request: pull_request:
branches: [ main ]
schedule: schedule:
- cron: '0 0 * * *' # Run every day at 00:00 UTC. - cron: '0 0 * * *' # Run every day at 00:00 UTC.
workflow_dispatch: workflow_dispatch:

View file

@ -6,14 +6,21 @@ on:
jobs: jobs:
security-scan-container: security-scan-container:
runs-on: ubuntu-latest strategy:
matrix:
include:
- runs-on: ubuntu-latest
arch: i686
- runs-on: macos-latest
arch: arm64
runs-on: ${{ matrix.runs-on }}
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Download container image for the latest release and load it - name: Download container image for the latest release and load it
run: | run: |
VERSION=$(curl https://api.github.com/repos/freedomofpress/dangerzone/releases/latest | jq -r '.tag_name') VERSION=$(curl https://api.github.com/repos/freedomofpress/dangerzone/releases/latest | grep "tag_name" | cut -d '"' -f 4)
CONTAINER_FILENAME=container-${VERSION:1}-i686.tar.gz CONTAINER_FILENAME=container-${VERSION:1}-${{ matrix.arch }}.tar.gz
wget https://github.com/freedomofpress/dangerzone/releases/download/${VERSION}/${CONTAINER_FILENAME} -O ${CONTAINER_FILENAME} wget https://github.com/freedomofpress/dangerzone/releases/download/${VERSION}/${CONTAINER_FILENAME} -O ${CONTAINER_FILENAME}
docker load -i ${CONTAINER_FILENAME} docker load -i ${CONTAINER_FILENAME}
# NOTE: Scan first without failing, else we won't be able to read the scan # NOTE: Scan first without failing, else we won't be able to read the scan
@ -30,7 +37,7 @@ jobs:
uses: github/codeql-action/upload-sarif@v3 uses: github/codeql-action/upload-sarif@v3
with: with:
sarif_file: ${{ steps.scan_container.outputs.sarif }} sarif_file: ${{ steps.scan_container.outputs.sarif }}
category: container category: container-${{ matrix.arch }}
- name: Inspect container scan report - name: Inspect container scan report
run: cat ${{ steps.scan_container.outputs.sarif }} run: cat ${{ steps.scan_container.outputs.sarif }}
- name: Scan container image - name: Scan container image

View file

@ -260,11 +260,17 @@ The following instructions require typing commands in a terminal in dom0.
``` ```
qvm-create --class AppVM --label red --template fedora-40-dz dz qvm-create --class AppVM --label red --template fedora-40-dz dz
qvm-volume resize dz:private $(numfmt --from=auto 20Gi)
``` ```
> :bulb: Alternatively, you can use a different app qube for Dangerzone > :bulb: Alternatively, you can use a different app qube for Dangerzone
> development. In that case, replace `dz` with the qube of your choice in the > development. In that case, replace `dz` with the qube of your choice in the
> steps below. > steps below.
>
> In the commands above, we also resize the private volume of the `dz` qube
> to 20GiB, since you may need some extra storage space when developing on
> Dangerzone (e.g., for container images, Tesseract data, and Python
> virtualenvs).
4. Add an RPC policy (`/etc/qubes/policy.d/50-dangerzone.policy`) that will 4. Add an RPC policy (`/etc/qubes/policy.d/50-dangerzone.policy`) that will
allow launching a disposable qube (`dz-dvm`) when Dangerzone converts a allow launching a disposable qube (`dz-dvm`) when Dangerzone converts a

View file

@ -7,6 +7,11 @@ since 0.4.1, and this project adheres to [Semantic Versioning](https://semver.or
## [Unreleased](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...HEAD) ## [Unreleased](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...HEAD)
### Added
- Disable gVisor's DirectFS feature ([#226](https://github.com/freedomofpress/dangerzone/issues/226)).
Thanks [EtiennePerot](https://github.com/EtiennePerot) for the contribution.
## [0.8.0](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...0.7.1) ## [0.8.0](https://github.com/freedomofpress/dangerzone/compare/v0.8.0...0.7.1)
### Added ### Added

View file

@ -74,9 +74,7 @@ FROM alpine:latest
RUN apk --no-cache -U upgrade && \ RUN apk --no-cache -U upgrade && \
apk --no-cache add python3 apk --no-cache add python3
# Temporarily pin gVisor to the latest working version (release-20240826.0). RUN GVISOR_URL="https://storage.googleapis.com/gvisor/releases/release/latest/$(uname -m)"; \
# See: https://github.com/freedomofpress/dangerzone/issues/928
RUN GVISOR_URL="https://storage.googleapis.com/gvisor/releases/release/20240826/$(uname -m)"; \
wget "${GVISOR_URL}/runsc" "${GVISOR_URL}/runsc.sha512" && \ wget "${GVISOR_URL}/runsc" "${GVISOR_URL}/runsc.sha512" && \
sha512sum -c runsc.sha512 && \ sha512sum -c runsc.sha512 && \
rm -f runsc.sha512 && \ rm -f runsc.sha512 && \

View file

@ -6,7 +6,7 @@ Take potentially dangerous PDFs, office documents, or images and convert them to
| ![Settings](./assets/screenshot1.png) | ![Converting](./assets/screenshot2.png) | ![Settings](./assets/screenshot1.png) | ![Converting](./assets/screenshot2.png)
|--|--| |--|--|
Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, in a separate sandbox, Dangerzone takes this pixel data and converts it back into a PDF. Dangerzone works like this: You give it a document that you don't know if you can trust (for example, an email attachment). Inside of a sandbox, Dangerzone converts the document to a PDF (if it isn't already one), and then converts the PDF into raw pixel data: a huge list of RGB color values for each page. Then, outside of the sandbox, Dangerzone takes this pixel data and converts it back into a PDF.
_Read more about Dangerzone in the [official site](https://dangerzone.rocks/about/)._ _Read more about Dangerzone in the [official site](https://dangerzone.rocks/about/)._

View file

@ -142,6 +142,9 @@ runsc_argv = [
"--rootless=true", "--rootless=true",
"--network=none", "--network=none",
"--root=/home/dangerzone/.containers", "--root=/home/dangerzone/.containers",
# Disable DirectFS for to make the seccomp filter even stricter,
# at some performance cost.
"--directfs=false",
] ]
if os.environ.get("RUNSC_DEBUG"): if os.environ.get("RUNSC_DEBUG"):
runsc_argv += ["--debug=true", "--alsologtostderr=true"] runsc_argv += ["--debug=true", "--alsologtostderr=true"]

View file

@ -0,0 +1,254 @@
#!/usr/bin/env python3
import argparse
import asyncio
import re
import sys
from datetime import datetime
from typing import Dict, List, Optional, Tuple
import httpx
REPOSITORY = "https://github.com/freedomofpress/dangerzone/"
TEMPLATE = "- {title} ([#{number}]({url}))"
def parse_version(version: str) -> Tuple[int, int]:
"""Extract major.minor from version string, ignoring patch"""
match = re.match(r"v?(\d+)\.(\d+)", version)
if not match:
raise ValueError(f"Invalid version format: {version}")
return (int(match.group(1)), int(match.group(2)))
async def get_last_minor_release(
client: httpx.AsyncClient, owner: str, repo: str
) -> Optional[str]:
"""Get the latest minor release date (ignoring patches)"""
response = await client.get(f"https://api.github.com/repos/{owner}/{repo}/releases")
response.raise_for_status()
releases = response.json()
if not releases:
return None
# Get the latest minor version by comparing major.minor numbers
current_version = parse_version(releases[0]["tag_name"])
latest_date = None
for release in releases:
try:
version = parse_version(release["tag_name"])
if version < current_version:
latest_date = release["published_at"]
break
except ValueError:
continue
return latest_date
async def get_issue_details(
client: httpx.AsyncClient, owner: str, repo: str, issue_number: int
) -> Optional[dict]:
"""Get issue title and number if it exists"""
response = await client.get(
f"https://api.github.com/repos/{owner}/{repo}/issues/{issue_number}"
)
if response.is_success:
data = response.json()
return {
"title": data["title"],
"number": data["number"],
"url": data["html_url"],
}
return None
def extract_issue_number(pr_body: Optional[str]) -> Optional[int]:
"""Extract issue number from PR body looking for common formats like 'Fixes #123' or 'Closes #123'"""
if not pr_body:
return None
patterns = [
r"(?:closes|fixes|resolves)\s*#(\d+)",
r"(?:close|fix|resolve)\s*#(\d+)",
]
for pattern in patterns:
match = re.search(pattern, pr_body.lower())
if match:
return int(match.group(1))
return None
async def verify_commit_in_master(
client: httpx.AsyncClient, owner: str, repo: str, commit_id: str
) -> bool:
"""Verify if a commit exists in master"""
response = await client.get(
f"https://api.github.com/repos/{owner}/{repo}/commits/{commit_id}"
)
return response.is_success and response.json().get("commit") is not None
async def process_issue_events(
client: httpx.AsyncClient, owner: str, repo: str, issue: Dict
) -> Optional[Dict]:
"""Process events for a single issue"""
events_response = await client.get(f"{issue['url']}/events")
if not events_response.is_success:
return None
for event in events_response.json():
if event["event"] == "closed" and event.get("commit_id"):
if await verify_commit_in_master(client, owner, repo, event["commit_id"]):
return {
"title": issue["title"],
"number": issue["number"],
"url": issue["html_url"],
}
return None
async def get_closed_issues(
client: httpx.AsyncClient, owner: str, repo: str, since: str
) -> List[Dict]:
"""Get issues closed by commits to master since the given date"""
response = await client.get(
f"https://api.github.com/repos/{owner}/{repo}/issues",
params={
"state": "closed",
"sort": "updated",
"direction": "desc",
"since": since,
"per_page": 100,
},
)
response.raise_for_status()
tasks = []
since_date = datetime.strptime(since, "%Y-%m-%dT%H:%M:%SZ")
for issue in response.json():
if "pull_request" in issue:
continue
closed_at = datetime.strptime(issue["closed_at"], "%Y-%m-%dT%H:%M:%SZ")
if closed_at <= since_date:
continue
tasks.append(process_issue_events(client, owner, repo, issue))
results = await asyncio.gather(*tasks)
return [r for r in results if r is not None]
async def process_pull_request(
client: httpx.AsyncClient,
owner: str,
repo: str,
pr: Dict,
closed_issues: List[Dict],
) -> Optional[str]:
"""Process a single pull request"""
issue_number = extract_issue_number(pr.get("body"))
if issue_number:
issue = await get_issue_details(client, owner, repo, issue_number)
if issue:
if not any(i["number"] == issue["number"] for i in closed_issues):
return TEMPLATE.format(**issue)
return None
return TEMPLATE.format(title=pr["title"], number=pr["number"], url=pr["html_url"])
async def get_changes_since_last_release(
owner: str, repo: str, token: Optional[str] = None
) -> List[str]:
headers = {
"Accept": "application/vnd.github.v3+json",
}
if token:
headers["Authorization"] = f"token {token}"
else:
print(
"Warning: No token provided. API rate limiting may occur.", file=sys.stderr
)
async with httpx.AsyncClient(headers=headers, timeout=30.0) as client:
# Get the date of last minor release
since = await get_last_minor_release(client, owner, repo)
if not since:
return []
changes = []
# Get issues closed by commits to master
closed_issues = await get_closed_issues(client, owner, repo, since)
changes.extend([TEMPLATE.format(**issue) for issue in closed_issues])
# Get merged PRs
response = await client.get(
f"https://api.github.com/repos/{owner}/{repo}/pulls",
params={
"state": "closed",
"sort": "updated",
"direction": "desc",
"per_page": 100,
},
)
response.raise_for_status()
# Process PRs in parallel
pr_tasks = []
for pr in response.json():
if not pr["merged_at"]:
continue
if since and pr["merged_at"] <= since:
break
pr_tasks.append(
process_pull_request(client, owner, repo, pr, closed_issues)
)
pr_results = await asyncio.gather(*pr_tasks)
changes.extend([r for r in pr_results if r is not None])
return changes
async def main_async():
parser = argparse.ArgumentParser(description="Generate release notes from GitHub")
parser.add_argument("--token", "-t", help="the file path to the GitHub API token")
args = parser.parse_args()
token = None
if args.token:
with open(args.token) as f:
token = f.read().strip()
try:
url_path = REPOSITORY.rstrip("/").split("github.com/")[1]
owner, repo = url_path.split("/")[-2:]
except (ValueError, IndexError):
print("Error: Invalid GitHub URL", file=sys.stderr)
sys.exit(1)
try:
notes = await get_changes_since_last_release(owner, repo, token)
print("\n".join(notes))
except httpx.HTTPError as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
def main():
asyncio.run(main_async())
if __name__ == "__main__":
main()

View file

@ -3,14 +3,20 @@
import abc import abc
import argparse import argparse
import difflib import difflib
import json
import logging import logging
import re import re
import selectors import selectors
import subprocess import subprocess
import sys import sys
import urllib.request
from pathlib import Path
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
PYTHON_VERSION = "3.12"
EOL_PYTHON_URL = "https://endoflife.date/api/python.json"
CONTENT_QA = r"""## QA CONTENT_QA = r"""## QA
To ensure that new releases do not introduce regressions, and support existing To ensure that new releases do not introduce regressions, and support existing
@ -776,6 +782,10 @@ class QABase(abc.ABC):
self.prompt("Does it pass?", choices=["y", "n"]) self.prompt("Does it pass?", choices=["y", "n"])
logger.info("Successfully completed QA scenarios") logger.info("Successfully completed QA scenarios")
@task("Download Tesseract data", auto=True)
def download_tessdata(self):
self.run("python", str(Path("install", "common", "download-tessdata.py")))
@classmethod @classmethod
@abc.abstractmethod @abc.abstractmethod
def get_id(cls): def get_id(cls):
@ -802,6 +812,40 @@ class QAWindows(QABase):
while msvcrt.kbhit(): while msvcrt.kbhit():
msvcrt.getch() msvcrt.getch()
def get_latest_python_release(self):
with urllib.request.urlopen(EOL_PYTHON_URL) as f:
resp = f.read()
releases = json.loads(resp)
for release in releases:
if release["cycle"] == PYTHON_VERSION:
# Transform the Python version string (e.g., "3.12.7") into a list
# (e.g., [3, 12, 7]), and return it
return [int(num) for num in release["latest"].split(".")]
raise RuntimeError(
f"Could not find a Python release for version {PYTHON_VERSION}"
)
@QABase.task(
f"Install the latest version of Python {PYTHON_VERSION}", ref=REF_BUILD
)
def install_python(self):
logger.info("Getting latest Python release")
try:
latest_version = self.get_latest_python_release()
except Exception:
logger.error("Could not verify that the latest Python version is installed")
cur_version = list(sys.version_info[:3])
if latest_version > cur_version:
self.prompt(
f"You need to install the latest Python version ({latest_version})"
)
elif latest_version == cur_version:
logger.info(
f"Verified that the latest Python version ({latest_version}) is installed"
)
@QABase.task("Install and Run Docker Desktop", ref=REF_BUILD) @QABase.task("Install and Run Docker Desktop", ref=REF_BUILD)
def install_docker(self): def install_docker(self):
logger.info("Checking if Docker Desktop is installed and running") logger.info("Checking if Docker Desktop is installed and running")
@ -816,7 +860,7 @@ class QAWindows(QABase):
) )
def install_poetry(self): def install_poetry(self):
self.run("python", "-m", "pip", "install", "poetry") self.run("python", "-m", "pip", "install", "poetry")
self.run("poetry", "install") self.run("poetry", "install", "--sync")
@QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True) @QABase.task("Build Dangerzone container image", ref=REF_BUILD, auto=True)
def build_image(self): def build_image(self):
@ -838,9 +882,11 @@ class QAWindows(QABase):
return "windows" return "windows"
def start(self): def start(self):
self.install_python()
self.install_docker() self.install_docker()
self.install_poetry() self.install_poetry()
self.build_image() self.build_image()
self.download_tessdata()
self.run_tests() self.run_tests()
self.build_dangerzone_exe() self.build_dangerzone_exe()
@ -933,6 +979,7 @@ class QALinux(QABase):
def start(self): def start(self):
self.build_dev_image() self.build_dev_image()
self.build_container_image() self.build_container_image()
self.download_tessdata()
self.run_tests() self.run_tests()
self.build_package() self.build_package()
self.build_qa_image() self.build_qa_image()

239
poetry.lock generated
View file

@ -11,6 +11,28 @@ files = [
{file = "altgraph-0.17.4.tar.gz", hash = "sha256:1b5afbb98f6c4dcadb2e2ae6ab9fa994bbb8c1d75f4fa96d340f9437ae454406"}, {file = "altgraph-0.17.4.tar.gz", hash = "sha256:1b5afbb98f6c4dcadb2e2ae6ab9fa994bbb8c1d75f4fa96d340f9437ae454406"},
] ]
[[package]]
name = "anyio"
version = "4.6.2.post1"
description = "High level compatibility layer for multiple asynchronous event loop implementations"
optional = false
python-versions = ">=3.9"
files = [
{file = "anyio-4.6.2.post1-py3-none-any.whl", hash = "sha256:6d170c36fba3bdd840c73d3868c1e777e33676a69c3a72cf0a0d5d6d8009b61d"},
{file = "anyio-4.6.2.post1.tar.gz", hash = "sha256:4c8bc31ccdb51c7f7bd251f51c609e038d63e34219b44aa86e47576389880b4c"},
]
[package.dependencies]
exceptiongroup = {version = ">=1.0.2", markers = "python_version < \"3.11\""}
idna = ">=2.8"
sniffio = ">=1.1"
typing-extensions = {version = ">=4.1", markers = "python_version < \"3.11\""}
[package.extras]
doc = ["Sphinx (>=7.4,<8.0)", "packaging", "sphinx-autodoc-typehints (>=1.2.0)", "sphinx-rtd-theme"]
test = ["anyio[trio]", "coverage[toml] (>=7)", "exceptiongroup (>=1.2.0)", "hypothesis (>=4.0)", "psutil (>=5.9)", "pytest (>=7.0)", "pytest-mock (>=3.6.1)", "trustme", "truststore (>=0.9.1)", "uvloop (>=0.21.0b1)"]
trio = ["trio (>=0.26.1)"]
[[package]] [[package]]
name = "appdirs" name = "appdirs"
version = "1.4.4" version = "1.4.4"
@ -220,73 +242,73 @@ files = [
[[package]] [[package]]
name = "coverage" name = "coverage"
version = "7.6.4" version = "7.6.7"
description = "Code coverage measurement for Python" description = "Code coverage measurement for Python"
optional = false optional = false
python-versions = ">=3.9" python-versions = ">=3.9"
files = [ files = [
{file = "coverage-7.6.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:5f8ae553cba74085db385d489c7a792ad66f7f9ba2ee85bfa508aeb84cf0ba07"}, {file = "coverage-7.6.7-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:108bb458827765d538abcbf8288599fee07d2743357bdd9b9dad456c287e121e"},
{file = "coverage-7.6.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:8165b796df0bd42e10527a3f493c592ba494f16ef3c8b531288e3d0d72c1f6f0"}, {file = "coverage-7.6.7-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c973b2fe4dc445cb865ab369df7521df9c27bf40715c837a113edaa2aa9faf45"},
{file = "coverage-7.6.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c7c8b95bf47db6d19096a5e052ffca0a05f335bc63cef281a6e8fe864d450a72"}, {file = "coverage-7.6.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3c6b24007c4bcd0b19fac25763a7cac5035c735ae017e9a349b927cfc88f31c1"},
{file = "coverage-7.6.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8ed9281d1b52628e81393f5eaee24a45cbd64965f41857559c2b7ff19385df51"}, {file = "coverage-7.6.7-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:acbb8af78f8f91b3b51f58f288c0994ba63c646bc1a8a22ad072e4e7e0a49f1c"},
{file = "coverage-7.6.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0809082ee480bb8f7416507538243c8863ac74fd8a5d2485c46f0f7499f2b491"}, {file = "coverage-7.6.7-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ad32a981bcdedb8d2ace03b05e4fd8dace8901eec64a532b00b15217d3677dd2"},
{file = "coverage-7.6.4-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:d541423cdd416b78626b55f123412fcf979d22a2c39fce251b350de38c15c15b"}, {file = "coverage-7.6.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:34d23e28ccb26236718a3a78ba72744212aa383141961dd6825f6595005c8b06"},
{file = "coverage-7.6.4-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:58809e238a8a12a625c70450b48e8767cff9eb67c62e6154a642b21ddf79baea"}, {file = "coverage-7.6.7-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:e25bacb53a8c7325e34d45dddd2f2fbae0dbc230d0e2642e264a64e17322a777"},
{file = "coverage-7.6.4-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:c9b8e184898ed014884ca84c70562b4a82cbc63b044d366fedc68bc2b2f3394a"}, {file = "coverage-7.6.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:af05bbba896c4472a29408455fe31b3797b4d8648ed0a2ccac03e074a77e2314"},
{file = "coverage-7.6.4-cp310-cp310-win32.whl", hash = "sha256:6bd818b7ea14bc6e1f06e241e8234508b21edf1b242d49831831a9450e2f35fa"}, {file = "coverage-7.6.7-cp310-cp310-win32.whl", hash = "sha256:796c9b107d11d2d69e1849b2dfe41730134b526a49d3acb98ca02f4985eeff7a"},
{file = "coverage-7.6.4-cp310-cp310-win_amd64.whl", hash = "sha256:06babbb8f4e74b063dbaeb74ad68dfce9186c595a15f11f5d5683f748fa1d172"}, {file = "coverage-7.6.7-cp310-cp310-win_amd64.whl", hash = "sha256:987a8e3da7da4eed10a20491cf790589a8e5e07656b6dc22d3814c4d88faf163"},
{file = "coverage-7.6.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:73d2b73584446e66ee633eaad1a56aad577c077f46c35ca3283cd687b7715b0b"}, {file = "coverage-7.6.7-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:7e61b0e77ff4dddebb35a0e8bb5a68bf0f8b872407d8d9f0c726b65dfabe2469"},
{file = "coverage-7.6.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:51b44306032045b383a7a8a2c13878de375117946d68dcb54308111f39775a25"}, {file = "coverage-7.6.7-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:1a5407a75ca4abc20d6252efeb238377a71ce7bda849c26c7a9bece8680a5d99"},
{file = "coverage-7.6.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0b3fb02fe73bed561fa12d279a417b432e5b50fe03e8d663d61b3d5990f29546"}, {file = "coverage-7.6.7-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:df002e59f2d29e889c37abd0b9ee0d0e6e38c24f5f55d71ff0e09e3412a340ec"},
{file = "coverage-7.6.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ed8fe9189d2beb6edc14d3ad19800626e1d9f2d975e436f84e19efb7fa19469b"}, {file = "coverage-7.6.7-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:673184b3156cba06154825f25af33baa2671ddae6343f23175764e65a8c4c30b"},
{file = "coverage-7.6.4-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b369ead6527d025a0fe7bd3864e46dbee3aa8f652d48df6174f8d0bac9e26e0e"}, {file = "coverage-7.6.7-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e69ad502f1a2243f739f5bd60565d14a278be58be4c137d90799f2c263e7049a"},
{file = "coverage-7.6.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ade3ca1e5f0ff46b678b66201f7ff477e8fa11fb537f3b55c3f0568fbfe6e718"}, {file = "coverage-7.6.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:60dcf7605c50ea72a14490d0756daffef77a5be15ed1b9fea468b1c7bda1bc3b"},
{file = "coverage-7.6.4-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:27fb4a050aaf18772db513091c9c13f6cb94ed40eacdef8dad8411d92d9992db"}, {file = "coverage-7.6.7-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:9c2eb378bebb2c8f65befcb5147877fc1c9fbc640fc0aad3add759b5df79d55d"},
{file = "coverage-7.6.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4f704f0998911abf728a7783799444fcbbe8261c4a6c166f667937ae6a8aa522"}, {file = "coverage-7.6.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:3c0317288f032221d35fa4cbc35d9f4923ff0dfd176c79c9b356e8ef8ef2dff4"},
{file = "coverage-7.6.4-cp311-cp311-win32.whl", hash = "sha256:29155cd511ee058e260db648b6182c419422a0d2e9a4fa44501898cf918866cf"}, {file = "coverage-7.6.7-cp311-cp311-win32.whl", hash = "sha256:951aade8297358f3618a6e0660dc74f6b52233c42089d28525749fc8267dccd2"},
{file = "coverage-7.6.4-cp311-cp311-win_amd64.whl", hash = "sha256:8902dd6a30173d4ef09954bfcb24b5d7b5190cf14a43170e386979651e09ba19"}, {file = "coverage-7.6.7-cp311-cp311-win_amd64.whl", hash = "sha256:5e444b8e88339a2a67ce07d41faabb1d60d1004820cee5a2c2b54e2d8e429a0f"},
{file = "coverage-7.6.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:12394842a3a8affa3ba62b0d4ab7e9e210c5e366fbac3e8b2a68636fb19892c2"}, {file = "coverage-7.6.7-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f07ff574986bc3edb80e2c36391678a271d555f91fd1d332a1e0f4b5ea4b6ea9"},
{file = "coverage-7.6.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2b6b4c83d8e8ea79f27ab80778c19bc037759aea298da4b56621f4474ffeb117"}, {file = "coverage-7.6.7-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:49ed5ee4109258973630c1f9d099c7e72c5c36605029f3a91fe9982c6076c82b"},
{file = "coverage-7.6.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1d5b8007f81b88696d06f7df0cb9af0d3b835fe0c8dbf489bad70b45f0e45613"}, {file = "coverage-7.6.7-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f3e8796434a8106b3ac025fd15417315d7a58ee3e600ad4dbcfddc3f4b14342c"},
{file = "coverage-7.6.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b57b768feb866f44eeed9f46975f3d6406380275c5ddfe22f531a2bf187eda27"}, {file = "coverage-7.6.7-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a3b925300484a3294d1c70f6b2b810d6526f2929de954e5b6be2bf8caa1f12c1"},
{file = "coverage-7.6.4-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5915fcdec0e54ee229926868e9b08586376cae1f5faa9bbaf8faf3561b393d52"}, {file = "coverage-7.6.7-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c42ec2c522e3ddd683dec5cdce8e62817afb648caedad9da725001fa530d354"},
{file = "coverage-7.6.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0b58c672d14f16ed92a48db984612f5ce3836ae7d72cdd161001cc54512571f2"}, {file = "coverage-7.6.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0266b62cbea568bd5e93a4da364d05de422110cbed5056d69339bd5af5685433"},
{file = "coverage-7.6.4-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:2fdef0d83a2d08d69b1f2210a93c416d54e14d9eb398f6ab2f0a209433db19e1"}, {file = "coverage-7.6.7-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:e5f2a0f161d126ccc7038f1f3029184dbdf8f018230af17ef6fd6a707a5b881f"},
{file = "coverage-7.6.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:8cf717ee42012be8c0cb205dbbf18ffa9003c4cbf4ad078db47b95e10748eec5"}, {file = "coverage-7.6.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c132b5a22821f9b143f87446805e13580b67c670a548b96da945a8f6b4f2efbb"},
{file = "coverage-7.6.4-cp312-cp312-win32.whl", hash = "sha256:7bb92c539a624cf86296dd0c68cd5cc286c9eef2d0c3b8b192b604ce9de20a17"}, {file = "coverage-7.6.7-cp312-cp312-win32.whl", hash = "sha256:7c07de0d2a110f02af30883cd7dddbe704887617d5c27cf373362667445a4c76"},
{file = "coverage-7.6.4-cp312-cp312-win_amd64.whl", hash = "sha256:1032e178b76a4e2b5b32e19d0fd0abbce4b58e77a1ca695820d10e491fa32b08"}, {file = "coverage-7.6.7-cp312-cp312-win_amd64.whl", hash = "sha256:fd49c01e5057a451c30c9b892948976f5d38f2cbd04dc556a82743ba8e27ed8c"},
{file = "coverage-7.6.4-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:023bf8ee3ec6d35af9c1c6ccc1d18fa69afa1cb29eaac57cb064dbb262a517f9"}, {file = "coverage-7.6.7-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:46f21663e358beae6b368429ffadf14ed0a329996248a847a4322fb2e35d64d3"},
{file = "coverage-7.6.4-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:b0ac3d42cb51c4b12df9c5f0dd2f13a4f24f01943627120ec4d293c9181219ba"}, {file = "coverage-7.6.7-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:40cca284c7c310d622a1677f105e8507441d1bb7c226f41978ba7c86979609ab"},
{file = "coverage-7.6.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f8fe4984b431f8621ca53d9380901f62bfb54ff759a1348cd140490ada7b693c"}, {file = "coverage-7.6.7-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77256ad2345c29fe59ae861aa11cfc74579c88d4e8dbf121cbe46b8e32aec808"},
{file = "coverage-7.6.4-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5fbd612f8a091954a0c8dd4c0b571b973487277d26476f8480bfa4b2a65b5d06"}, {file = "coverage-7.6.7-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:87ea64b9fa52bf395272e54020537990a28078478167ade6c61da7ac04dc14bc"},
{file = "coverage-7.6.4-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dacbc52de979f2823a819571f2e3a350a7e36b8cb7484cdb1e289bceaf35305f"}, {file = "coverage-7.6.7-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2d608a7808793e3615e54e9267519351c3ae204a6d85764d8337bd95993581a8"},
{file = "coverage-7.6.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:dab4d16dfef34b185032580e2f2f89253d302facba093d5fa9dbe04f569c4f4b"}, {file = "coverage-7.6.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:cdd94501d65adc5c24f8a1a0eda110452ba62b3f4aeaba01e021c1ed9cb8f34a"},
{file = "coverage-7.6.4-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:862264b12ebb65ad8d863d51f17758b1684560b66ab02770d4f0baf2ff75da21"}, {file = "coverage-7.6.7-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:82c809a62e953867cf57e0548c2b8464207f5f3a6ff0e1e961683e79b89f2c55"},
{file = "coverage-7.6.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5beb1ee382ad32afe424097de57134175fea3faf847b9af002cc7895be4e2a5a"}, {file = "coverage-7.6.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:bb684694e99d0b791a43e9fc0fa58efc15ec357ac48d25b619f207c41f2fd384"},
{file = "coverage-7.6.4-cp313-cp313-win32.whl", hash = "sha256:bf20494da9653f6410213424f5f8ad0ed885e01f7e8e59811f572bdb20b8972e"}, {file = "coverage-7.6.7-cp313-cp313-win32.whl", hash = "sha256:963e4a08cbb0af6623e61492c0ec4c0ec5c5cf74db5f6564f98248d27ee57d30"},
{file = "coverage-7.6.4-cp313-cp313-win_amd64.whl", hash = "sha256:182e6cd5c040cec0a1c8d415a87b67ed01193ed9ad458ee427741c7d8513d963"}, {file = "coverage-7.6.7-cp313-cp313-win_amd64.whl", hash = "sha256:14045b8bfd5909196a90da145a37f9d335a5d988a83db34e80f41e965fb7cb42"},
{file = "coverage-7.6.4-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:a181e99301a0ae128493a24cfe5cfb5b488c4e0bf2f8702091473d033494d04f"}, {file = "coverage-7.6.7-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:f2c7a045eef561e9544359a0bf5784b44e55cefc7261a20e730baa9220c83413"},
{file = "coverage-7.6.4-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:df57bdbeffe694e7842092c5e2e0bc80fff7f43379d465f932ef36f027179806"}, {file = "coverage-7.6.7-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5dd4e4a49d9c72a38d18d641135d2fb0bdf7b726ca60a103836b3d00a1182acd"},
{file = "coverage-7.6.4-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0bcd1069e710600e8e4cf27f65c90c7843fa8edfb4520fb0ccb88894cad08b11"}, {file = "coverage-7.6.7-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5c95e0fa3d1547cb6f021ab72f5c23402da2358beec0a8e6d19a368bd7b0fb37"},
{file = "coverage-7.6.4-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:99b41d18e6b2a48ba949418db48159d7a2e81c5cc290fc934b7d2380515bd0e3"}, {file = "coverage-7.6.7-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f63e21ed474edd23f7501f89b53280014436e383a14b9bd77a648366c81dce7b"},
{file = "coverage-7.6.4-cp313-cp313t-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a6b1e54712ba3474f34b7ef7a41e65bd9037ad47916ccb1cc78769bae324c01a"}, {file = "coverage-7.6.7-cp313-cp313t-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ead9b9605c54d15be228687552916c89c9683c215370c4a44f1f217d2adcc34d"},
{file = "coverage-7.6.4-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:53d202fd109416ce011578f321460795abfe10bb901b883cafd9b3ef851bacfc"}, {file = "coverage-7.6.7-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:0573f5cbf39114270842d01872952d301027d2d6e2d84013f30966313cadb529"},
{file = "coverage-7.6.4-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:c48167910a8f644671de9f2083a23630fbf7a1cb70ce939440cd3328e0919f70"}, {file = "coverage-7.6.7-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:e2c8e3384c12dfa19fa9a52f23eb091a8fad93b5b81a41b14c17c78e23dd1d8b"},
{file = "coverage-7.6.4-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:cc8ff50b50ce532de2fa7a7daae9dd12f0a699bfcd47f20945364e5c31799fef"}, {file = "coverage-7.6.7-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:70a56a2ec1869e6e9fa69ef6b76b1a8a7ef709972b9cc473f9ce9d26b5997ce3"},
{file = "coverage-7.6.4-cp313-cp313t-win32.whl", hash = "sha256:b8d3a03d9bfcaf5b0141d07a88456bb6a4c3ce55c080712fec8418ef3610230e"}, {file = "coverage-7.6.7-cp313-cp313t-win32.whl", hash = "sha256:dbba8210f5067398b2c4d96b4e64d8fb943644d5eb70be0d989067c8ca40c0f8"},
{file = "coverage-7.6.4-cp313-cp313t-win_amd64.whl", hash = "sha256:f3ddf056d3ebcf6ce47bdaf56142af51bb7fad09e4af310241e9db7a3a8022e1"}, {file = "coverage-7.6.7-cp313-cp313t-win_amd64.whl", hash = "sha256:dfd14bcae0c94004baba5184d1c935ae0d1231b8409eb6c103a5fd75e8ecdc56"},
{file = "coverage-7.6.4-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9cb7fa111d21a6b55cbf633039f7bc2749e74932e3aa7cb7333f675a58a58bf3"}, {file = "coverage-7.6.7-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:37a15573f988b67f7348916077c6d8ad43adb75e478d0910957394df397d2874"},
{file = "coverage-7.6.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:11a223a14e91a4693d2d0755c7a043db43d96a7450b4f356d506c2562c48642c"}, {file = "coverage-7.6.7-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:b6cce5c76985f81da3769c52203ee94722cd5d5889731cd70d31fee939b74bf0"},
{file = "coverage-7.6.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a413a096c4cbac202433c850ee43fa326d2e871b24554da8327b01632673a076"}, {file = "coverage-7.6.7-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a1ab9763d291a17b527ac6fd11d1a9a9c358280adb320e9c2672a97af346ac2c"},
{file = "coverage-7.6.4-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:00a1d69c112ff5149cabe60d2e2ee948752c975d95f1e1096742e6077affd376"}, {file = "coverage-7.6.7-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6cf96ceaa275f071f1bea3067f8fd43bec184a25a962c754024c973af871e1b7"},
{file = "coverage-7.6.4-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1f76846299ba5c54d12c91d776d9605ae33f8ae2b9d1d3c3703cf2db1a67f2c0"}, {file = "coverage-7.6.7-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aee9cf6b0134d6f932d219ce253ef0e624f4fa588ee64830fcba193269e4daa3"},
{file = "coverage-7.6.4-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fe439416eb6380de434886b00c859304338f8b19f6f54811984f3420a2e03858"}, {file = "coverage-7.6.7-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:2bc3e45c16564cc72de09e37413262b9f99167803e5e48c6156bccdfb22c8327"},
{file = "coverage-7.6.4-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:0294ca37f1ba500667b1aef631e48d875ced93ad5e06fa665a3295bdd1d95111"}, {file = "coverage-7.6.7-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:623e6965dcf4e28a3debaa6fcf4b99ee06d27218f46d43befe4db1c70841551c"},
{file = "coverage-7.6.4-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:6f01ba56b1c0e9d149f9ac85a2f999724895229eb36bd997b61e62999e9b0901"}, {file = "coverage-7.6.7-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:850cfd2d6fc26f8346f422920ac204e1d28814e32e3a58c19c91980fa74d8289"},
{file = "coverage-7.6.4-cp39-cp39-win32.whl", hash = "sha256:bc66f0bf1d7730a17430a50163bb264ba9ded56739112368ba985ddaa9c3bd09"}, {file = "coverage-7.6.7-cp39-cp39-win32.whl", hash = "sha256:c296263093f099da4f51b3dff1eff5d4959b527d4f2f419e16508c5da9e15e8c"},
{file = "coverage-7.6.4-cp39-cp39-win_amd64.whl", hash = "sha256:c481b47f6b5845064c65a7bc78bc0860e635a9b055af0df46fdf1c58cebf8e8f"}, {file = "coverage-7.6.7-cp39-cp39-win_amd64.whl", hash = "sha256:90746521206c88bdb305a4bf3342b1b7316ab80f804d40c536fc7d329301ee13"},
{file = "coverage-7.6.4-pp39.pp310-none-any.whl", hash = "sha256:3c65d37f3a9ebb703e710befdc489a38683a5b152242664b973a7b7b22348a4e"}, {file = "coverage-7.6.7-pp39.pp310-none-any.whl", hash = "sha256:0ddcb70b3a3a57581b450571b31cb774f23eb9519c2aaa6176d3a84c9fc57671"},
{file = "coverage-7.6.4.tar.gz", hash = "sha256:29fc0f17b1d3fea332f8001d4558f8214af7f1d87a345f3a133c901d60347c73"}, {file = "coverage-7.6.7.tar.gz", hash = "sha256:d79d4826e41441c9a118ff045e4bccb9fdbdcb1d02413e7ea6eb5c87b5439d24"},
] ]
[package.dependencies] [package.dependencies]
@ -404,6 +426,63 @@ files = [
[package.extras] [package.extras]
test = ["pytest (>=6)"] test = ["pytest (>=6)"]
[[package]]
name = "h11"
version = "0.14.0"
description = "A pure-Python, bring-your-own-I/O implementation of HTTP/1.1"
optional = false
python-versions = ">=3.7"
files = [
{file = "h11-0.14.0-py3-none-any.whl", hash = "sha256:e3fe4ac4b851c468cc8363d500db52c2ead036020723024a109d37346efaa761"},
{file = "h11-0.14.0.tar.gz", hash = "sha256:8f19fbbe99e72420ff35c00b27a34cb9937e902a8b810e2c88300c6f0a3b699d"},
]
[[package]]
name = "httpcore"
version = "1.0.7"
description = "A minimal low-level HTTP client."
optional = false
python-versions = ">=3.8"
files = [
{file = "httpcore-1.0.7-py3-none-any.whl", hash = "sha256:a3fff8f43dc260d5bd363d9f9cf1830fa3a458b332856f34282de498ed420edd"},
{file = "httpcore-1.0.7.tar.gz", hash = "sha256:8551cb62a169ec7162ac7be8d4817d561f60e08eaa485234898414bb5a8a0b4c"},
]
[package.dependencies]
certifi = "*"
h11 = ">=0.13,<0.15"
[package.extras]
asyncio = ["anyio (>=4.0,<5.0)"]
http2 = ["h2 (>=3,<5)"]
socks = ["socksio (==1.*)"]
trio = ["trio (>=0.22.0,<1.0)"]
[[package]]
name = "httpx"
version = "0.27.2"
description = "The next generation HTTP client."
optional = false
python-versions = ">=3.8"
files = [
{file = "httpx-0.27.2-py3-none-any.whl", hash = "sha256:7bb2708e112d8fdd7829cd4243970f0c223274051cb35ee80c03301ee29a3df0"},
{file = "httpx-0.27.2.tar.gz", hash = "sha256:f7c2be1d2f3c3c3160d441802406b206c2b76f5947b11115e6df10c6c65e66c2"},
]
[package.dependencies]
anyio = "*"
certifi = "*"
httpcore = "==1.*"
idna = "*"
sniffio = "*"
[package.extras]
brotli = ["brotli", "brotlicffi"]
cli = ["click (==8.*)", "pygments (==2.*)", "rich (>=10,<14)"]
http2 = ["h2 (>=3,<5)"]
socks = ["socksio (==1.*)"]
zstd = ["zstandard (>=0.18.0)"]
[[package]] [[package]]
name = "idna" name = "idna"
version = "3.10" version = "3.10"
@ -865,6 +944,21 @@ pytest = "*"
dev = ["pre-commit", "tox"] dev = ["pre-commit", "tox"]
doc = ["sphinx", "sphinx-rtd-theme"] doc = ["sphinx", "sphinx-rtd-theme"]
[[package]]
name = "pytest-rerunfailures"
version = "14.0"
description = "pytest plugin to re-run tests to eliminate flaky failures"
optional = false
python-versions = ">=3.8"
files = [
{file = "pytest-rerunfailures-14.0.tar.gz", hash = "sha256:4a400bcbcd3c7a4ad151ab8afac123d90eca3abe27f98725dc4d9702887d2e92"},
{file = "pytest_rerunfailures-14.0-py3-none-any.whl", hash = "sha256:4197bdd2eaeffdbf50b5ea6e7236f47ff0e44d1def8dae08e409f536d84e7b32"},
]
[package.dependencies]
packaging = ">=17.1"
pytest = ">=7.2"
[[package]] [[package]]
name = "pytest-subprocess" name = "pytest-subprocess"
version = "1.5.2" version = "1.5.2"
@ -945,13 +1039,13 @@ use-chardet-on-py3 = ["chardet (>=3.0.2,<6)"]
[[package]] [[package]]
name = "setuptools" name = "setuptools"
version = "75.4.0" version = "75.5.0"
description = "Easily download, build, install, upgrade, and uninstall Python packages" description = "Easily download, build, install, upgrade, and uninstall Python packages"
optional = false optional = false
python-versions = ">=3.9" python-versions = ">=3.9"
files = [ files = [
{file = "setuptools-75.4.0-py3-none-any.whl", hash = "sha256:b3c5d862f98500b06ffdf7cc4499b48c46c317d8d56cb30b5c8bce4d88f5c216"}, {file = "setuptools-75.5.0-py3-none-any.whl", hash = "sha256:87cb777c3b96d638ca02031192d40390e0ad97737e27b6b4fa831bea86f2f829"},
{file = "setuptools-75.4.0.tar.gz", hash = "sha256:1dc484f5cf56fd3fe7216d7b8df820802e7246cfb534a1db2aa64f14fcb9cdcb"}, {file = "setuptools-75.5.0.tar.gz", hash = "sha256:5c4ccb41111392671f02bb5f8436dfc5a9a7185e80500531b133f5775c4163ef"},
] ]
[package.extras] [package.extras]
@ -976,6 +1070,17 @@ files = [
{file = "shiboken6-6.8.0.2-cp39-abi3-win_amd64.whl", hash = "sha256:b11e750e696bb565d897e0f5836710edfb86bd355f87b09988bd31b2aad404d3"}, {file = "shiboken6-6.8.0.2-cp39-abi3-win_amd64.whl", hash = "sha256:b11e750e696bb565d897e0f5836710edfb86bd355f87b09988bd31b2aad404d3"},
] ]
[[package]]
name = "sniffio"
version = "1.3.1"
description = "Sniff out which async library your code is running under"
optional = false
python-versions = ">=3.7"
files = [
{file = "sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2"},
{file = "sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc"},
]
[[package]] [[package]]
name = "strip-ansi" name = "strip-ansi"
version = "0.1.1" version = "0.1.1"
@ -1084,4 +1189,4 @@ type = ["pytest-mypy"]
[metadata] [metadata]
lock-version = "2.0" lock-version = "2.0"
python-versions = ">=3.9,<3.13" python-versions = ">=3.9,<3.13"
content-hash = "c1c1f32bef21cdea01a5b4b309bc1dfdc5766259cb552c7f38fe8ea8de6e9c38" content-hash = "5d1ff28aa04c3a814280e55c0b2a307efe5ca953cd4cb281056c35fd2e53fdf0"

View file

@ -52,10 +52,14 @@ pytest-qt = "^4.2.0"
pytest-cov = "^5.0.0" pytest-cov = "^5.0.0"
strip-ansi = "*" strip-ansi = "*"
pytest-subprocess = "^1.5.2" pytest-subprocess = "^1.5.2"
pytest-rerunfailures = "^14.0"
[tool.poetry.group.container.dependencies] [tool.poetry.group.container.dependencies]
pymupdf = "1.24.11" # Last version to support python 3.8 (needed for Ubuntu Focal support) pymupdf = "1.24.11" # Last version to support python 3.8 (needed for Ubuntu Focal support)
[tool.poetry.group.dev.dependencies]
httpx = "^0.27.2"
[tool.isort] [tool.isort]
profile = "black" profile = "black"
skip_gitignore = true skip_gitignore = true

View file

@ -335,6 +335,7 @@ class TestCliConversion(TestCliBasic):
class TestExtraFormats(TestCli): class TestExtraFormats(TestCli):
@for_each_external_doc("*hwp*") @for_each_external_doc("*hwp*")
@pytest.mark.flaky(reruns=2)
def test_hancom_office(self, doc: str) -> None: def test_hancom_office(self, doc: str) -> None:
if is_qubes_native_conversion(): if is_qubes_native_conversion():
pytest.skip("HWP / HWPX formats are not supported on this platform") pytest.skip("HWP / HWPX formats are not supported on this platform")