Document the accepted MVP and retire superseded development plans
Rust development slice / linux (push) Failing after 14s

This commit is contained in:
2026-11-18 17:21:12 +01:00
parent f37b6dc4f7
commit 6963ad332c
188 changed files with 370 additions and 34933 deletions
+3 -430
View File
@@ -1,432 +1,5 @@
# Root-helper APT transaction provider / explicit first native acceptance
# APT-Transaktionsprüfungen
**Latest 2026-11-13:** The separately WebUI-approved configure-only attempt
`33775478…` reached APT 0 and completely installed fixture 1.0, but old provider
`5ff260…` rejected acceptance on the actual host reboot flag. Its private worker
reported false; the historical job had no HOST-before sample. It remains
**unknown** (`FIXTURE_UNEXPECTED_REBOOT_REQUIRED`), with no acceptance record,
Root scope closed and APT endpoint stopped. No before measurement or successful
acceptance is inferred retrospectively.
Dieses Verzeichnis enthält den lokalen APT-Prüfprovider und den Generator seiner Testpakete. Seine eigenen Tests liegen unter `tests/apt_transaction` und prüfen beispielsweise, welche Paketaktionen zur freigegebenen Transaktion gehören. [Provider](apt_transaction.py) · [Paketgenerator](build_fixture.py) · [Tests](../../tests/apt_transaction/test_guard.py)
New source implements an original-bound readonly failed-effect classification
and a parent-namespace host-before/after contract for future jobs, matching
Spec 1.13. Ambient true/true is reported honestly; changed host flags, private
worker flags or changed boot reject acceptance. All **81 Python tests passed**
in the readonly unprivileged offline container, including actual APT argument
regressions with guarded fake effects and unchanged real/copied status. Native
classification/new-runtime 1.1 acceptance remain pending. See
[`HELPER-CONTRACT.md`](HELPER-CONTRACT.md) for the exact versioned contract.
**Earlier 2026-11-13:** The Cloud fixture retry reached armed hook and APT exit 0,
leaving fixture 1.0 unpacked/unconfigured. An original-bound readonly proof and
normal signed RootSync now classified that original attempt as **failed**
(`APT_PARTIAL_UNCONFIGURED_VERIFIED`), preserving all histories/tombstones.
No native acceptance record or package repair exists. Root test policy remains
closed. Detailed real evidence: `../../docs/implementation-status.md`.
The configure-only recovery path, requiring its own new fixed WebUI approval, is implemented in source
(schema 4 manifest, transaction schema 2), with a dedicated passive recovery
projection and separate immutable recovery records. Its qualified runtime is now installed and its actual Cloud copied-database
sandbox readiness probe passed, preserving original state. **No configure-only
package effect or native package acceptance has yet run.** All **75 Python tests passed** in the offline readonly
unprivileged container, including actual APT 2.6.2 configure-only protocol and
guarded fake-dpkg arguments with unchanged real/copied status. See
[`HELPER-CONTRACT.md`](HELPER-CONTRACT.md) for the precise recovery boundary.
Source configuration now disables APT's `dpkg::selection::remove::approved`
rewrite, retaining the exact package's Configure operation while both pending
modes remain disabled and hook-pinned. Upstream explanation:
[APT 2.7.14 dpkgpm.cc](https://github.com/Debian/apt/blob/2.7.14/apt-pkg/deb/dpkgpm.cc#L1626-L1657).
Local actual APT 2.6.2 regression in an offline readonly unprivileged container
proved unpack + exact `--configure rmm-mvp-test:all`, no `--pending` or foreign
target; independent dpkg guard simulated effects, real/copied status unchanged.
67 Python tests passed with this opt-in regression. **This correction is now installed and sandbox-probed on the Cloud VM;
its native package effect remains unaccepted.** No automatic repair,
marker reset or implicit unknown-resolution is introduced. Older sections below
describe historical increments; current operational status is the linked log.
Reproduce that **local** configure regression using the existing development
image with APT 2.6.2 (no host APT command):
```sh
docker run --rm --read-only --network none --user 1000:1000 \
--tmpfs /tmp:rw,exec,nosuid,nodev,mode=1777 \
--tmpfs /tmp/opencode:rw,exec,nosuid,nodev,mode=1777 \
-v "$PWD/tools/apt-transaction:/src/tools/apt-transaction:ro" \
-v "$PWD/tests/apt_transaction:/src/tests/apt_transaction:ro" \
-v "$PWD/deploy/linux:/src/deploy/linux:ro" -w /src \
-e PYTHONDONTWRITEBYTECODE=1 -e RMM_APT_ARGUMENT_REGRESSION=1 \
rust-discovery-dev-build:local \
python3 -m unittest discover -s tests/apt_transaction -p 'test_*.py' -q
```
Current dedicated runner/probe/profile/agent integration and executed local
checks: [`../../deploy/linux/apt-helper-README.md`](../../deploy/linux/apt-helper-README.md).
The general helper unit remains unchanged; actual native VM measurements are
still required. Readiness alone advertises no accepted package capability.
**Helper integration contract update:**
[`HELPER-CONTRACT.md`](HELPER-CONTRACT.md) defines the implemented Rust helper
bridge, root installation/store layout, schema-2 embedded manifest and exact
remaining additive domain fields. It supersedes the older standalone signed
helper-action example below: signed-action.json must now contain the **full
original JobPayload bytes**, with `action.apt_manifest`, including exact
`transaction_bytes`. No detached/unsigned manifest is accepted. No agent
capability or native acceptance follows from this bridge build.
**Implemented, locally tested; NOT integrated or VM-tested.** No host package
installation, VM access, real native acceptance, production evidence, grant
issuance or WebUI approval has been performed here. The native process path is
real code (unshare/mount/capabilities/seccomp/APT/dpkg), not merely a parser;
its root/native execution is still unverified. Local root/installer execution
in tests is explicitly mocked. No real acceptance record was created.
Python >=3.9, standard library; OS dependencies: APT/dpkg, util-linux
unshare/mount, libseccomp.so.2, Linux x86-64 namespaces/seccomp/capabilities.
Only Ubuntu 24.04 amd64 is eligible; this is not a support/acceptance claim.
No Rust API, Cargo workspace changes, network endpoint, signature verifier,
grant service or automatic retry is included.
## The circular gate is replaced, not bypassed
There are now two distinct helper-only CLI operations:
```
/usr/bin/python3 -I /usr/lib/rmm-apt-transaction/apt_transaction.py native-acceptance
/usr/bin/python3 -I /usr/lib/rmm-apt-transaction/apt_transaction.py execute
```
* `native-acceptance`: dedicated **dev-test** action, exactly one code-pinned
fixture install OR upgrade, its own fixed job/attempt and actual WebUI approval.
It permits only absence of a *previous* native record for that fixture recipe.
It uses the same signed binding, archives, native sandbox, hooks, locks,
deadlines and independent verification as normal execution. No force/insecure/
skip-validation flag exists.
* `execute`: requires an existing matching record **and** a freshly verified,
separately UI-approved normal job/startgrant. An acceptance record never
substitutes for any signature or authorization.
**Scope matrix:** records produced here authorize only their exact fixture
recipe, package closure, runtime identity and **dev-test** profile. Production
is rejected (`PRODUCTION_ACCEPTANCE_REQUIRED`), even for a test record. General
catalog packages are rejected (`FIXTURE_RECIPE`); the bootstrap does not certify
arbitrary packages or widen provider support. Supporting those needs a separate
reviewed native-acceptance policy/provider revision, not relabelling this record.
The legacy unsigned administrator-created `native-acceptance.json` schema 1
is no longer used. Old helper contexts (schema 1) are rejected as well.
Each of these is a separate, one-time bootstrap recipe (recipe_version `1`):
| recipe_id | exact transition | architecture | origin |
|---|---|---|---|
| `rmm-apt-fixture-install-1.0` | absent -> 1.0 | all | rmm-test |
| `rmm-apt-fixture-update-1.1` | 1.0 -> 1.1 | all | rmm-test |
Exactly one package, `rmm-mvp-test`, no dependencies/scripts/triggers. One job
never installs both versions. Upgrade requires another fixed UI-approved job.
Pinned deterministic Debian ar/USTAR archives, **20,672 bytes each**:
```
1.0 74036ae81223deab3aae994fd2a2a695af408741777e1ce17dcb4e32f1235867
1.1 ef67c9fe0ba1443fcfe465768e1204763063c369a2a1da04af6027d8a9c00491
```
`build_fixture.py EXISTING_PRIVATE_OUTPUT_DIR` creates those exact bytes with
fixed metadata. It refuses overwrites and never installs/imports/publishes.
Real local dpkg-deb control/payload inspection confirms the only installed file
is `/usr/share/rmm-mvp-test/version`, with the selected version plus newline.
The builder now uses deterministic stdlib ar/USTAR construction, not the previous
timestamp-dependent `dpkg-deb --build` output. Older built archives do **not**
qualify for bootstrap; import these pinned bytes manually into the approved
catalog, verify and seal them before requesting the final 60-second grant.
## Mandatory signed-action and verified-helper contract
This module does **not** verify signatures. The root helper is the trust boundary.
Before writing any handoff files or invoking either mode, it MUST independently:
1. Verify the signed grant type/schema/algorithm/key, online validity and actual
human WebUI authorization (or explicitly labelled allowed dev-test UI
simulation), tenant/asset/agent identity and generation/assignment bindings.
2. Verify plan hash, job payload hash **of exact bytes**, fixed job/attempt,
action, execution profile, recipe/version, code/hook/config/runtime hashes,
exact complete resolved transaction and deadlines against that signed grant.
3. Enforce an explicit helper operation allowlist: `apt-native-acceptance` only
from the dedicated dev-test fixture-bootstrap action, never a generic
diagnostic/shell/API request. `apt-execute` must not invoke bootstrap mode.
4. Recheck trusted clocks/current rights and durably consume the grant and
attempt in its parent journal BEFORE spawning. Reject previous/uncertain
attempts. Unprivileged network/MCP callers cannot write these files or invoke
an unrestricted privileged subprocess API. Preserve peer-credential checks.
### Signed job payload: exact schema 1
The following object itself is the **signed/grant-hash-bound job payload**,
stored unchanged as `active/signed-action.json`; it is NOT an extra unsigned
root manifest or a caller's `verified: true` assertion. Fields are mandatory:
```json
{
"type": "rmm.apt.helper-action",
"schema_version": 1,
"action": "apt-native-acceptance",
"execution_profile": "dev-test",
"transaction_sha256": "<SHA256 of exact expectation.json bytes>",
"resolved_apt_transaction": "<complete OBJECT defined below, not a string/reference>",
"provider_sha256": "<installed apt_transaction.py SHA256>",
"hook_sha256": "<fixed installed hook SHA256>",
"apt_config_sha256": "<apt_config() bytes SHA256>",
"runtime_sha256": "<canonical runtime_identity() SHA256>",
"test_result_id": "<fixed acceptance evidence ID>"
}
```
The string placeholder for `resolved_apt_transaction` above MUST be replaced
with the complete nested object in actual payloads. Normal mode uses action
`apt-execute`. Production is a recognized profile but cannot use these test
records. The helper must verify the signed action, not merely trust ownership
of a separately supplied hash. The provider rechecks exact signed-payload hash,
type/schema/action/profile, nested transaction equality and code/config/runtime
bindings; it relies on the helper for cryptography and grant consumption.
### ResolvedAptTransaction schema 1
All fields mandatory; unknown fields/types and duplicate JSON keys are rejected:
```
schema_version: integer = 1
recipe_id: one exact fixture recipe above
recipe_version: "1"
platform: "ubuntu-24.04-amd64"
resolver_config_version: "rmm-apt-local-v1"
no_reboot: true
status_fingerprint: 64 lowercase hex characters
baseline_sha256: 64 lowercase hex characters
packages: [PackageArchive] # exactly one for this accepted scope
```
`PackageArchive` exact fields:
```
name: "rmm-mvp-test"
architecture: "all"
old_version: null for install, "1.0" for update
new_version: "1.0" for install, "1.1" for update
origin: "rmm-test"
sha256: corresponding code-pinned hash above
size: 20672
archive_path: "/var/lib/rmm-apt-transaction/archives/<same sha256>.deb"
```
The general parser still understands up to 128 package changes / 1 MiB, but that
does not authorize a broader execution scope. Frozen dataclasses/tuple closure;
strict Debian version ordering via read-only typed `dpkg --compare-versions`.
Status fingerprint hashes raw status/architecture/extended_states and trigger
database state. Dirty dpkg updates, pending triggers, repair, reinstall,
downgrade, removal, crossgrade and core/kernel/boot packages are unsupported.
`active/baseline.json` is the complete canonical all-package snapshot from
`installed_snapshot()` (`dpkg-query`), not just the changed package:
```json
{"package:architecture":{"version":"<version>","status":"install ok installed"}}
```
Its canonical byte SHA256 binds `baseline_sha256`. A new install requires actual
absence, not leftover config-files. Existing consistent noninstalled entries may
be present for other packages. Fingerprint/baseline are rechecked in the native
hook, not accepted from simulation as authorization.
### VerifiedHelperContext schema 2
Root helper writes `active/context.json` only after the checks/consumption above:
```json
{
"schema_version": 2,
"transaction_sha256": "<exact expectation.json bytes SHA256>",
"job_payload_sha256": "<exact signed-action.json bytes SHA256>",
"action": "apt-native-acceptance",
"execution_profile": "dev-test",
"effect_armed_ledger_id": "<stable parent journal row>",
"grant_id": "<verified grant ID>",
"job_id": "<fixed job ID>",
"attempt_id": "<fixed attempt ID>",
"authorization_id": "<UI authorization ID>",
"plan_sha256": "<verified plan hash>",
"not_before_ms": 1794484800000,
"start_before_ms": 1794484860000,
"boot_id": "<current boot UUID>",
"monotonic_start_before_ns": 123456789000
}
```
UTC integer milliseconds; start window <=60,000ms, exclusive upper deadline.
Helper translates that same verified deadline to same-boot CLOCK_MONOTONIC ns;
never grants another fresh window. Both clocks/boot are checked after expensive
verification, before native launch and at the final hook gate. An installation
validly started is not aborted merely because its start deadline later expires.
## Deployment / exact invocation
Planned, **not deployed here**. Root helper/operator installs reviewed files:
* `apt_transaction.py` -> `/usr/lib/rmm-apt-transaction/apt_transaction.py`, 0644
* fixed `hook` -> `/usr/lib/rmm-apt-transaction/hook`, 0755
* `/var/lib/rmm-apt-transaction` and `active`, 0700, root-owned
* sealed archives in `archives/`, root-owned; no nonroot writer
* active expectation/context/baseline/signed-action/apt.conf files, 0600
* empty root-owned active `empty`, `lists`, `cache` directories, 0700
All path components must be real root-owned nonsymlink/non-group/world-writable
paths. `apt.conf` must equal `apt_config()` bytes. Read-only `print-config` emits
them; root-only `inspect-runtime` emits canonical measured runtime identity for
the helper/planner's reviewed signed payload, not an acceptance claim.
For **either** operation, stdin is exactly:
```json
{"transaction_sha256":"<same approved hash>","effect_armed_ledger_id":"<same parent row>"}
```
Use sanitized environment and close inherited private fds. No secrets in argv.
An acceptance request must invoke `native-acceptance`; a normal signed action
must invoke `execute`. Mixing the two is rejected. Internal `hook` and
`sandbox-worker` modes are not exposed as privileged IPC operations.
## Native pipeline and evidence publication
Both modes run the same pipeline:
1. Root/signed action/recipe/runtime/preconditions/actual archive validation;
constant hook/config/code binding before APT. Native provider flock only
serializes this provider, never replaces APT's locks.
2. Bootstrap additionally writes/fsyncs O_EXCL
`bootstrap-attempt-<fixed recipe_id>.json` in BASE, independently of active/
and grant IDs. One-time per recipe; no automatic deletion/reset/retry, even
after a failed native precheck. An operator must reconcile failures and use
a separately reviewed lifecycle/revision, not remove tombstones to retry.
3. Separate native child via fixed `unshare --mount --pid --fork --mount-proc
--net`, private propagation, tmpfs `/run`/`tmp`/`var/tmp`/`dev`, protected
readonly/masked kernel/boot/control-plane/tool/archive paths.
4. Drop bounding/effective/permitted capability sets to exactly filesystem/UID
subset `{0,1,3,4,6,7,8,31}`; inherited/ambient empty. No SYS_BOOT/SYS_ADMIN/
SYS_PTRACE/KILL/RAWIO/NET_ADMIN/MKNOD. Set NoNewPrivileges, load inherited
seccomp denying reboot/kexec, module loading, mount/namespace changes,
ptrace/process writes and other kernel-control operations.
5. **Actual native prechecks inside that child:** different PID/mount/net ns,
expected namespace PID1, exact caps/NNP/Seccomp, readonly mount flags, tmpfs
identity, masked sysrq, no conventional systemd/dbus/helper sockets; invalid
reboot magic and invalid kexec flags/fds must return EPERM. These calls never
contain valid reboot magic/commands or valid kexec input. No PID1 signals.
6. Hold real existing `/var/lib/dpkg/lock-frontend` with fcntl; a separate native
`apt-get -o DPkg::Lock::Timeout=0 check` must report that frontend-lock conflict
(100 + expected lock diagnostic). `check` is read-only, not an installation
probe. Never delete locks, kill owners or use simulation to prove locking.
A busy foreign lock vetoes the bootstrap. Release only this process's fd.
7. Fsync native-prechecks record, then sandbox-ready marker. Native APT uses
exact local archives, `--no-remove --no-install-recommends` and
constant isolated config. No global APT hooks/sources/preferences; reviewed
dpkg global config is runtime-hash-bound in the signed action. Native foreign
lock wait remains bounded to 10 seconds.
**2026-11-13 native diagnosis:** `--no-download` prevented APT's local file
acquisition and emitted `**ERROR**` instead of the absolute archive filename
in protocol v3. The provider permits local acquisition of the presealed
archive; the AF_UNIX-only private worker still rejects actual IPv4/IPv6
socket creation. Both denials are now checked inside worker/hook/probe.
A permitted IP socket or unrelated resource error vetoes; this is not a
network-download allowance or acceptance/manifest/hash/path bypass.
8. Mandatory v3 pre-dpkg hook repeats signed/runtime/scope/sandbox checks,
validates the whole actual action set, actual hashes/control/payload,
status/baseline and final deadline; O_EXCL + file/directory fsync effect-armed
marker **before returning 0**. Second hook is vetoed. No fallback.
Immediately after successful file/directory fsync, UTC/monotonic deadline and
boot identity are checked again. Expiry vetoes the hook while retaining its
armed marker: conservative unknown/reconciliation, never marker reuse.
9. After native child exit 0 and valid bound markers/result, independently query
**all** installed versions/statuses, detect unexpected changes, require empty
`dpkg --audit` and exact installed fixture version-file bytes. Private /run
reboot flag is combined with host flag. Bootstrap requires no reboot flag.
10. Only then publish root-owned 0600 schema-2
`acceptance-<fixed recipe_id>.json`, staging/fsync complete bytes before
exclusive publication and directory fsync. No failed/stale-start/partial/
unknown execution is promoted. Parent must treat publication/IO uncertainty
as unknown and reconcile; a record is never an excuse to retry that job.
Record binds exact approved recipe/packages, full measured runtime (tool, hook,
APT config, resolved binaries including APT/dpkg/dpkg-deb/dpkg-query/Python/
unshare/mount/shell/libseccomp hashes and version strings, OS/kernel identity,
dpkg config fingerprint), transaction/job-payload/plan hashes, grant/job/attempt/
authorization/test-result IDs, precheck/hook/postcondition hashes and completion
time. It explicitly contains `test_acceptance: true`, `noProductionEvidence: true`.
Install and upgrade have separate records; one does not certify the other.
## Result / safety bounds
One JSON result on stdout: `effect_state` = `known_failed`, `verified`, or
`unknown`; stable `code`; structured `evidence`. Every structured bootstrap
result, including veto/unknown, carries `evidence.test_acceptance=true` and
`evidence.noProductionEvidence=true`. Verified bootstrap additionally reports
record hash/test-result ID. Exit 0 means structured result, not job success.
Malformed CLI/hook input exits 100 with generic diagnostics. Parent must journal
missing output/death/partial markers after possible launch as unknown.
No `awaitreboot` output in this version: no post-boot completion condition exists.
No reboot is invoked. Any reboot/follow-up/retry needs another fixed approved job.
No auto-configure-all, autoremove, dist-upgrade, repair, index refresh, installer
timeout killing, or automatic record/attempt clearing is provided.
APT retains its frontend lock and normally hands off the inner dpkg lock.
**No claim that both locks are held during the hook.** Direct foreign dpkg and
malicious root are outside the frontend-lock guarantee; native inner locking
still applies. Root scripts are intrinsically trusted host-filesystem code;
this is not general malicious-root containment or arbitrary deferred-side-effect
verification. Bootstrap avoids that expansion by accepting only pinned
script/trigger/dependency-free fixtures. Existing dpkg triggers/global config,
kernel, root administrators, trusted binaries/helper remain in the TCB.
## Primary sources and actual local verification
APT 2.7.14 sources read before parser implementation:
* <https://github.com/Debian/apt/blob/2.7.14/doc/apt.conf.5.xml>
* <https://github.com/Debian/apt/blob/2.7.14/apt-pkg/deb/dpkgpm.cc>
* <https://github.com/Debian/apt/blob/2.7.14/apt-pkg/deb/debsystem.cc>
Native read-only lock probe checked against `DoCheck` (`Cache.Open/CheckDeps`,
not dpkg/install) in:
<https://github.com/Debian/apt/blob/2.7.14/cmdline/apt-get.cc>.
Hook protocol **VERSION 3**, nine fields:
`name old-version old-arch old-multiarch direction new-version new-arch
new-multiarch action`. New install old triple `- - none`; `<` install/upgrade,
`>` downgrade/removal; action absolute archive path or `**CONFIGURE**`/
`**REMOVE**`. Config lines precede a blank separator and support percent escaping
and repeated unnamed lists. SHA/size/origin are NOT transmitted in v3: actual
sealed files are independently checked. Whole closure must match, including one
configure per unpack; extra/missing/repeated actions are vetoed.
`tests/apt_transaction/fixtures/install-v3.txt` is source-derived, **not a native
installation capture**.
Actually executed unprivileged, with **no host package mutation**:
```
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests/apt_transaction -p 'test_*.py' -v
```
**39 tests passed** at the documentation update. Original 22 guard/archive/
parser/preconditions tests plus pinned fixture/control inspection, context/
signed manifest/recipe/profile/runtime binding, native-record matrix, fsync/
partial/negative evidence rejection, mocked full driver success/unknown/failure/
stale/redelivery, mocked native mount/capability/namespace/socket introspection,
and local unprivileged seccomp invalid-input EPERM probes.
The post-fsync deadline regression advances mocked UTC/monotonic clocks (both
and each independently), observes nonzero hook outcome/no modeled dpkg start,
and verifies that the durable marker survives and prevents a second handoff.
Native root/installer/VM parts are explicitly mocked under private temporary
BASE paths; mock-generated records are deleted with those dirs and are NOT
native evidence. Safe invalid-input probes and seccomp loading really ran in
short-lived unprivileged children. No valid reboot/kexec operation or host PID1
signal was attempted. Deterministic .deb bytes were really built/read/inspected,
never installed. Full root sandbox, real APT lock conflict, native fixture
install/update, helper signature integration and VM acceptance remain **unrun**.
Im MVP planen WebUI und MCP Linux-Wartung als Skriptauftrag. Ein zur Wartung berechtigter Benutzer reicht ein Shellskript für das Zielgerät ein. Die WebUI zeigt Skript und Rechte vor der persönlichen Entscheidung. Für den Ablauf im Produkt gilt der [Geräte- und Skriptvertrag](../../proto/BACKEND_CONTRACT.md).