Files
BuTzZ 6c3203d725
Rust development slice / linux (push) Failing after 1m10s
feat: snapshot Linux-first RMM MVP and pause handover
2026-11-12 22:06:25 +01:00

312 lines
19 KiB
Markdown

# Agent/helper runtime — integrated source, VM acceptance pending
This is the current runtime contract. It supersedes the earlier module-only
handoffs in `README.md` and `tests/agent_sync/README.md`. No installation, target
job, service/reboot/package mutation, SSH tunnel or server restart was performed
by this integration. Actual VM execution/acceptance belongs to the next worker.
Nothing here approves an RMM job or substitutes for the WebUI approval boundary.
## Binaries and complete CLI/environment
Both binaries support `--help`; unknown arguments fail closed. Normal agent
operation has no positional arguments. Planned bootstrap invocation:
```text
RMM_AGENT_STATE=/var/lib/rmm-agent
RMM_CA_FILE=/etc/rmm/ca.pem
RMM_CONNECT_ADDR=127.0.0.1:18443
RMM_TRANSPORT=https
/usr/local/libexec/rmm-agent
```
These are environment assignments, not an instruction to source a credential
file. The trusted CA and TLS name **rmm-dev.internal** are always checked. TCP
override changes neither DNS/SNI nor certificate verification. No proxy trust,
invalid-certificate bypass or public Docker port is introduced.
| Variable | Meaning/default |
|---|---|
| `RMM_AGENT_STATE` | Private 0700 owned agent state; default `devstate/agent` |
| `RMM_CA_FILE` | Public trusted CA; default `devstate/pki/ca.pem` |
| `RMM_CONNECT_ADDR` | Parsed TCP SocketAddr; default `127.0.0.1:18443` |
| `RMM_TRANSPORT` | `auto` (default), `wss`, or `https`; HTTPS can be explicitly selected for VM acceptance |
| `RMM_ENROLLMENT_TOKEN_FILE` | First registration only; remove the bootstrap token after enrollment |
| `RMM_JOB_BINDING_FILE` | Optional private agent-owned JSON Binding; absent means passive, no job capabilities |
| `RMM_JOB_SIGNING_KEY_BOOTSTRAP=1` | Authenticated device-mTLS GET, using the current rotation-store identity; pins without replacing existing trust |
| `RMM_HELPER_SOCKET` | Configured general helper socket, e.g. `/run/rmm-helper/control.sock` |
| `RMM_HELPER_POLICY_FILE` | Root-owned, agent-readable public general policy pin; required with general socket |
| `RMM_APT_HELPER_SOCKET` | Dedicated fixed `/run/rmm-apt-helper/helper.sock`, never generic fallback |
| `RMM_APT_HELPER_PROOF_FILE` | Root-owned public APT stack/profile/identity pin; see `apt-helper-README.md` |
| `RMM_LOG_UNITS_FILE` | Private agent-owned JSON array of exact allowed service units for unprivileged live logs |
| `RMM_EXTENDED_INVENTORY` | `1` by default with the now-compatible backend, `0` for legacy base-only JSON |
| `RMM_AGENT_PROFILE` | `development` required only for the rotation-test interval override |
| `RMM_ROTATION_TEST_AFTER_SECONDS` | Positive development-only certificate-age trigger; never bypasses chain/expiry/EKU/key/identity verification |
Binding fields: tenant_id, asset_id, agent_id, enrollment_generation,
assignment_version, public_key_hex. Device/asset must match enrollment. Empty key
is allowed only with authenticated signing-key bootstrap; later startup must
reauthenticate that bootstrap or use a fully provisioned binding. Key response is
Ed25519/schema 1/raw 32-byte key with SHA-256 key ID, bounded and no redirects.
The immutable agent cache is `job-signing-key.json`. Signing-pin replacement and
identity/assignment recovery are controlled provisioning operations, not automatic
trust in a key from a job. Helper trust stays independently root-provisioned.
The unit leaves the job-binding optional in `/etc/rmm/agent.env`, allowing a first
passive enrollment. Set binding/key-bootstrap/helper variables only after the
authenticated operator resolves the enrolled IDs and provisions their pins.
Do not delete an old identity/journal to get a new first-session shortcut.
## Integrated transport and sync
One bounded actor and one dispatcher serve WSS and HTTPS with identical envelopes,
journal, result receipts, cancellation and signature checks. `auto` tries WSS
first and falls back only after two typed HTTP upgrade-block classifications;
401/403, TLS/identity failures and ambiguous errors never select a bypass.
WSS Ping/Pong are handled. HTTPS uses the existing MessageBatch/25-second poll
contract. Poll cancellation cannot create a new attempt; fresh heartbeat IDs/
sequences are never reused or queued as offline telemetry. Actor queues are bounded.
Every authenticated Hello advertises `rmm.sync.v1` and closes all local effect
gates. Agent obtains actual root snapshots, durably reserves the fresh SyncRequest,
checks exact signed SyncResponse bytes and all device/channel/boot/checkpoint/
request-hash/time bindings, commits targeted journal/receipt changes, applies its
own consumption decisions, forwards the **original** signed response plus original
request protobuf to every configured helper, then confirms actual applied root
checkpoints. No generic ready boolean opens either process.
Both root profiles share actual DB/lock inodes. They join an unchanged pending
root capture created after both process births, so ONE signed response/checkpoint
can be independently applied by both. A new process never inherits readiness
from the shared DB. Stale process captures are fenced; known outcomes may unlock
specifically while an unknown retains the resource and global gates stay closed.
Consumed grant/attempt tombstones are never removed.
Scheduler AND immediate worker effect admission check the current signed lease;
the worker also rechecks fresh grant/time and the disconnect/rotation atomic fence.
Helper checks its own process gate under the same executor guard before consumption.
Unknown, missing authoritative history, clock failure and rollback remain closed.
Clock/helper sync failure does not stop independent passive heartbeat/inventory
traffic. Sync retries are bounded, happen on fresh inventory/result ACK and on a
lease timer, and never retry a lost permit request as a new grant.
After reboot, old/new boot ID alone is not success. Fresh current-boot inventory
is durably ACKed, a new signed decision must bind the persisted original boot and
current authenticated boot plus inventory evidence, then each helper independently
applies that specific canonical terminal outcome/unlock. Thus an initial negative
sync does not deadlock waiting forever without the post-inventory resync.
Result envelopes are archived before concrete ACK deletion in one SQLite
transaction. Historical message identity/output is retained for reconciliation.
Inventory mapping preserves source/time/quality/completeness and unsupported states.
Live journal permission warnings become permission_denied, not a fake empty success.
## Helper bootstrap/policy inspection
Helper normal daemon: `RMM_HELPER_CONFIG=/etc/rmm/helper.json` (root-owned 0600).
Both profiles use `/var/lib/rmm-helper/consumption.sqlite` and `executor.lock`.
Helper config uses the existing binding, optional independent signing_key_file,
agent_uid, socket_gid, socket, shared state, allowed_services, allow_reboot,
allow_root_scripts; APT adds its existing sealed apt_provider configuration.
Public general pin, **planned command, not executed on VM here**:
```text
RMM_HELPER_CONFIG=/etc/rmm/helper.json /usr/local/libexec/rmm-privileged-helper --inspect-general-profile
```
Install the exact JSON as root:rmm-agent 0640 in non-writable root-owned parents,
then set RMM_HELPER_POLICY_FILE. It binds profile, device identity/key and exact
policy hash. General policy probe uses root socket/peer credentials and a fresh
challenge; capabilities reflect actual policy/interpreter availability. Root
scripts remain default-off broad administration. Services are exact allowed
running/stopped units with >=180s runtime, reboot separately enabled with >=60s.
Root jobs are NEVER relabelled into agent_service.
APT install/probe/inspection modes and profile sandbox remain exactly documented
in `apt-helper-README.md`. Both daemons need this same installed build and reviewed
new proof hashes (binary bytes are part of the APT proof). Prefetched sealed
archives, receipts predating the grant and exact baselines are still prerequisites;
this integration introduces no unsafe generic APT download/install/simulation path.
Root native-acceptance and accepted-recipe/manifest capabilities come only from
authenticated real readiness. Successful native acceptance triggers a controlled
reconnect to advertise newly accepted recipes; periodic safe capability refresh
also reconnects rather than silently changing the Hello contract.
## Passive APT planning and declarative no-change
The normal extended inventory worker now runs at startup and every 60 seconds
(one inflight collector; not in the heartbeat handler). Unacknowledged durable
traffic backpressures new collection without deleting old data; job results and
control receipts retain priority. It includes the fixed
`linux.apt.planning` scope from `root-helper:apt_planning_v1`. This currently
refreshes the existing extended inventory scopes too; their actual individual
source timestamps/quality remain intact. There is no arbitrary package query,
requested live diagnosis or software effect hidden in this collector.
`apt_planning_v1` uses the existing UID-authenticated dedicated Unix socket,
fresh challenge, device/generation/signing-key identity, root peer and separately
root-provisioned stack pin. Its response is the readiness envelope with a typed
`planning` result and **empty capability/acceptance fields**. It does not depend
on a prior native-acceptance record and cannot create one. Root validates the
installed code, exact runtime/config policy pins and actual archive bytes, sizes,
ownership/modes/link count and private prefetch receipts. No artifact is fetched
or installed by this RPC. Administratively stage the known fixture archives via
`provision-fixture-store` before planning/the final 60-second grant.
The fixed metadata child has a 4-second whole-process/output budget (32 KiB),
the socket has 5-second IO limits, and root measurement timestamps are preserved.
Dirty dpkg updates/triggers, foreign locks, unsafe/missing receipts, permission
failure, timeout or missing helper yield an unknown observation with null data,
not fake hashes or accepted capability. Runtime policy drift fails closed.
Complete observations contain the suite's `fixture_package`, `baseline_sha256`,
`sealed_baseline_sha256`, `status_fingerprint`, `dpkg_updates_empty`,
`pending_triggers_empty`, `runtime`, `runtime_sha256`, `provider_sha256`,
`hook_sha256`, `apt_config_sha256` and `prefetch`. The full package baseline stays
root-private at `baselines/<sha256>.json`; only hashes, fixture projection and
minimal public runtime/artifact pins cross the wire. Stable sampling is checked
before and after content-addressed exclusive write/file+directory fsync; existing
identical seals are verified, never overwritten. This is passive metadata, not
job authority. The ordinary durable inventory outbox/ACK path persists it.
Normal accepted fixture `ensure_package`/`apply_updates` may return
`POSTCONDITION_ALREADY_MET`, `changed=false`, `already_desired=true`,
`postcondition_already_met=true`, `transaction_executed=false` only when the
**entire selected set** is independently installed at its exact target version
and architecture, clean, matches the current approved baseline/fingerprint and
has the expected fixture payload. Root signature, all original bindings,
expiry/sync/local-policy/resource gates, all manifest/code/runtime/artifact pins,
native accepted-recipe record and durable grant/attempt consumption remain
mandatory. Only the historical `old_version` condition is skipped in this
no-mutation branch. No APT/hook/native launch/effect marker is created. A foreign
lock, dirty/unknown state or changed baseline cannot become success; native
acceptance itself never uses this branch. Other versions still follow the
unchanged no-reinstall/no-downgrade/no-remove mutation guards.
VM suite owner must update its absent/1.0-only adapter: after native update to
1.1, a normal exact desired-1.1 job must use the accepted update recipe with a
fresh planning seal. A desired-1.0 job on 1.1 is still correctly denied; this
implementation does not authorize removal/downgrade to make a stale test pass.
No backend/domain/protobuf signatures or plan bytes have been changed.
Provider source changes change the embedded provider/helper/stack/runtime pins.
Deploy both helpers from this build and provision reviewed new proof/policy
pins **before** native acceptance; old native records cannot silently certify
new runtime bytes. Manifest schema 2 / transaction schema 1 / fixture archives
1.0 and 1.1 (20,672 bytes, unchanged hashes) remain unchanged.
These are implemented source/local-fixture semantics, not VM or MVP acceptance.
## Certificate rotation and recovery
`RMM_AGENT_STATE/certificates/current.json` is the authoritative atomic selector
over private versioned device key/cert files. Existing enrolled files initialize
the store exactly once; later connections/bootstrap use the selected identity.
Rotation checks chain, current validity, exact agent CN, clientAuth purpose and
public/private key match. It persists one pending key/CSR and retries those SAME
CSR bytes after response loss. A verified swap closes the old actor/effect fence
and requires a new Hello/Sync. Agent/enrollment IDs and independent job/helper
signing pins do not change. Expired/revoked/invalid certs remain blocked and need
authorized identity recovery; no auto-reenrollment or TLS downgrade.
## Verification boundaries / concrete remaining policy issue
Local Rust source/unit/integration tests cover actual loopback mTLS dispatcher
WSS/HTTPS clean reconnect, Ping, forged sync rejection, passive traffic with an
untrusted effect clock, journal reopen/replay and independent helper checkpoint/
outcome/boot checks. Unsafe OS providers are mocked or denied; no local service,
APT install or reboot was executed. This is not a real WebUI-approved VM result,
native package acceptance, production auth acceptance or full Linux-MVP sign-off.
Concrete backend policy limitation, not bypassed here: `RootScope::permits` in
`crates/server/src/root_policy.rs` does not match `Action::QueryLogs` and falls
through to false. The typed, bounded root helper log handler exists, and the
unprivileged allowlisted path is live, but a root live-log plan will be denied by
that backend policy until its owner adds an explicit permitted scope. Do NOT
work around it by relabelling identity, direct approval or a generic root shell.
## Executed local verification
Using `/home/Kevin/.cargo/bin/cargo`, actually passed during integration:
```sh
cargo test -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked
cargo clippy -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked -- -D warnings
cargo build -p rmm-agent -p rmm-privileged-helper --locked
cargo fmt -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux -- --check
python3 tests/agent_jobs/test_install.py
target/debug/rmm-agent --help
target/debug/rmm-privileged-helper --help
```
111 executed Rust test cases passed (including reused transport tests), two
create-only installer guard tests passed, both debug binaries built and both
help commands returned successfully without loading secret configuration. No
warnings/type errors were suppressed. A boxed tungstenite HTTP-response test
constructor was corrected, not bypassed. No coverage percentage was measured.
Coverage boundaries: real loopback TLS/dispatcher WSS and HTTPS reconnect and
forged sync rejection are exercised; helper cryptography/checkpoints/two-process
shared signature and replay/unknown/reboot decisions use real SQLite/signatures
with local fixtures. Native service/APT/reboot effects and positive root UID-0
daemon/VM RPC acceptance are NOT substituted by these tests. The built artifacts
are `target/debug/rmm-agent` and `target/debug/rmm-privileged-helper`; a reviewed
VM-compatible release artifact still needs the target worker's ELF/glibc/library
preflight and actual authorized VM run.
## Executed local verification — planning/no-change increment
After **all** source changes, the following complete targeted block passed with
`/home/Kevin/.cargo/bin` on PATH (2026-11-12):
```sh
cargo test -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests/apt_transaction -p 'test_*.py' -v
cargo clippy -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked -- -D warnings
cargo fmt -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux -- --check
cargo build -p rmm-agent -p rmm-privileged-helper --locked
cargo build -p rmm-agent -p rmm-privileged-helper --release --locked
python3 tests/agent_jobs/test_install.py
target/debug/rmm-agent --help
target/debug/rmm-privileged-helper --help
```
Results: **115 Rust tests**, **56 Python APT tests**, **2 installer guard tests**
passed; warning-denying all-target Clippy and formatting passed; both Debug and
Release binaries built; both help commands succeeded. Separate final AST parsing
passed for **8 Python files**. No full workspace/backend/web test suite was run.
New regressions cover shared Python/Rust canonical golden bytes/hash, receipt
and timestamp freshness, private content-addressed seal reuse/change without
full-baseline disclosure, sampling drift/dirty/foreign-lock/bad-artifact failure,
normal exact already-desired install/update postconditions, no launch/effect
markers, old-version mutation rejection, wrong architecture/newer version,
audit/payload/expiry rejection, no-change outcome binding and durable duplicate
grant/attempt rejection, honest missing-helper data and collection backpressure.
Root ownership/kernel state and unsafe native package processes are mocked;
existing harmless unprivileged seccomp tests really execute. No native host
package mutation, root systemd/probe/VM action, approval or reboot was performed.
Earlier intermediate failures were corrected, not suppressed: the inherited two
unformatted source edits, a mock reader missing the `limit` keyword argument,
and a newly inserted function accidentally inheriting a test-only cfg attribute.
The final complete block above passed after these corrections.
Final artifact SHA-256 (native developer-PC builds; **not** Bookworm/VM proof):
| Artifact | SHA-256 |
|---|---|
| `target/debug/rmm-agent` | `90a4eab064301740cd938a41a76a510ad3b4abc3e48a188ba12bfe22f123266c` |
| `target/debug/rmm-privileged-helper` | `54db2fe17841fef9af48bbeb446f45bda52f62e442a4900e0543e358095900b7` |
| `target/release/rmm-agent` | `eac5434fd7a1e509c0971e4e05d4e1aedbde1bb90f7d7a8f9ec4d48ab4d34597` |
| `target/release/rmm-privileged-helper` | `581a033478b41365066df2dc9ac0d11e314160909b5fab612be0be97be1bbbdc` |
| `tools/apt-transaction/apt_transaction.py` | `57b695bf57ea2703393e39f92a1b6ef2314b4838f3dfb29e4e70da79687c9987` |
Remaining: suite-owner adapter update for exact desired 1.1 after native upgrade,
reviewed deployment/new pins, actual dedicated-profile metadata RPC and native
UI-approved fixture/no-change VM proof, target ELF/library compatibility, and
VM resource/retention measurements for the new 60-second extended collection.
No real current baseline change was induced on the host/VM: freshness tests use
changed temporary/mock package state, not a disguised package installation.
Linux-MVP and production acceptance remain open. Backend Root QueryLogs policy
is owned by its separate worker and was neither changed nor reverified here.