# Agent/helper runtime — integrated source, VM acceptance pending This is the current runtime contract. It supersedes the earlier module-only handoffs in `README.md` and `tests/agent_sync/README.md`. No installation, target job, service/reboot/package mutation, SSH tunnel or server restart was performed by this integration. Actual VM execution/acceptance belongs to the next worker. Nothing here approves an RMM job or substitutes for the WebUI approval boundary. ## Binaries and complete CLI/environment Both binaries support `--help`; unknown arguments fail closed. Normal agent operation has no positional arguments. Planned bootstrap invocation: ```text RMM_AGENT_STATE=/var/lib/rmm-agent RMM_CA_FILE=/etc/rmm/ca.pem RMM_CONNECT_ADDR=127.0.0.1:18443 RMM_TRANSPORT=https /usr/local/libexec/rmm-agent ``` These are environment assignments, not an instruction to source a credential file. The trusted CA and TLS name **rmm-dev.internal** are always checked. TCP override changes neither DNS/SNI nor certificate verification. No proxy trust, invalid-certificate bypass or public Docker port is introduced. | Variable | Meaning/default | |---|---| | `RMM_AGENT_STATE` | Private 0700 owned agent state; default `devstate/agent` | | `RMM_CA_FILE` | Public trusted CA; default `devstate/pki/ca.pem` | | `RMM_CONNECT_ADDR` | Parsed TCP SocketAddr; default `127.0.0.1:18443` | | `RMM_TRANSPORT` | `auto` (default), `wss`, or `https`; HTTPS can be explicitly selected for VM acceptance | | `RMM_ENROLLMENT_TOKEN_FILE` | First registration only; remove the bootstrap token after enrollment | | `RMM_JOB_BINDING_FILE` | Optional private agent-owned JSON Binding; absent means passive, no job capabilities | | `RMM_JOB_SIGNING_KEY_BOOTSTRAP=1` | Authenticated device-mTLS GET, using the current rotation-store identity; pins without replacing existing trust | | `RMM_HELPER_SOCKET` | Configured general helper socket, e.g. `/run/rmm-helper/control.sock` | | `RMM_HELPER_POLICY_FILE` | Root-owned, agent-readable public general policy pin; required with general socket | | `RMM_APT_HELPER_SOCKET` | Dedicated fixed `/run/rmm-apt-helper/helper.sock`, never generic fallback | | `RMM_APT_HELPER_PROOF_FILE` | Root-owned public APT stack/profile/identity pin; see `apt-helper-README.md` | | `RMM_LOG_UNITS_FILE` | Private agent-owned JSON array of exact allowed service units for unprivileged live logs | | `RMM_EXTENDED_INVENTORY` | `1` by default with the now-compatible backend, `0` for legacy base-only JSON | | `RMM_AGENT_PROFILE` | `development` required only for the rotation-test interval override | | `RMM_ROTATION_TEST_AFTER_SECONDS` | Positive development-only certificate-age trigger; never bypasses chain/expiry/EKU/key/identity verification | Binding fields: tenant_id, asset_id, agent_id, enrollment_generation, assignment_version, public_key_hex. Device/asset must match enrollment. Empty key is allowed only with authenticated signing-key bootstrap; later startup must reauthenticate that bootstrap or use a fully provisioned binding. Key response is Ed25519/schema 1/raw 32-byte key with SHA-256 key ID, bounded and no redirects. The immutable agent cache is `job-signing-key.json`. Signing-pin replacement and identity/assignment recovery are controlled provisioning operations, not automatic trust in a key from a job. Helper trust stays independently root-provisioned. The unit leaves the job-binding optional in `/etc/rmm/agent.env`, allowing a first passive enrollment. Set binding/key-bootstrap/helper variables only after the authenticated operator resolves the enrolled IDs and provisions their pins. Do not delete an old identity/journal to get a new first-session shortcut. ## Integrated transport and sync One bounded actor and one dispatcher serve WSS and HTTPS with identical envelopes, journal, result receipts, cancellation and signature checks. `auto` tries WSS first and falls back only after two typed HTTP upgrade-block classifications; 401/403, TLS/identity failures and ambiguous errors never select a bypass. WSS Ping/Pong are handled. HTTPS uses the existing MessageBatch/25-second poll contract. Poll cancellation cannot create a new attempt; fresh heartbeat IDs/ sequences are never reused or queued as offline telemetry. Actor queues are bounded. Every authenticated Hello advertises `rmm.sync.v1` and closes all local effect gates. Agent obtains actual root snapshots, durably reserves the fresh SyncRequest, checks exact signed SyncResponse bytes and all device/channel/boot/checkpoint/ request-hash/time bindings, commits targeted journal/receipt changes, applies its own consumption decisions, forwards the **original** signed response plus original request protobuf to every configured helper, then confirms actual applied root checkpoints. No generic ready boolean opens either process. Both root profiles share actual DB/lock inodes. They join an unchanged pending root capture created after both process births, so ONE signed response/checkpoint can be independently applied by both. A new process never inherits readiness from the shared DB. Stale process captures are fenced; known outcomes may unlock specifically while an unknown retains the resource and global gates stay closed. Consumed grant/attempt tombstones are never removed. Scheduler AND immediate worker effect admission check the current signed lease; the worker also rechecks fresh grant/time and the disconnect/rotation atomic fence. Helper checks its own process gate under the same executor guard before consumption. Unknown, missing authoritative history, clock failure and rollback remain closed. Clock/helper sync failure does not stop independent passive heartbeat/inventory traffic. Sync retries are bounded, happen on fresh inventory/result ACK and on a lease timer, and never retry a lost permit request as a new grant. After reboot, old/new boot ID alone is not success. Fresh current-boot inventory is durably ACKed, a new signed decision must bind the persisted original boot and current authenticated boot plus inventory evidence, then each helper independently applies that specific canonical terminal outcome/unlock. Thus an initial negative sync does not deadlock waiting forever without the post-inventory resync. Result envelopes are archived before concrete ACK deletion in one SQLite transaction. Historical message identity/output is retained for reconciliation. Inventory mapping preserves source/time/quality/completeness and unsupported states. Live journal permission warnings become permission_denied, not a fake empty success. ## Helper bootstrap/policy inspection Helper normal daemon: `RMM_HELPER_CONFIG=/etc/rmm/helper.json` (root-owned 0600). Both profiles use `/var/lib/rmm-helper/consumption.sqlite` and `executor.lock`. Helper config uses the existing binding, optional independent signing_key_file, agent_uid, socket_gid, socket, shared state, allowed_services, allow_reboot, allow_root_scripts; APT adds its existing sealed apt_provider configuration. Public general pin, **planned command, not executed on VM here**: ```text RMM_HELPER_CONFIG=/etc/rmm/helper.json /usr/local/libexec/rmm-privileged-helper --inspect-general-profile ``` Install the exact JSON as root:rmm-agent 0640 in non-writable root-owned parents, then set RMM_HELPER_POLICY_FILE. It binds profile, device identity/key and exact policy hash. General policy probe uses root socket/peer credentials and a fresh challenge; capabilities reflect actual policy/interpreter availability. Root scripts remain default-off broad administration. Services are exact allowed running/stopped units with >=180s runtime, reboot separately enabled with >=60s. Root jobs are NEVER relabelled into agent_service. APT install/probe/inspection modes and profile sandbox remain exactly documented in `apt-helper-README.md`. Both daemons need this same installed build and reviewed new proof hashes (binary bytes are part of the APT proof). Prefetched sealed archives, receipts predating the grant and exact baselines are still prerequisites; this integration introduces no unsafe generic APT download/install/simulation path. Root native-acceptance and accepted-recipe/manifest capabilities come only from authenticated real readiness. Successful native acceptance triggers a controlled reconnect to advertise newly accepted recipes; periodic safe capability refresh also reconnects rather than silently changing the Hello contract. ## Passive APT planning and declarative no-change The normal extended inventory worker now runs at startup and every 60 seconds (one inflight collector; not in the heartbeat handler). Unacknowledged durable traffic backpressures new collection without deleting old data; job results and control receipts retain priority. It includes the fixed `linux.apt.planning` scope from `root-helper:apt_planning_v1`. This currently refreshes the existing extended inventory scopes too; their actual individual source timestamps/quality remain intact. There is no arbitrary package query, requested live diagnosis or software effect hidden in this collector. `apt_planning_v1` uses the existing UID-authenticated dedicated Unix socket, fresh challenge, device/generation/signing-key identity, root peer and separately root-provisioned stack pin. Its response is the readiness envelope with a typed `planning` result and **empty capability/acceptance fields**. It does not depend on a prior native-acceptance record and cannot create one. Root validates the installed code, exact runtime/config policy pins and actual archive bytes, sizes, ownership/modes/link count and private prefetch receipts. No artifact is fetched or installed by this RPC. Administratively stage the known fixture archives via `provision-fixture-store` before planning/the final 60-second grant. The fixed metadata child has a 4-second whole-process/output budget (32 KiB), the socket has 5-second IO limits, and root measurement timestamps are preserved. Dirty dpkg updates/triggers, foreign locks, unsafe/missing receipts, permission failure, timeout or missing helper yield an unknown observation with null data, not fake hashes or accepted capability. Runtime policy drift fails closed. Complete observations contain the suite's `fixture_package`, `baseline_sha256`, `sealed_baseline_sha256`, `status_fingerprint`, `dpkg_updates_empty`, `pending_triggers_empty`, `runtime`, `runtime_sha256`, `provider_sha256`, `hook_sha256`, `apt_config_sha256` and `prefetch`. The full package baseline stays root-private at `baselines/.json`; only hashes, fixture projection and minimal public runtime/artifact pins cross the wire. Stable sampling is checked before and after content-addressed exclusive write/file+directory fsync; existing identical seals are verified, never overwritten. This is passive metadata, not job authority. The ordinary durable inventory outbox/ACK path persists it. Normal accepted fixture `ensure_package`/`apply_updates` may return `POSTCONDITION_ALREADY_MET`, `changed=false`, `already_desired=true`, `postcondition_already_met=true`, `transaction_executed=false` only when the **entire selected set** is independently installed at its exact target version and architecture, clean, matches the current approved baseline/fingerprint and has the expected fixture payload. Root signature, all original bindings, expiry/sync/local-policy/resource gates, all manifest/code/runtime/artifact pins, native accepted-recipe record and durable grant/attempt consumption remain mandatory. Only the historical `old_version` condition is skipped in this no-mutation branch. No APT/hook/native launch/effect marker is created. A foreign lock, dirty/unknown state or changed baseline cannot become success; native acceptance itself never uses this branch. Other versions still follow the unchanged no-reinstall/no-downgrade/no-remove mutation guards. VM suite owner must update its absent/1.0-only adapter: after native update to 1.1, a normal exact desired-1.1 job must use the accepted update recipe with a fresh planning seal. A desired-1.0 job on 1.1 is still correctly denied; this implementation does not authorize removal/downgrade to make a stale test pass. No backend/domain/protobuf signatures or plan bytes have been changed. Provider source changes change the embedded provider/helper/stack/runtime pins. Deploy both helpers from this build and provision reviewed new proof/policy pins **before** native acceptance; old native records cannot silently certify new runtime bytes. Manifest schema 2 / transaction schema 1 / fixture archives 1.0 and 1.1 (20,672 bytes, unchanged hashes) remain unchanged. These are implemented source/local-fixture semantics, not VM or MVP acceptance. ## Certificate rotation and recovery `RMM_AGENT_STATE/certificates/current.json` is the authoritative atomic selector over private versioned device key/cert files. Existing enrolled files initialize the store exactly once; later connections/bootstrap use the selected identity. Rotation checks chain, current validity, exact agent CN, clientAuth purpose and public/private key match. It persists one pending key/CSR and retries those SAME CSR bytes after response loss. A verified swap closes the old actor/effect fence and requires a new Hello/Sync. Agent/enrollment IDs and independent job/helper signing pins do not change. Expired/revoked/invalid certs remain blocked and need authorized identity recovery; no auto-reenrollment or TLS downgrade. ## Verification boundaries / concrete remaining policy issue Local Rust source/unit/integration tests cover actual loopback mTLS dispatcher WSS/HTTPS clean reconnect, Ping, forged sync rejection, passive traffic with an untrusted effect clock, journal reopen/replay and independent helper checkpoint/ outcome/boot checks. Unsafe OS providers are mocked or denied; no local service, APT install or reboot was executed. This is not a real WebUI-approved VM result, native package acceptance, production auth acceptance or full Linux-MVP sign-off. Concrete backend policy limitation, not bypassed here: `RootScope::permits` in `crates/server/src/root_policy.rs` does not match `Action::QueryLogs` and falls through to false. The typed, bounded root helper log handler exists, and the unprivileged allowlisted path is live, but a root live-log plan will be denied by that backend policy until its owner adds an explicit permitted scope. Do NOT work around it by relabelling identity, direct approval or a generic root shell. ## Executed local verification Using `/home/Kevin/.cargo/bin/cargo`, actually passed during integration: ```sh cargo test -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked cargo clippy -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked -- -D warnings cargo build -p rmm-agent -p rmm-privileged-helper --locked cargo fmt -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux -- --check python3 tests/agent_jobs/test_install.py target/debug/rmm-agent --help target/debug/rmm-privileged-helper --help ``` 111 executed Rust test cases passed (including reused transport tests), two create-only installer guard tests passed, both debug binaries built and both help commands returned successfully without loading secret configuration. No warnings/type errors were suppressed. A boxed tungstenite HTTP-response test constructor was corrected, not bypassed. No coverage percentage was measured. Coverage boundaries: real loopback TLS/dispatcher WSS and HTTPS reconnect and forged sync rejection are exercised; helper cryptography/checkpoints/two-process shared signature and replay/unknown/reboot decisions use real SQLite/signatures with local fixtures. Native service/APT/reboot effects and positive root UID-0 daemon/VM RPC acceptance are NOT substituted by these tests. The built artifacts are `target/debug/rmm-agent` and `target/debug/rmm-privileged-helper`; a reviewed VM-compatible release artifact still needs the target worker's ELF/glibc/library preflight and actual authorized VM run. ## Executed local verification — planning/no-change increment After **all** source changes, the following complete targeted block passed with `/home/Kevin/.cargo/bin` on PATH (2026-11-12): ```sh cargo test -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests/apt_transaction -p 'test_*.py' -v cargo clippy -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux --all-targets --locked -- -D warnings cargo fmt -p rmm-agent -p rmm-privileged-helper -p rmm-agent-linux -- --check cargo build -p rmm-agent -p rmm-privileged-helper --locked cargo build -p rmm-agent -p rmm-privileged-helper --release --locked python3 tests/agent_jobs/test_install.py target/debug/rmm-agent --help target/debug/rmm-privileged-helper --help ``` Results: **115 Rust tests**, **56 Python APT tests**, **2 installer guard tests** passed; warning-denying all-target Clippy and formatting passed; both Debug and Release binaries built; both help commands succeeded. Separate final AST parsing passed for **8 Python files**. No full workspace/backend/web test suite was run. New regressions cover shared Python/Rust canonical golden bytes/hash, receipt and timestamp freshness, private content-addressed seal reuse/change without full-baseline disclosure, sampling drift/dirty/foreign-lock/bad-artifact failure, normal exact already-desired install/update postconditions, no launch/effect markers, old-version mutation rejection, wrong architecture/newer version, audit/payload/expiry rejection, no-change outcome binding and durable duplicate grant/attempt rejection, honest missing-helper data and collection backpressure. Root ownership/kernel state and unsafe native package processes are mocked; existing harmless unprivileged seccomp tests really execute. No native host package mutation, root systemd/probe/VM action, approval or reboot was performed. Earlier intermediate failures were corrected, not suppressed: the inherited two unformatted source edits, a mock reader missing the `limit` keyword argument, and a newly inserted function accidentally inheriting a test-only cfg attribute. The final complete block above passed after these corrections. Final artifact SHA-256 (native developer-PC builds; **not** Bookworm/VM proof): | Artifact | SHA-256 | |---|---| | `target/debug/rmm-agent` | `90a4eab064301740cd938a41a76a510ad3b4abc3e48a188ba12bfe22f123266c` | | `target/debug/rmm-privileged-helper` | `54db2fe17841fef9af48bbeb446f45bda52f62e442a4900e0543e358095900b7` | | `target/release/rmm-agent` | `eac5434fd7a1e509c0971e4e05d4e1aedbde1bb90f7d7a8f9ec4d48ab4d34597` | | `target/release/rmm-privileged-helper` | `581a033478b41365066df2dc9ac0d11e314160909b5fab612be0be97be1bbbdc` | | `tools/apt-transaction/apt_transaction.py` | `57b695bf57ea2703393e39f92a1b6ef2314b4838f3dfb29e4e70da79687c9987` | Remaining: suite-owner adapter update for exact desired 1.1 after native upgrade, reviewed deployment/new pins, actual dedicated-profile metadata RPC and native UI-approved fixture/no-change VM proof, target ELF/library compatibility, and VM resource/retention measurements for the new 60-second extended collection. No real current baseline change was induced on the host/VM: freshness tests use changed temporary/mock package state, not a disguised package installation. Linux-MVP and production acceptance remain open. Backend Root QueryLogs policy is owned by its separate worker and was neither changed nor reverified here.