Compare commits

77 Commits
Author SHA1 Message Date
kevin_janssenandClaude Opus 5.5 dbfa76184d Record the Proxmox host acceptance: guests, backups, automatic VM mapping and failure cases
Rust development slice / linux (push) Failing after 7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 14:05:05 +01:00
kevin_janssenandClaude Opus 5.5 a1b22070de Count a vzdump run that fails before its first guest as a failed backup
Rust development slice / linux (push) Failing after 7s
Found on the Proxmox test host: with the target storage disabled the scheduled
run ends with an error and its log names no guest, which was reported as
'missed'. Such a job-level or same-guest run now marks the covered guests failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 14:01:29 +01:00
kevin_janssenandClaude Opus 5.5 29acaa5b08 Pin backup evaluation to the report of the Proxmox VE 9.2 test host
Rust development slice / linux (push) Failing after 10s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:48:37 +01:00
kevin_janssenandClaude Opus 5.5 9fa6c05af7 Document the Proxmox host, its read-only access and the local Proxmox test VM
Rust development slice / linux (push) Failing after 15s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:17:23 +01:00
kevin_janssenandClaude Opus 5.5 63447f1b87 Provision a read-only Proxmox VE token during HTTPS installation
Rust development slice / linux (push) Failing after 15s
On hosts with pveversion and pveum the installer creates rmm-discovery@pve with
a privilege-separated PVEAuditor token, copies the cluster CA and passes both to
the agent through a systemd credential drop-in. Existing tokens are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:14:05 +01:00
kevin_janssenandClaude Opus 5.5 ce356f0de7 Show Proxmox guests and scheduled backup status in device details
Rust development slice / linux (push) Failing after 14s
The host detail lists VMs and containers with linked inventory devices and
last successful backup, each vzdump job with its due time and result, an
overall 'all scheduled backups ran' answer and guests without a job. VM hosts
detected from a Proxmox guest list are labelled and the assignment form can
hand the decision back to automatic detection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:12:54 +01:00
kevin_janssenandClaude Opus 5.5 70977df577 Validate Proxmox observations, evaluate scheduled backups and link VMs to their Proxmox host
Rust development slice / linux (push) Failing after 15s
linux.proxmox gets a strict validator and a 128 KiB scope budget. Device
details expose guests, backup jobs with ok/failed/missed/running status and
linked devices. Base snapshots reconcile unique guest MACs into
assets.hypervisor_asset_id with hypervisor_source='proxmox'; manual choices
are kept and can be handed back to automatic detection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:10:35 +01:00
kevin_janssenandClaude Opus 5.5 1d2d52847c Collect Proxmox VE guests, backup jobs and vzdump runs as a read-only linux.proxmox scope
Rust development slice / linux (push) Failing after 12s
The Linux agent queries the local PVE API with a PVEAuditor token from the
systemd credential rmm-proxmox-token and the pinned cluster CA. Guests carry
MAC addresses and SMBIOS UUIDs for hypervisor mapping; jobs carry covered
guests and the scheduler's last expected run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 12:04:13 +01:00
kevin_janssenandClaude Opus 5.5 4459fbf79f Merge live Windows agent 0.1.27 source into the deployed server state
Rust development slice / linux (push) Failing after 14s
main now matches what runs on discovery.ease-technologies.de on 2026-11-20:
- Server, gateway, public API and WebUI: image rmm-dev-server:79f236b
  (schema 47), plus the follow-up docs commit fb35dbd.
- Windows agent and helper 0.1.27: signed release build 28 from 52c352c;
  hashes verified on the Windows test guest.
- Linux agent 0.1.0-mvp.14 (62cb52b) and OPNsense agent 0.1.0-opnsense.6
  are already ancestors of 79f236b.

Conflict in crates/agent-windows/src/native.rs resolved to the released
Windows side (os_inventory with missing DisplayVersion handling).
cargo check passes for the Linux workspace and for rmm-agent-windows on
x86_64-pc-windows-gnu.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 10:50:18 +01:00
kevin_janssen fb35dbd16f Document disk measurements and preserve published helper pins in inventory releases
Rust development slice / linux (push) Failing after 14s
2026-11-20 09:44:28 +01:00
kevin_janssen 79f236b3ea Show disk, pool and unassigned Linux capacity in device fleet views 2026-11-20 09:35:05 +01:00
kevin_janssen eaaa6ff228 Inventory Linux whole disks and account mounted filesystems by block topology 2026-11-20 09:34:17 +01:00
kevin_janssen 131cc53de2 Document Windows drive-letter inventory and native multi-drive checks
Rust development slice / linux (push) Failing after 14s
2026-11-20 08:58:56 +01:00
kevin_janssen 04037c7acf Display Windows fixed volumes individually in both fleet views 2026-11-20 08:53:20 +01:00
kevin_janssen 52c352c0d9 Collect capacity for every Windows fixed drive letter
Rust development slice / linux (push) Failing after 11s
2026-11-20 08:51:54 +01:00
kevin_janssen 9c61f8f2eb Collect capacity for every Windows fixed drive letter 2026-11-20 08:50:49 +01:00
kevin_janssen b5e96853c3 Show occupied disk and RAM capacity with matching used and free segments
Rust development slice / linux (push) Failing after 9s
2026-11-20 08:40:48 +01:00
kevin_janssen 957bf3cd58 Swap free and used colors on RAM and disk capacity bars
Rust development slice / linux (push) Failing after 10s
2026-11-20 08:37:18 +01:00
kevin_janssen 021fdfc1bc Use muted terracotta for occupied RAM and disk capacity
Rust development slice / linux (push) Failing after 10s
2026-11-20 08:35:50 +01:00
kevin_janssen 9057808d16 Show free disk capacity with the same bar and caption as RAM
Rust development slice / linux (push) Failing after 10s
2026-11-20 08:31:15 +01:00
kevin_janssen 0b0790de71 Run approved FreeBSD root jobs through independently pinned and synchronized local helper
Rust development slice / linux (push) Failing after 11s
2026-11-20 08:27:13 +01:00
kevin_janssen 2e3916b84b Seed versioned OPNsense scripts and label the shared POSIX interpreter 2026-11-20 08:22:59 +01:00
kevin_janssen 1f3cd5ea0c Route OPNsense library and custom jobs through the standard order composer 2026-11-20 08:15:07 +01:00
kevin_janssen 8a1ea58d86 Plan OPNsense POSIX root scripts with explicit device policy, profile and approval 2026-11-20 08:12:52 +01:00
kevin_janssen 47249644b9 Record approved OPNsense kernel probe and matching native start evidence
Rust development slice / linux (push) Failing after 6s
2026-11-20 08:06:32 +01:00
kevin_janssen 6792b0dd11 Verify signed OPNsense kernel probes on the service account and retain exactly-once evidence
Rust development slice / linux (push) Failing after 13s
2026-11-19 20:55:29 +01:00
kevin_janssen 2e060e9507 Plan an exact service-account OPNsense kernel probe through normal approval 2026-11-19 20:48:41 +01:00
kevin_janssen 71adc2adf6 Record live OPNsense enrollment, WireGuard handshakes and remaining job parity
Rust development slice / linux (push) Failing after 14s
2026-11-19 20:14:34 +01:00
kevin_janssen 8acbee6a60 Qualify FreeBSD kernel time and supervise bounded POSIX process groups natively 2026-11-19 20:14:34 +01:00
kevin_janssen 6bf6e2539c Exercise appliance secret projection, truncation and device-only UI states
Rust development slice / linux (push) Failing after 12s
2026-11-19 20:10:36 +01:00
kevin_janssen 50b1fa4033 Run bounded OPNsense inventory and durable mTLS heartbeats as separate FreeBSD services
Rust development slice / linux (push) Failing after 11s
2026-11-19 20:07:49 +01:00
kevin_janssen b7d9046996 Project native FreeBSD packages, interfaces, hardware and login observations 2026-11-19 20:06:56 +01:00
kevin_janssen d05caf6fd3 Allow OPNsense snapshots in the tenant-bound observations table 2026-11-19 19:54:55 +01:00
kevin_janssen 89a58c609c Accept separate OPNsense inventory and show VPN and PF observations in device details
Rust development slice / linux (push) Failing after 13s
2026-11-19 19:53:51 +01:00
kevin_janssen ac6f940e2e Record public RAM bar verification
Rust development slice / linux (push) Failing after 12s
2026-11-19 19:38:18 +01:00
kevin_janssen 577a6eea75 Clarify the free and used RAM bar in device inventory
Rust development slice / linux (push) Failing after 12s
2026-11-19 19:37:41 +01:00
kevin_janssen 51ce8cde77 Record Windows inventory hotfix publication and native verification 2026-11-19 16:36:10 +01:00
kevin_janssen 037b5e39fa Handle missing DisplayVersion in Windows inventory 2026-11-19 16:30:39 +01:00
kevin_janssen 58236bc8f0 Record public rollout of device purpose
Rust development slice / linux (push) Failing after 13s
2026-11-19 15:11:58 +01:00
kevin_janssen 2070ae95e7 Add editable purpose to device inventory
Rust development slice / linux (push) Failing after 13s
2026-11-19 15:07:26 +01:00
kevin_janssen 5cfa0cfdcf Record Windows local clock rollout and idle CPU verification
Rust development slice / linux (push) Failing after 10s
2026-11-19 11:58:48 +01:00
kevin_janssen b002202d62 Qualify enrolled Windows job clocks from the local time service
Rust development slice / linux (push) Failing after 15s
2026-11-19 11:47:49 +01:00
kevin_janssen 6aeb618170 Record local Windows time policy and native diagnostic limits
Rust development slice / linux (push) Failing after 14s
2026-11-19 11:32:25 +01:00
kevin_janssen 5a9eafbff1 Record Windows idle CPU and SYSTEM follow-up
Rust development slice / linux (push) Failing after 13s
2026-11-19 10:21:07 +01:00
kevin_janssen f01e81418e Verify damaged history still blocks durable no-start
Rust development slice / linux (push) Failing after 13s
2026-11-19 10:10:02 +01:00
kevin_janssen d4cbc40979 Avoid replaying the helper journal history during idle sync 2026-11-19 10:09:38 +01:00
kevin_janssen 478ee1fc1a Avoid repeated full journal scans and fast ticks in Windows idle 2026-11-19 10:04:26 +01:00
kevin_janssen 01960477f8 Record verified production customer transfer cutover
Rust development slice / linux (push) Failing after 8s
2026-11-18 23:01:25 +01:00
kevin_janssen 480e3749a6 Complete customer transfers on first authenticated inventory
Rust development slice / linux (push) Failing after 8s
2026-11-18 22:57:05 +01:00
kevin_janssen 65d84a19c0 Record the production VM host display verification
Rust development slice / linux (push) Failing after 7s
2026-11-18 22:44:20 +01:00
kevin_janssen cad188aeb7 Use complete certificate evidence in the hypervisor database fixture
Rust development slice / linux (push) Failing after 7s
2026-11-18 22:42:14 +01:00
kevin_janssen bcd3bd86ad Show known hypervisors and the Hetzner placeholder for virtual machines 2026-11-18 22:39:29 +01:00
kevin_janssen 08177d8843 Merge Windows helper idle CPU fix and native verification
Rust development slice / linux (push) Failing after 8s
2026-11-18 22:20:14 +01:00
kevin_janssen 2378a54bd4 Record production customer transfer availability
Rust development slice / linux (push) Failing after 7s
2026-11-18 21:47:27 +01:00
kevin_janssen 9e957e6d8d Slow successful Windows helper time refreshes
Rust development slice / linux (push) Failing after 8s
(cherry picked from commit a4bcebf440)
2026-11-18 21:40:10 +01:00
kevin_janssen 5253a9f596 Support interactive customer transfers for enrolled devices
Rust development slice / linux (push) Failing after 8s
2026-11-18 21:35:28 +01:00
kevin_janssen dd59052e7c Record native Windows idle performance and SYSTEM acceptance
Rust development slice / linux (push) Failing after 8s
2026-11-18 21:31:48 +01:00
kevin_janssen a4bcebf440 Slow successful Windows helper time refreshes 2026-11-18 21:19:14 +01:00
kevin_janssen db6325131a Record production device inventory verification
Rust development slice / linux (push) Failing after 8s
2026-11-18 20:30:24 +01:00
kevin_janssen 1f60183b7e Fit larger Linux package inventories within bounded snapshots
Rust development slice / linux (push) Failing after 8s
2026-11-18 20:20:10 +01:00
kevin_janssen 55d159cab1 Retry Linux inventory when durable traffic delays collection
Rust development slice / linux (push) Failing after 8s
2026-11-18 20:12:05 +01:00
kevin_janssen aea2d5b379 Expand the passive inventory shown in device details
Rust development slice / linux (push) Failing after 9s
2026-11-18 20:02:41 +01:00
kevin_janssen 38f43f887a Record the native OTRS closure and dashboard verification
Rust development slice / linux (push) Failing after 8s
2026-11-18 19:08:27 +01:00
kevin_janssen 915e126f24 Reconcile a human closure when the helper has no result
Rust development slice / linux (push) Failing after 9s
2026-11-18 19:03:02 +01:00
kevin_janssen abad2022e2 Record the live RAM and CPU dashboard checks
Rust development slice / linux (push) Failing after 8s
2026-11-18 18:54:40 +01:00
kevin_janssen 6ed57c4d0b Show the same capacity bar for RAM and disk space
Rust development slice / linux (push) Failing after 7s
2026-11-18 18:47:46 +01:00
kevin_janssen 7006ce7420 Keep the last valid CPU reading during RAM-only heartbeats
Rust development slice / linux (push) Failing after 8s
2026-11-18 18:46:04 +01:00
kevin_janssen b3fe35b86b Show available RAM and measured CPU usage for devices
Rust development slice / linux (push) Failing after 8s
2026-11-18 18:37:18 +01:00
kevin_janssen fb2376443c Document OTRS ARM64 root-script helper activation
Rust development slice / linux (push) Failing after 7s
2026-11-18 18:21:51 +01:00
kevin_janssen f2a2d44d84 Require pushing completed changes to Gitea
Rust development slice / linux (push) Failing after 7s
2026-11-18 18:14:50 +01:00
kevin_janssen 7215ea9733 Merge remote-tracking branch 'origin/main' into codex/linux-arm64-installer 2026-11-18 18:14:35 +01:00
kevin_janssen b5907d681d Record successful OTRS enrollment and corrected HTTPS installer 2026-11-18 18:13:16 +01:00
kevin_janssen 4b3d0136e9 Wait for the device certificate before probing Linux enrollment 2026-11-18 18:09:52 +01:00
kevin_janssen bfdc3da8af Record native ARM64 HTTPS release and installer verification 2026-11-18 18:07:31 +01:00
kevin_janssen 6d15f1b3f5 Use the shared target triple for the release catalog 2026-11-18 17:59:34 +01:00
kevin_janssen 8b5365f4bc Verify Linux installer architecture selection and display available targets 2026-11-18 17:59:19 +01:00
kevin_janssen 247646a2ba Build and serve signed Linux ARM64 agent releases 2026-11-18 17:58:12 +01:00
155 changed files with 11737 additions and 493 deletions
+1
View File
@@ -8,6 +8,7 @@ Der abgenommene MVP liegt auf dem Branch `main`. Der [Produktvertrag](Rust_RMM_I
2. In der [Übergabe](docs/handover.md) den betroffenen Einstiegspunkt finden. Vor Änderungen den tatsächlichen Aufrufer, die Berechtigungsprüfung und die vorhandenen Tests lesen. Ein datierter Beleg in `docs/evidence/` dokumentiert das Ereignis zu seinem Messzeitpunkt.
3. Den normalen Bedienweg und den konkreten Fehlerfall prüfen. Bei einem Linux- oder Windows-spezifischen Verhalten den betreffenden Gast verwenden. Ein Build belegt die Erzeugung des Artefakts. Das Ergebnis des Zielgeräts belegt seine Wirkung.
4. Verändert sich das Produktverhalten, die zuständige Anleitung und den Produktvertrag abgleichen. Eine neue Abnahme im [Prüfstand](docs/mvp-completion-plan.md) mit Quelle, Ziel und beobachtetem Ergebnis nachtragen.
5. Nach jeder abgeschlossenen Änderung die zugehörigen Dateien committen und auf den passenden Branch in Gitea pushen. Das gilt auch für reine Dokuänderungen. Den gepushten Commit in Gitea prüfen. [Entwicklungsablauf](docs/development.md#änderungen-in-gitea-sichern)
## Regeln für Änderungen
Generated
+24
View File
@@ -2187,8 +2187,10 @@ dependencies = [
"chrono",
"ed25519-dalek",
"hex",
"libc",
"openssl",
"prost",
"reqwest",
"rmm-domain",
"rmm-protocol",
"rmm-updater-handoff",
@@ -2208,6 +2210,7 @@ dependencies = [
"rmm-domain",
"serde",
"serde_json",
"tokio",
"uuid",
]
@@ -2284,6 +2287,27 @@ dependencies = [
"zeroize",
]
[[package]]
name = "rmm-opnsense-jobs"
version = "0.1.0"
dependencies = [
"anyhow",
"chrono",
"ed25519-dalek",
"hex",
"libc",
"prost",
"rmm-agent-core",
"rmm-agent-linux",
"rmm-domain",
"rmm-protocol",
"rusqlite",
"serde",
"serde_json",
"sha1",
"uuid",
]
[[package]]
name = "rmm-persistence"
version = "0.1.0"
+1 -1
View File
@@ -1,7 +1,7 @@
[workspace]
resolver = "2"
exclude = ["crates/oidc-auth"]
members = ["crates/domain", "crates/protocol", "crates/persistence", "crates/server", "crates/agent-linux", "crates/agent-core", "crates/agent-windows", "crates/agent", "crates/privileged-helper", "crates/updater", "crates/updater-handoff", "crates/updater-linux", "crates/updater-worker", "crates/update-history", "crates/time-attestation", "crates/time-source-linux", "crates/time-broker-linux"]
members = ["crates/domain", "crates/protocol", "crates/persistence", "crates/server", "crates/agent-linux", "crates/agent-core", "crates/agent-windows", "crates/agent", "crates/agent-opnsense-jobs", "crates/privileged-helper", "crates/updater", "crates/updater-handoff", "crates/updater-linux", "crates/updater-worker", "crates/update-history", "crates/time-attestation", "crates/time-source-linux", "crates/time-broker-linux"]
[workspace.package]
version = "0.1.0"
+23 -3
View File
@@ -10,10 +10,28 @@ Jede Installation ist an Kunde, Gerät und Agentenidentität gebunden. Die Stand
Der Agent meldet den Gerätetyp mit dem passiven Hardwareinventar. `physical` bezeichnet einen physischen Rechner, `virtual` ein virtualisiertes System und `hypervisor` eine erkannte Hostrolle. Proxmox VE erscheint als `hypervisor` mit `Proxmox VE` in der Spalte OS. Eine erkannte Hostrolle hat Vorrang, auch bei verschachtelter Virtualisierung. Bei fehlender Angabe zeigt die WebUI „Nicht gemeldet“. Die Geräteabfragen in WebUI und MCP verwenden denselben gespeicherten Typ. [Inventarprojektion](crates/server/src/devices.rs)
Kunden lassen sich in der WebUI anlegen und umbenennen. Beim Kundenwechsel erhält das Gerät eine neue Kopplung. Eine ausgeschiedene Installation kann nach Widerruf und persönlicher Bestätigung endgültig gelöscht werden. Ein noch gültiger offener Genehmigungsantrag oder ein nicht abgeschlossener Job einschließlich `unknown` sperrt die Löschung. Daten weiterhin beteiligter Zielgeräte bleiben erhalten. [Regeln zur Löschung](migrations/0036_delete_device_pairing.sql)
Für aktuelle virtuelle Geräte kann die WebUI einen registrierten Hypervisor aus demselben Kundenbestand zuordnen. Die Geräteübersicht zeigt diese Verbindung unter dem Hostnamen der VM und verlinkt auf die Gerätedetails des Hypervisors. Die Zuordnung gilt, solange der verknüpfte Host im Bestand ist und sich als Hypervisor meldet. Eine VM mit gemeldetem Hersteller Hetzner erhält bei fehlendem konkreten Host den Anzeigeplatzhalter „Hetzner Cloud“. Der Platzhalter hat keine Kopplung und zählt nicht als registriertes Gerät.
Ein aufgenommener Proxmox-VE-Host meldet über einen vom Installer eingerichteten, nur lesenden API-Token (`PVEAuditor`) die Gäste seines Nodes, die geplanten vzdump-Backupjobs und die Läufe der letzten acht Tage. Die Gerätedetails des Hosts listen VMs und Container mit Status, Ressourcen, zugeordnetem Gerät und letzter erfolgreicher Sicherung. Darunter folgen die Backupjobs mit Zeitplan, Storage, gesicherten Gästen, letztem fälligen Zeitpunkt, Ergebnis und nächstem Lauf sowie die Gäste ohne Backupjob. Der Server bewertet jeden aktiven Job als gelaufen, fehlgeschlagen, verpasst, laufend oder noch nicht fällig. Er beantwortet damit, ob alle geplanten Backups gelaufen sind. Ein virtuelles Gerät, dessen MAC-Adresse eindeutig in der Gastliste eines Proxmox-Hosts desselben Kunden steht, wird diesem Host automatisch als Hypervisor zugeordnet und so gekennzeichnet. Eine manuelle Zuordnung hat Vorrang und lässt sich wieder an die automatische Erkennung zurückgeben. [Proxmox-Host](deploy/proxmox/README.md)
Ab Windows-Client `0.1.27` zeigt die Spalte **Speicher belegt / gesamt** jedes lokale Festplattenvolume mit Laufwerksbuchstaben separat an, das Systemlaufwerk zuerst. Ein Laufwerk mit unbekannter Kapazität behält seinen Buchstaben ohne Belegungsbalken. Die Zuordnung erfolgt pro logischem Volume, nicht pro physischem Datenträger. Netzlaufwerke, optische Medien und Volumes ohne Laufwerksbuchstaben gehören nicht zu dieser Liste. Bei älteren Inventarmeldungen bleibt die vorhandene Systemlaufwerkskapazität sichtbar.
Unter Linux zeigt die Übersicht die vom Kernel erkannten ganzen Festplatten bzw. virtuellen Gastplatten mit Gerätename, Modell (falls verfügbar) und Rohkapazität. Für direkt zuordenbare lokale Dateisysteme zählt sie belegte und für Anwendungen verfügbare Bytes über alle Partitionen, dasselbe Dateisystem nur einmal. Rot bezeichnet nachgewiesene Belegung, Grün verfügbaren Dateisystemplatz und Grau den Rest, einschließlich Reserven, unpartitioniertem Platz und nicht messbarer Belegung. Ein vollständig unbekannter Wert ist kein Nullwert. Bei gestapelten Blockgeräten und plattenübergreifenden Dateisystemen erscheint die logische Belegung separat mit den ermittelbaren Mitgliedern; sie wird nicht auf physische Platten verteilt. ZFS meldet die gemeinsame Belegung des Pools bei lesbaren ZFS-Eigenschaften, nicht die Summe der freien Bytes seiner Datasets. Alte Clients weisen den Wert für `/` als Root-Dateisystem aus.
Geräteübersicht und Gerätedetails zeigen für Laufwerke und RAM „belegt / gesamt“ in GiB. Belegt ist die Gesamtkapazität abzüglich des freien Laufwerksplatzes bzw. des für Anwendungen verfügbaren Arbeitsspeichers. Die Balken in der Übersicht zeigen links den belegten Anteil in gedämpftem Rot und rechts den freien Anteil in Grün. Darunter steht der belegte Anteil in Prozent. Ohne gültiges Wertepaar erscheint kein Balken. Die CPU-Spalte ergänzt die zuletzt gemessene Systemauslastung als Prozentwert. Die Agenten gewinnen sie aus den CPU-Zeitdifferenzen zwischen zwei Heartbeats im Abstand von ungefähr 30 Sekunden. Linux liefert systemweite Werte. Windows liefert den Wert, wenn alle logischen CPUs zu einer einzigen Prozessorgruppe gehören. Vor dem ersten aussagekräftigen Intervall bleibt die Auslastung offen. Die WebUI und MCP lesen dieselbe letzte Messung. Für Agenten ohne Ressourcenmeldung stammen verfügbare RAM-Werte aus dem vorhandenen Inventar. [Inventarprojektion](crates/server/src/devices.rs)
Kunden lassen sich in der WebUI anlegen und umbenennen. Unter **Gerät verwalten → Kunden wechseln** wählt eine berechtigte Person den Zielkunden. Der Server prüft die Rechte zur Geräteverwaltung im bisherigen Kunden und zur Geräteaufnahme im Zielkunden. Der bereits installierte Agent empfängt den Wechsel über seine authentisierte Verbindung und erhält ein neues Zertifikat für eine neue Kopplung. Mit der ersten gültigen Inventarmeldung der authentisierten neuen Kopplung schließt der Server den Wechsel automatisch ab. Er sperrt die bisherige Geräteidentität und schaltet den Bestand in derselben Transaktion auf den Zielkunden um. Inventar und Auftragsergebnisse der bisherigen Kopplung bleiben beim bisherigen Kunden. Noch nicht abgeschlossene Aufträge, ungeklärte Versuche und gültige offene Genehmigungen müssen vor dem Wechsel abgeschlossen werden. Gerätegebundene erhöhte Rechte erfordern eine eigene Bindung für die neue Kopplung.
Eine ausgeschiedene Installation kann nach Widerruf und persönlicher Bestätigung endgültig gelöscht werden. Ein noch gültiger offener Genehmigungsantrag oder ein nicht abgeschlossener Job einschließlich `unknown` sperrt die Löschung. Daten weiterhin beteiligter Zielgeräte bleiben erhalten. [Regeln zur Löschung](migrations/0036_delete_device_pairing.sql)
In den Gerätedetails erscheinen OS-Installationszeit, letzter Systemstart, Hersteller, BIOS-Seriennummer, Netzwerkkarten samt IPv4- und IPv6-Adressen, installierte Software und die bei der letzten Inventur angemeldeten Benutzer. Windows meldet den vom Betriebssystem erfassten Installationszeitpunkt. Linux kennzeichnet den Zeitstempel aus dem Installer-Verzeichnis oder dem Root-Dateisystem als Schätzung. Das Softwareinventar erfasst Linux-Pakete aus dpkg und Windows-Systeminstallationen aus der Uninstall-Registrierung. Begrenzte Meldungen werden ausgewiesen. Herstellergarantie und Firmwaredaten erscheinen bei verfügbarem Befund. Die Daten gehören zum kundengebundenen Inventar des Geräts. [Geräteansicht](web/src/device-inventory.tsx)
Die Spalte **Zweck** und das Gerätedetail zeigen optionalen Freitext. Unter **Zweck bearbeiten** können Betreiber mit Inventarlese- und Geräteverwaltungsrecht für den betreffenden Kunden bis zu 500 Zeichen speichern oder den Text entfernen. Der Zweck gehört zum Gerät beim bisherigen Kunden und wird beim Kundenwechsel nicht übernommen. Die Änderung wird ohne den Text auditiert und erzeugt keinen Geräteauftrag.
## Vom Skript zum Ergebnis
**OPNsense:** Das Gerätegateway akzeptiert `opnsense.base/1` als eigene Inventarplattform. Im Gerätedetail erscheinen aktive PF-Filterregeln sowie getrennte Listen für konfigurierte VPNs und native Laufzeitbefunde. Fehlende Berechtigung bzw. ausgefallene Quellen sind kein Nachweis einer leeren Liste. Skriptaufträge gehören wie bei Linux und Windows unter **Aufträge**. Die WebUI bindet eine Bibliotheksrevision oder ein einmaliges POSIX-Skript an die ausgewählten OPNsense-Geräte. Für die Ausführung als `root` gelten zusätzlich zum normalen signierten Abgleich und persönlichen Freigabeverfahren `root:execute`, die zentrale Geräte- und Principalbindung sowie die unabhängige Prüfung durch den lokal provisionierten FreeBSD-Helper. [OPNsense-Betrieb](deploy/opnsense/README.md)
| Schritt | Verhalten |
| --- | --- |
| Skript veröffentlichen | In der Bibliothek eines Kunden erhält ein Skript einen Namen, einen Interpreter, einen Standard-Ausführungskontext und eine unveränderliche Version. Eine Bearbeitung veröffentlicht eine weitere Version. |
@@ -24,7 +42,9 @@ Kunden lassen sich in der WebUI anlegen und umbenennen. Beim Kundenwechsel erhä
Linux führt Shellskripte unter dem Dienstkonto oder über den eingerichteten Helper als root aus. Windows führt PowerShell unter LocalService oder über den Helper als SYSTEM aus. Neustart und Installation von Windows-Updates sind Skriptaufträge. Bei einem Neustart kann das Skript bereits vor dem Herunterfahren erfolgreich enden. Der Betriebsnachweis verbindet den neuen Bootbezug mit dem nächsten authentisierten Kontakt und einem Folgeauftrag. [Skriptbibliothek](crates/server/src/script_library.rs) · [Auftragsverarbeitung](crates/server/src/jobs.rs)
Ein Abbruchwunsch wird am Auftrag gespeichert. Bei unterbrochener Rückmeldung gleichen Server, Agent und gegebenenfalls Helper den gespeicherten Versuch ab. Bei `unknown` bleibt die Wirkung ungeklärt. Die WebUI kann einen solchen Fall dokumentiert abschließen. Dieser Abschluss erteilt keine Freigabe für einen weiteren Versuch.
Agent und privilegierter Helper prüfen jeweils den Synchronisationsstatus, eine begrenzte Zeitunsicherheit und die Kontinuität der lokalen Systemzeit. Die serverseitig signierte Starterlaubnis wird an der lokal geprüften Uhr gemessen. Linux liest dafür den Kernelstatus. Auf regulär aufgenommenen Windows-Geräten fragen der Agent und der dauerhaft eingerichtete SYSTEM-Helper den lokalen Windows-Zeitdienst jeweils selbst ab. Beide prüfen die Identität des Zeitdienstes an der Verbindung, die den Status liefert. Sie akzeptieren den zuletzt erfolgreichen Abgleich innerhalb eines Tages und eine geschätzte Unsicherheit bis zu zwei Sekunden. Bei einem erkannten Zeitsprung oder Standby warten sie auf einen neueren erfolgreichen Abgleich. Der bisherige signierte Zeitvergleich wird noch im separaten Windows-Update-Wartungsprofil und bei Installationen mit manueller Auftragsbindung verwendet. [Entscheidung zur Uhrprüfung](docs/project-decisions.md)
Ein Abbruchwunsch wird am Auftrag gespeichert. Bei unterbrochener Rückmeldung gleichen Server, Agent und gegebenenfalls Helper den gespeicherten Versuch ab. Bei `unknown` bleibt die Wirkung ungeklärt. Die WebUI kann einen solchen Fall dokumentiert abschließen. Agent und Helper übernehmen den Abschluss im signierten Abgleich auch dann, wenn der Agent den Abbruch seiner eigenen Ausführung protokolliert hat und im unabhängig gebundenen Verbrauchsjournal des Helpers ein Ergebnis fehlt. Beide bewahren die ursprünglichen Belege. Der Abschluss erteilt keine Freigabe für einen weiteren Versuch.
## Öffentliche Verwaltung und Berechtigungen
@@ -38,6 +58,6 @@ Für administrative Skripte gilt zusätzlich das Plattformrecht `root:execute` o
Die WebUI erzeugt einen kundengebundenen Einmalcode und zeigt die HTTPS-Installationsanleitung für die verfügbare Plattform. Zum Erstbezug braucht das Gerät eine vertrauenswürdige TLS-Verbindung. Der Installer prüft Vertrauensmaterial und Release, richtet die lokalen Dienste ein und nimmt das Gerät auf. Anschließend verwendet die Installation ihre eigene Identität. Der lokale Administrator richtet den privilegierten Skript-Helper ein, wenn das Gerät auch Administratoraufträge erhalten soll. [Serverseitiger Bootstrap](crates/server/src/bootstrap)
Der Server stellt je Kunde und Plattform Agent und Helper mit einem gemeinsam signierten Manifest bereit. Das Manifest bindet die Hashes und Größen der Binärdateien. Für die Erstaufnahme liefert der Server auch den passenden Bootstrap-Updater. Linux x64 verwendet den GNU/Linux-Kanal, Windows x64 offiziell `x86_64-pc-windows-gnu`. Der lokale Updater fragt regelmäßig nach einer neueren veröffentlichten Generation, prüft die Bindung an Plattform, Schlüsselring, Releasefolge und Binärhashes und aktiviert das geprüfte Paar unter derselben Geräteidentität. Auf Linux steuert ein systemd-Timer den Abruf, auf Windows ein eigener Updatedienst.
Der Server stellt je Kunde und Plattform Agent und Helper mit einem gemeinsam signierten Manifest bereit. Das Manifest bindet die Hashes und Größen der Binärdateien. Für die Erstaufnahme liefert der Server auch den passenden Bootstrap-Updater. Das gemeinsame Linux-Installationsskript erkennt x86-64 und ARM64 auf dem Zielgerät. Beide GNU/Linux-Architekturen haben einen eigenen Releasekanal, Signierschlüssel und Bootstrap-Pin. Windows x64 verwendet offiziell `x86_64-pc-windows-gnu`. Der lokale Updater fragt regelmäßig nach einer neueren veröffentlichten Generation, prüft die Bindung an Plattform, Schlüsselring, Releasefolge und Binärhashes und aktiviert das geprüfte Paar unter derselben Geräteidentität. Auf Linux steuert ein systemd-Timer den Abruf, auf Windows ein eigener Updatedienst.
Für Erstinstallation und Update muss das Manifest zum Prüfzeitpunkt gültig sein. Beim bereits installierten Administrator-Helper prüft der Server die Gültigkeit zum gespeicherten Registrierungszeitpunkt des Releases und die Signatur gegen den aktuellen Schlüsselring. Der Ablauf des Manifests allein blockiert daher keinen Adminauftrag auf diesem Client. [Releaseprüfung](crates/server/src/agent_updates.rs) · [Build und Veröffentlichung](docs/development.md)
+4
View File
@@ -20,6 +20,7 @@ rmm-domain = { path = "../domain" }
rmm-protocol = { path = "../protocol" }
rmm-updater-handoff = { path = "../updater-handoff" }
rusqlite.workspace = true
reqwest.workspace = true
ed25519-dalek = "=2.2.0"
[dev-dependencies]
@@ -30,3 +31,6 @@ openssl = { workspace = true, optional = true }
[target.'cfg(unix)'.dev-dependencies]
openssl.workspace = true
[target.'cfg(target_os = "freebsd")'.dependencies]
libc = "0.2"
+391
View File
@@ -0,0 +1,391 @@
//! A handoff changes the installation, not the old journal or its trust.
use anyhow::{Context, Result, ensure};
use ed25519_dalek::{Signature, VerifyingKey};
use reqwest::Client;
use rmm_domain::customer_transfer::{CertificateRequest, CertificateResponse, Offer};
use rmm_protocol::EnrollmentResponse;
use serde::{Deserialize, de::DeserializeOwned};
use std::{
fs::{self, File, OpenOptions},
io::{Read, Write},
path::{Path, PathBuf},
time::Duration,
};
use uuid::Uuid;
const PREFIX: &str = "customer-move-";
const INTENT: &str = "customer-move-intent.json";
const MAX_RECORD: usize = 32768;
pub fn idle(journal: &rusqlite::Connection, ledger: &rusqlite::Connection) -> Result<bool> {
let busy: bool = journal.query_row(
"SELECT EXISTS(SELECT 1 FROM job_attempts WHERE state IN (
'accepted','requesting','running','verifying','unknown','awaiting_reboot'))",
[],
|row| row.get(0),
)?;
let consumed: bool = ledger.query_row(
"SELECT EXISTS(SELECT 1 FROM consumed WHERE result IS NULL)",
[],
|row| row.get(0),
)?;
// The new customer executes under the same service account. Retire
// local history only after the old installation has delivered its work.
if busy || consumed {
return Ok(false);
}
Ok(crate::jobs::events(journal)?.iter().all(|event| {
matches!(
event.payload,
Some(rmm_protocol::Payload::InventorySnapshot(_))
)
}))
}
fn read(path: &Path) -> Result<Vec<u8>> {
let info = fs::symlink_metadata(path)?;
ensure!(
info.is_file() && !info.file_type().is_symlink() && info.len() <= MAX_RECORD as u64,
"TRANSFER_RECORD"
);
let mut data = Vec::new();
File::open(path)?
.take(MAX_RECORD as u64 + 1)
.read_to_end(&mut data)?;
ensure!(data.len() <= MAX_RECORD, "TRANSFER_RECORD");
Ok(data)
}
fn sync_dir(path: &Path) -> Result<()> {
#[cfg(unix)]
File::open(path)?.sync_all()?;
#[cfg(not(unix))]
let _ = path;
Ok(())
}
fn publish(path: &Path, bytes: &[u8]) -> Result<()> {
if path.try_exists()? {
ensure!(read(path)? == bytes, "TRANSFER_RECORD_CONFLICT");
return Ok(());
}
let dir = path.parent().context("TRANSFER_RECORD_PARENT")?;
let temporary = dir.join(format!("{PREFIX}tmp-{}", Uuid::new_v4()));
let result = (|| {
let mut options = OpenOptions::new();
options.write(true).create_new(true);
#[cfg(unix)]
{
use std::os::unix::fs::OpenOptionsExt;
options.mode(0o600);
}
let mut file = options.open(&temporary)?;
file.write_all(bytes)?;
file.sync_all()?;
drop(file);
fs::rename(&temporary, path)?;
sync_dir(dir)
})();
if result.is_err() {
let _ = fs::remove_file(temporary);
}
result
}
fn file(dir: &Path, id: Uuid, role: &str) -> PathBuf {
dir.join(format!("{PREFIX}{id}-{role}"))
}
fn verify(response: &CertificateResponse, signer: &str) -> Result<()> {
let binding = &response.receipt.binding;
let signature = Signature::from_slice(&hex::decode(&response.receipt.signature_hex)?)?;
let public: [u8; 32] = hex::decode(signer)?
.try_into()
.map_err(|_| anyhow::anyhow!("TRANSFER_SIGNER"))?;
VerifyingKey::from_bytes(&public)?.verify_strict(&binding.signing_bytes()?, &signature)?;
ensure!(
binding.offer.source_tenant_id != binding.offer.target_tenant_id
&& binding.agent_id != binding.offer.source_agent_id
&& !binding.agent_id.is_nil()
&& !binding.asset_id.is_nil()
&& binding.certificate_sha256
== rmm_domain::jobs::hash(response.certificate_pem.as_bytes()),
"TRANSFER_BINDING"
);
Ok(())
}
async fn json<T: DeserializeOwned>(mut response: reqwest::Response) -> Result<T> {
ensure!(response.status().is_success(), "TRANSFER_HTTP");
let mut bytes = Vec::new();
while let Some(chunk) = response.chunk().await? {
ensure!(
bytes.len() + chunk.len() <= MAX_RECORD,
"TRANSFER_RESPONSE_SIZE"
);
bytes.extend_from_slice(&chunk);
}
Ok(serde_json::from_slice(&bytes)?)
}
pub async fn stage(
client: &Client,
origin: &str,
dir: &Path,
current: &crate::auth::Binding,
key_and_csr: impl FnOnce(Option<&[u8]>) -> Result<(Vec<u8>, String)>,
) -> Result<bool> {
#[derive(Deserialize)]
struct Response {
offer: Option<Offer>,
}
let route = format!("{origin}/agent/v1/customer-transfer");
let response = client
.get(&route)
.timeout(Duration::from_secs(4))
.send()
.await?;
if response.status() == reqwest::StatusCode::NOT_FOUND {
return Ok(false); // agent updates can precede the server upgrade
}
let Some(offer) = json::<Response>(response).await?.offer else {
return Ok(false);
};
ensure!(
offer.source_agent_id == current.agent_id
&& offer.source_tenant_id == current.tenant_id
&& offer.target_tenant_id != current.tenant_id
&& !offer.transfer_id.is_nil(),
"TRANSFER_SOURCE"
);
let id = offer.transfer_id;
let key_path = file(dir, id, "key.pem");
let previous = key_path
.try_exists()?
.then(|| read(&key_path))
.transpose()?;
let (key, csr_pem) = key_and_csr(previous.as_deref())?;
publish(&key_path, &key)?;
let response = json::<CertificateResponse>(
client
.post(format!("{route}/{id}/certificate"))
.timeout(Duration::from_secs(10))
.json(&CertificateRequest { csr_pem })
.send()
.await?,
)
.await?;
verify(&response, &current.public_key_hex)?;
ensure!(
response.receipt.binding.offer == offer,
"TRANSFER_OFFER_CHANGED"
);
publish(
&file(dir, id, "response.json"),
&serde_json::to_vec(&response)?,
)?;
publish(&dir.join(INTENT), &serde_json::to_vec(&id)?)?;
Ok(true)
}
fn archive(dir: &Path, id: Uuid) -> Result<()> {
for item in fs::read_dir(dir)? {
let item = item?;
let name = item.file_name();
if name.to_string_lossy().starts_with(PREFIX)
|| ["agent.lock", "job-signing-key.json"]
.iter()
.any(|kept| name == *kept)
{
continue;
}
if name == "jobs" && item.file_type()?.is_dir() {
// Keep the installer-owned directory and its Windows ACL.
archive(&item.path(), id)?;
continue;
}
let mut backup = std::ffi::OsString::from(format!("{PREFIX}{id}-prior-"));
backup.push(name);
let target = dir.join(backup);
ensure!(!target.try_exists()?, "TRANSFER_ARCHIVE_EXISTS");
fs::rename(item.path(), target)?;
}
sync_dir(dir)
}
fn retire_prior_state(dir: &Path, id: Uuid) -> Result<()> {
let prefix = format!("{PREFIX}{id}-prior-");
for item in fs::read_dir(dir)? {
let item = item?;
let name = item.file_name();
if name == "jobs" && item.file_type()?.is_dir() {
retire_prior_state(&item.path(), id)?;
} else if name.to_string_lossy().starts_with(&prefix) {
if item.file_type()?.is_dir() {
fs::remove_dir_all(item.path())?;
} else {
fs::remove_file(item.path())?;
}
}
}
sync_dir(dir)
}
pub fn pending(dir: &Path) -> Result<bool> {
Ok(dir.join(INTENT).try_exists()?)
}
pub fn activate(dir: &Path, signer: &str) -> Result<()> {
let intent = dir.join(INTENT);
if !intent.try_exists()? {
return Ok(());
}
let id: Uuid = serde_json::from_slice(&read(&intent)?)?;
let response_file = file(dir, id, "response.json");
let data = read(&response_file)?;
let response: CertificateResponse = serde_json::from_slice(&data)?;
let binding = &response.receipt.binding;
verify(&response, signer)?;
ensure!(binding.offer.transfer_id == id, "TRANSFER_INTENT");
let relocated = file(dir, id, "relocated");
if !relocated.try_exists()? {
let current: EnrollmentResponse =
serde_json::from_slice(&read(&dir.join("identity.json")).or_else(|error| {
let backup = file(dir, id, "prior-identity.json");
if backup.exists() {
read(&backup)
} else {
Err(error)
}
})?)?;
ensure!(
current.agent_id == binding.offer.source_agent_id.to_string(),
"TRANSFER_SOURCE"
);
archive(dir, id)?;
publish(&relocated, b"ok")?;
}
let key_file = file(dir, id, "key.pem");
let identity = EnrollmentResponse {
agent_id: binding.agent_id.to_string(),
asset_id: binding.asset_id.to_string(),
certificate_pem: response.certificate_pem,
};
publish(&dir.join("device.key"), &read(&key_file)?)?;
publish(&dir.join("device.pem"), identity.certificate_pem.as_bytes())?;
publish(&dir.join("identity.json"), &serde_json::to_vec(&identity)?)?;
publish(&file(dir, binding.agent_id, "active.json"), &data)?;
// Do not expose old-tenant credentials or job outputs to scripts running
// under this same local service account in the new tenant.
retire_prior_state(dir, id)?;
fs::remove_file(intent)?;
sync_dir(dir)?;
// The live private key is durable. This copy only existed for upload retry.
let _ = fs::remove_file(key_file);
let _ = fs::remove_file(response_file);
let _ = fs::remove_file(relocated);
Ok(())
}
pub fn assigned_tenant(
dir: &Path,
identity: &EnrollmentResponse,
signer: &str,
) -> Result<Option<Uuid>> {
let path = file(dir, Uuid::parse_str(&identity.agent_id)?, "active.json");
if !path.try_exists()? {
return Ok(None);
}
let response: CertificateResponse = serde_json::from_slice(&read(&path)?)?;
verify(&response, signer)?;
let binding = response.receipt.binding;
ensure!(
response.certificate_pem == identity.certificate_pem
&& binding.agent_id.to_string() == identity.agent_id
&& binding.asset_id.to_string() == identity.asset_id,
"TRANSFER_ACTIVE_IDENTITY"
);
Ok(Some(binding.offer.target_tenant_id))
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::{Signer, SigningKey};
use rmm_domain::customer_transfer::{Binding, Receipt};
#[test]
fn authenticated_handoff_recovers_partial_archive_and_isolates_local_state() -> Result<()> {
let dir = std::env::temp_dir().join(format!("rmm-transfer-{}", Uuid::new_v4()));
fs::create_dir(&dir)?;
let result = (|| {
let id = Uuid::new_v4();
let source = Uuid::new_v4();
let target = Uuid::new_v4();
let previous = EnrollmentResponse {
agent_id: source.to_string(),
asset_id: Uuid::new_v4().to_string(),
certificate_pem: "old certificate fixture".into(),
};
publish(&dir.join("identity.json"), &serde_json::to_vec(&previous)?)?;
publish(&dir.join("device.key"), b"old key")?;
publish(&dir.join("journal.sqlite"), b"old journal")?;
let signer = SigningKey::from_bytes(&[7; 32]);
let public = hex::encode(signer.verifying_key().as_bytes());
let certificate_pem = "new certificate fixture".to_string();
let binding = Binding {
offer: Offer {
transfer_id: id,
source_agent_id: source,
source_tenant_id: Uuid::new_v4(),
target_tenant_id: Uuid::new_v4(),
},
asset_id: Uuid::new_v4(),
agent_id: target,
certificate_sha256: rmm_domain::jobs::hash(certificate_pem.as_bytes()),
};
let tenant = binding.offer.target_tenant_id;
let mut response = CertificateResponse {
certificate_pem,
receipt: Receipt {
signature_hex: hex::encode(signer.sign(&binding.signing_bytes()?).to_bytes()),
binding,
},
};
publish(&file(&dir, id, "key.pem"), b"new key")?;
publish(&dir.join(INTENT), &serde_json::to_vec(&id)?)?;
response.receipt.binding.offer.target_tenant_id = Uuid::new_v4();
fs::write(
file(&dir, id, "response.json"),
serde_json::to_vec(&response)?,
)?;
assert!(activate(&dir, &public).is_err());
assert_eq!(fs::read(dir.join("device.key"))?, b"old key");
response.receipt.binding.offer.target_tenant_id = tenant;
fs::write(
file(&dir, id, "response.json"),
serde_json::to_vec(&response)?,
)?;
// Simulate a stop immediately after relocating the first original file.
fs::rename(
dir.join("identity.json"),
file(&dir, id, "prior-identity.json"),
)?;
activate(&dir, &public)?;
activate(&dir, &public)?;
assert!(!pending(&dir)?);
assert!(!file(&dir, id, "prior-identity.json").exists());
assert!(!file(&dir, id, "prior-device.key").exists());
assert!(!file(&dir, id, "prior-journal.sqlite").exists());
assert_eq!(fs::read(dir.join("device.key"))?, b"new key");
let active: EnrollmentResponse =
serde_json::from_slice(&read(&dir.join("identity.json"))?)?;
assert_eq!(active.agent_id, target.to_string());
assert_eq!(assigned_tenant(&dir, &active, &public)?, Some(tenant));
Ok(())
})();
fs::remove_dir_all(dir)?;
result
}
}
+82
View File
@@ -0,0 +1,82 @@
//! Read-only FreeBSD kernel discipline for the signed reconciliation clock.
//! Never adjust the guest clock. No cached qualification across a failed read.
use crate::reconcile::{ClockReading, SyncClock};
use anyhow::{Context, Result, ensure};
use chrono::{DateTime, TimeDelta, Utc};
use std::{mem::MaybeUninit, time::Instant};
#[derive(Clone)]
pub struct NativeClock {
origin: DateTime<Utc>,
monotonic: Instant,
}
impl Default for NativeClock {
fn default() -> Self {
Self {
origin: Utc::now(),
monotonic: Instant::now(),
}
}
}
impl NativeClock {
fn observation(&self) -> Result<(DateTime<Utc>, TimeDelta)> {
let start = Instant::now();
let mut raw = MaybeUninit::<libc::ntptimeval>::zeroed();
// SAFETY: the read-only syscall writes the platform's ntptimeval.
let result = unsafe { libc::ntp_gettime(raw.as_mut_ptr()) };
ensure!(result == libc::TIME_OK, "FREEBSD_CLOCK_UNQUALIFIED");
let sample = unsafe { raw.assume_init() };
let wall =
DateTime::from_timestamp(sample.time.tv_sec, u32::try_from(sample.time.tv_nsec)?)
.context("FREEBSD_CLOCK_INVALID")?;
let duration = TimeDelta::from_std(start.elapsed())?;
ensure!(
sample.time_state == libc::TIME_OK
&& (0..=5_000_000).contains(&sample.maxerror)
&& (0..=sample.maxerror).contains(&sample.esterror)
&& sample.time.tv_nsec < 1_000_000_000
&& duration <= TimeDelta::milliseconds(100)
&& ((wall - self.origin) - TimeDelta::from_std(self.monotonic.elapsed())?)
.num_milliseconds()
.unsigned_abs()
<= 5000,
"FREEBSD_CLOCK_UNQUALIFIED"
);
Ok((wall, TimeDelta::microseconds(sample.maxerror) + duration))
}
}
impl SyncClock for NativeClock {
fn now(&self) -> Result<DateTime<Utc>> {
Ok(self.observation()?.0)
}
fn reading(&self) -> Result<ClockReading> {
let (wall, error) = self.observation()?;
Ok(ClockReading {
own_wall: wall,
upper: wall + error,
})
}
fn bounds(&self) -> Result<(DateTime<Utc>, DateTime<Utc>)> {
let (wall, error) = self.observation()?;
Ok((wall - error, wall + error))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
#[ignore = "requires a FreeBSD guest synchronized by ntpd"]
fn native_bounds_and_monotonic_fence() -> Result<()> {
let mut clock = NativeClock::default();
let (low, high) = clock.bounds()?;
ensure!(low <= high && high - low <= TimeDelta::milliseconds(10200));
clock.origin += TimeDelta::minutes(1);
ensure!(clock.bounds().is_err());
Ok(())
}
}
+92 -42
View File
@@ -205,6 +205,7 @@ pub struct HelperSync<V = DefaultVerifier> {
born: DateTime<Utc>,
capture: Option<Capture>,
lease: Option<Lease>,
integrity: Option<crate::ledger::IntegrityMonitor>,
}
impl HelperSync<DefaultVerifier> {
/// No persisted ready bit can open a newly started helper.
@@ -222,6 +223,7 @@ impl<V: Ed25519Verifier> HelperSync<V> {
born: clock.now()?,
capture: None,
lease: None,
integrity: None,
})
}
/// Select exactly one protected helper execution identity, closing leases.
@@ -230,6 +232,12 @@ impl<V: Ed25519Verifier> HelperSync<V> {
self.identity = identity;
self
}
/// Use with one protected SQLite connection retained for this instance.
/// The native owner must run a full check before any new host effect.
pub fn with_periodic_idle_integrity_checks(mut self) -> Self {
self.integrity = Some(crate::ledger::IntegrityMonitor::default());
self
}
pub fn close(&mut self) {
self.lease = None;
self.capture = None;
@@ -288,7 +296,8 @@ impl<V: Ed25519Verifier> HelperSync<V> {
ledger_fingerprint: lease.fingerprint.clone(),
captured_at: capture.captured_at,
};
self.verify_no_start_anchor(db, guard, &anchor, clock, boot)?;
let audit_history = self.integrity.is_none();
self.verify_no_start_anchor(db, guard, &anchor, clock, boot, audit_history)?;
Ok(anchor)
}
/// Consume exactly the approved update attempt and delegate its ORIGINAL
@@ -389,9 +398,9 @@ impl<V: Ed25519Verifier> HelperSync<V> {
tx.commit()?;
return Ok(outcome);
}
self.verify_no_start_anchor(&tx, guard, anchor, clock, boot)?;
self.verify_no_start_anchor(&tx, guard, anchor, clock, boot, true)?;
let observation = guard.verify_no_dispatch(request, anchor)?;
let now = self.verify_no_start_anchor(&tx, guard, anchor, clock, boot)?;
let now = self.verify_no_start_anchor(&tx, guard, anchor, clock, boot, true)?;
ensure!(
!observation.intent_id.is_nil()
&& digest(&observation.intent_sha256)
@@ -421,6 +430,7 @@ impl<V: Ed25519Verifier> HelperSync<V> {
anchor: &NoStartAnchor,
clock: &impl Clock,
boot: &impl BootSource,
audit_history: bool,
) -> Result<DateTime<Utc>> {
guard.check(db)?;
let now = clock.now()?;
@@ -445,15 +455,18 @@ impl<V: Ed25519Verifier> HelperSync<V> {
&& ledger_fingerprint(db)? == anchor.ledger_fingerprint,
"DENIAL_HELPER_LINEAGE_CHANGED"
);
// Exact equality is deliberate: a first implementation refuses even
// legitimate intervening progress rather than inventing lineage links.
verify_checkpoint_repair_history_with(
db,
&self.binding,
None,
self.identity,
&self.verifier,
)?;
// Idle anchor capture checks the current signed head. The native
// executor audits the full history before execution, and abandonment
// always audits it under the same transaction before recording evidence.
if audit_history {
verify_checkpoint_repair_history_with(
db,
&self.binding,
None,
self.identity,
&self.verifier,
)?;
}
let (response, raw): (Vec<u8>, Vec<u8>) = db.query_row(
"SELECT response,request FROM helper_sync_decisions WHERE decision_id=?1",
[anchor.decision_id.to_string()],
@@ -483,8 +496,12 @@ impl<V: Ed25519Verifier> HelperSync<V> {
&& decision.helper_journal_id == Some(anchor.journal_id)
&& decision.helper_checkpoint == anchor.checkpoint
&& decision.connection_generation == anchor.connection_generation
&& decision.request_hash == anchor.sync_request_sha256
&& capture.journal_id == anchor.journal_id
&& capture.checkpoint == anchor.checkpoint
&& capture.captured_at == anchor.captured_at
&& capture.boot_id == anchor.boot_id
&& capture.captured_at <= decision.issued_at
&& capture.captured_at <= now,
"DENIAL_CONFIRMED_READY_REQUIRED"
);
@@ -523,7 +540,12 @@ impl<V: Ed25519Verifier> HelperSync<V> {
);
let boot_id = boot.current_boot()?;
ensure!(!boot_id.is_nil(), "BOOT_ID_REQUIRED");
initialize(db, &self.binding)?;
if let Some(integrity) = &mut self.integrity {
integrity.check(db, false, "ROOT_LEDGER_INTEGRITY")?;
initialize_schema(db, &self.binding)?;
} else {
initialize(db, &self.binding)?;
}
let tx = db.transaction_with_behavior(TransactionBehavior::Immediate)?;
let meta = metadata(&tx)?;
ensure!(meta.fault.is_none(), "HELPER_MANUAL_RECOVERY_REQUIRED");
@@ -782,10 +804,10 @@ impl<V: Ed25519Verifier> HelperSync<V> {
let d = decisions
.get(&id)
.context("ROOT_ATTEMPT_OMITTED_FROM_DECISION")?;
validate_summary(row, local)?;
validate_decision(row, local, d)?;
let unreported = validate_summary(row, local)?;
validate_decision(row, local, d, unreported)?;
if d.unlock {
let bytes = unlock_outcome(row, d, current_boot)?;
let bytes = unlock_outcome(row, d, current_boot, unreported.then_some(local))?;
if d.outcome_state == "closed_unknown" {
crate::ledger::retain_closure_original(
&tx,
@@ -1031,13 +1053,17 @@ struct Meta {
agent_checkpoint: u64,
}
fn initialize(db: &Connection, binding: &Binding) -> Result<()> {
let check: String = db.query_row("PRAGMA integrity_check", [], |r| r.get(0))?;
ensure!(check == "ok", "ROOT_LEDGER_INTEGRITY");
crate::ledger::check_integrity(db, "ROOT_LEDGER_INTEGRITY")?;
initialize_schema(db, binding)
}
fn initialize_schema(db: &Connection, binding: &Binding) -> Result<()> {
db.execute_batch("PRAGMA synchronous=FULL;
CREATE TABLE IF NOT EXISTS helper_sync_meta(id INTEGER PRIMARY KEY CHECK(id=1),journal_id TEXT NOT NULL,checkpoint INTEGER NOT NULL DEFAULT 0,generation INTEGER NOT NULL DEFAULT 0,binding_sha256 TEXT NOT NULL,last_decision TEXT,fault TEXT,agent_journal_id TEXT,agent_checkpoint INTEGER NOT NULL DEFAULT 0);
CREATE TABLE IF NOT EXISTS helper_sync_rows(grant_id TEXT PRIMARY KEY,revision INTEGER NOT NULL,fingerprint TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS helper_sync_captures(checkpoint INTEGER PRIMARY KEY,snapshot BLOB NOT NULL,fingerprint TEXT NOT NULL,captured_at TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS helper_sync_decisions(decision_id TEXT PRIMARY KEY,checkpoint INTEGER NOT NULL,response BLOB NOT NULL,request BLOB NOT NULL,fingerprint TEXT NOT NULL);")?;
CREATE TABLE IF NOT EXISTS helper_sync_decisions(decision_id TEXT PRIMARY KEY,checkpoint INTEGER NOT NULL,response BLOB NOT NULL,request BLOB NOT NULL,fingerprint TEXT NOT NULL);
CREATE INDEX IF NOT EXISTS helper_sync_decisions_checkpoint ON helper_sync_decisions(checkpoint);")?;
let legacy_captures: bool = db.query_row(
"SELECT EXISTS(SELECT 1 FROM sqlite_schema WHERE type='table' AND name='helper_sync_captures')",
[], |row| row.get(0),
@@ -1175,23 +1201,23 @@ pub fn verify_checkpoint_repair_history_with(
) -> Result<()> {
let meta = metadata(db)?;
validate_history(db, binding, &meta, meta.generation, verifier)?;
let entry_count: u64 = db.query_row(
"SELECT count(*) FROM helper_sync_decisions",
[],
|row| row.get(0),
)?;
ensure!(entry_count <= 100000, "REPAIR_HISTORY_LIMIT");
let mut statement = db.prepare(
"SELECT response,request,checkpoint FROM helper_sync_decisions ORDER BY checkpoint",
)?;
let entries = statement
.query_map([], |r| {
Ok((
r.get::<_, Vec<u8>>(0)?,
r.get::<_, Vec<u8>>(1)?,
r.get::<_, u64>(2)?,
))
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
ensure!(entries.len() <= 100000, "REPAIR_HISTORY_LIMIT");
let mut entries = statement.query([])?;
let mut previous = None;
let mut previous_checkpoint = 0;
let mut previous_agent_checkpoint = 0;
for (signed, raw, checkpoint) in entries {
while let Some(entry) = entries.next()? {
let signed: Vec<u8> = entry.get(0)?;
let raw: Vec<u8> = entry.get(1)?;
let checkpoint: u64 = entry.get(2)?;
let response: SignedResponse = serde_json::from_slice(&signed)?;
let parsed: ReconcileDecisionPayload = serde_json::from_slice(&response.decision_bytes)?;
let decision = verify(verifier, binding, &response, parsed.issued_at)?;
@@ -1636,22 +1662,33 @@ fn definitive(row: &LocalAttempt) -> bool {
"succeeded" | "failed" | "cancelled" | "expired" | "closed_unknown"
) && !row.outcome_bytes.is_empty()
}
fn validate_summary(row: &LocalAttempt, a: &AttemptView) -> Result<()> {
fn validate_summary(row: &LocalAttempt, a: &AttemptView) -> Result<bool> {
let unreported = row.state == "unknown"
&& row.outcome_sha256.is_empty()
&& row.outcome_bytes.is_empty()
&& row.update_handoff_bytes.is_empty()
&& crate::reconcile::uncertain_agent_outcome(&a.state, &a.outcome_bytes, &a.outcome_sha256);
ensure!(
a.job_payload_hash == row.job_payload_hash
&& a.grant_id == row.grant_id.to_string()
&& a.consumption == "consumed"
&& a.state == row.state
&& a.revision >= row.revision
&& a.original_boot_id == row.original_boot_id.to_string()
&& a.outcome_sha256 == row.outcome_sha256
&& a.update_handoff_bytes == row.update_handoff_bytes
&& (a.outcome_bytes.is_empty() || a.outcome_bytes == row.outcome_bytes),
&& (unreported
|| (a.state == row.state
&& a.outcome_sha256 == row.outcome_sha256
&& (a.outcome_bytes.is_empty() || a.outcome_bytes == row.outcome_bytes))),
"ROOT_SNAPSHOT_REQUEST_MISMATCH"
);
Ok(())
Ok(unreported)
}
fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -> Result<()> {
fn validate_decision(
row: &LocalAttempt,
a: &AttemptView,
d: &AttemptDecision,
unreported: bool,
) -> Result<()> {
if !row.update_handoff_bytes.is_empty() {
let proof =
rmm_domain::agent_update::UpdateInflightProof::from_bytes(&row.update_handoff_bytes)
@@ -1686,7 +1723,7 @@ fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -
&& d.original_boot_id == Some(row.original_boot_id)
&& (d.revision >= a.revision || (!d.unlock && d.outcome_state == "unknown"))
&& d.previous_outcome_sha256.as_deref()
== (!row.outcome_sha256.is_empty()).then_some(row.outcome_sha256.as_str()),
== (!a.outcome_sha256.is_empty()).then_some(a.outcome_sha256.as_str()),
"ROOT_SIGNED_ATTEMPT_BINDING"
);
ensure!(
@@ -1697,11 +1734,26 @@ fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -
),
"UNKNOWN_CANNOT_UNLOCK"
);
ensure!(
!unreported
|| (!d.unlock && d.outcome_state == "unknown")
|| (d.unlock
&& d.outcome_state == "closed_unknown"
&& d.reason == "OPERATOR_CLOSED_UNKNOWN"),
"ROOT_UNREPORTED_OUTCOME_CLOSURE_REQUIRED"
);
Ok(())
}
fn unlock_outcome(row: &LocalAttempt, d: &AttemptDecision, current_boot: Uuid) -> Result<Vec<u8>> {
fn unlock_outcome(
row: &LocalAttempt,
d: &AttemptDecision,
current_boot: Uuid,
unreported: Option<&AttemptView>,
) -> Result<Vec<u8>> {
let bytes = if d.outcome_bytes.is_empty() {
row.outcome_bytes.clone()
unreported
.map_or(&row.outcome_bytes, |a| &a.outcome_bytes)
.clone()
} else {
d.outcome_bytes.clone()
};
@@ -1712,9 +1764,7 @@ fn unlock_outcome(row: &LocalAttempt, d: &AttemptDecision, current_boot: Uuid) -
&& o.state == d.outcome_state,
"ROOT_CANONICAL_UNLOCK_OUTCOME"
);
if bytes != row.outcome_bytes
&& crate::reconcile::operator_closure(&d.reason, &row.state, &o)
{
if bytes != row.outcome_bytes && crate::reconcile::operator_closure(&d.reason, &row.state, &o) {
return Ok(bytes);
}
if bytes != row.outcome_bytes {
+72 -3
View File
@@ -7,7 +7,8 @@ use rmm_domain::{
jobs::hash,
reconciliation::{ReconciledOutcome, canonical_outcome_bytes},
};
use rusqlite::{OptionalExtension, TransactionBehavior, params};
use rusqlite::{Connection, OptionalExtension, TransactionBehavior, params};
use std::time::{Duration, Instant};
use uuid::Uuid;
/// Finite 1 GiB bound for durable agent/helper history. Unexpected page sizes
@@ -29,11 +30,48 @@ pub fn initialize_ledger(db: &rusqlite::Connection) -> Result<()> {
CREATE TABLE IF NOT EXISTS consumed(grant_id TEXT PRIMARY KEY, agent TEXT NOT NULL, generation INTEGER NOT NULL,
job TEXT NOT NULL, attempt TEXT NOT NULL, result BLOB, UNIQUE(agent,generation,job,attempt));
CREATE TABLE IF NOT EXISTS execution_evidence(grant_id TEXT PRIMARY KEY, start BLOB NOT NULL, outcome BLOB);")?;
let check: String = db.query_row("PRAGMA integrity_check", [], |r| r.get(0))?;
ensure!(check == "ok", "JOURNAL_INTEGRITY");
check_integrity(db, "JOURNAL_INTEGRITY")
}
pub fn check_integrity(db: &Connection, error_code: &'static str) -> Result<()> {
let check: String = db.query_row("PRAGMA integrity_check", [], |row| row.get(0))?;
ensure!(check == "ok", "{error_code}");
Ok(())
}
/// Full integrity scans for an owner holding one protected, long-lived SQLite
/// connection. External commits and schema changes require another scan even
/// when the interval has not elapsed. No cached result authorizes an effect.
#[derive(Default)]
pub(crate) struct IntegrityMonitor {
checked: Option<(u64, u64, Instant)>,
}
impl IntegrityMonitor {
pub(crate) fn check(
&mut self,
db: &Connection,
required: bool,
error_code: &'static str,
) -> Result<()> {
let data: u64 = db.query_row("PRAGMA data_version", [], |row| row.get(0))?;
let schema: u64 = db.query_row("PRAGMA schema_version", [], |row| row.get(0))?;
if !required
&& self.checked.is_some_and(|(previous_data, previous_schema, checked)| {
data == previous_data
&& schema == previous_schema
&& checked.elapsed() < Duration::from_secs(300)
})
{
return Ok(());
}
self.checked = None;
check_integrity(db, error_code)?;
self.checked = Some((data, schema, Instant::now()));
Ok(())
}
}
/// Before a human WebUI closure replaces the unresolved local outcome, keep the
/// exact original outcome bytes once. A differing second copy fails closed.
pub(crate) fn retain_closure_original(
@@ -379,3 +417,34 @@ pub fn start_witness_bytes(
"reauthentication_required":true,"postconditions_passed":false});
Ok(serde_json::to_vec(&evidence)?)
}
#[cfg(test)]
mod integrity_tests {
use super::*;
#[test]
fn idle_integrity_reuses_the_check_but_reaudits_external_changes_and_expiry() {
let path = std::env::temp_dir().join(format!("rmm-integrity-{}.sqlite", Uuid::new_v4()));
{
let db = Connection::open(&path).unwrap();
db.execute_batch("CREATE TABLE sample(id INTEGER PRIMARY KEY);").unwrap();
let mut monitor = IntegrityMonitor::default();
monitor.check(&db, false, "JOURNAL_INTEGRITY").unwrap();
let first = monitor.checked.unwrap();
db.execute("INSERT INTO sample VALUES (1)", []).unwrap();
monitor.check(&db, false, "JOURNAL_INTEGRITY").unwrap();
assert_eq!(monitor.checked, Some(first));
let peer = Connection::open(&path).unwrap();
peer.execute("INSERT INTO sample VALUES (2)", []).unwrap();
monitor.check(&db, false, "JOURNAL_INTEGRITY").unwrap();
let changed = monitor.checked.unwrap();
assert_ne!(changed.0, first.0);
monitor.checked = Some((changed.0, changed.1, changed.2 - Duration::from_secs(300)));
monitor.check(&db, false, "JOURNAL_INTEGRITY").unwrap();
assert!(monitor.checked.unwrap().2 >= changed.2);
}
std::fs::remove_file(path).unwrap();
}
}
+3
View File
@@ -3,7 +3,10 @@
//! Callers must authenticate their transport, protect/lock their own journals and
//! supply a qualified clock. A successful signature is never UI approval itself.
pub mod auth;
pub mod customer_transfer;
pub mod crypto;
#[cfg(target_os = "freebsd")]
pub mod freebsd_clock;
pub mod helper_reconcile;
pub mod jobs;
pub mod ledger;
+143 -16
View File
@@ -101,6 +101,7 @@ pub struct ClientSync<V = DefaultVerifier> {
verifier: V,
helper_identity: HelperIdentity,
pending: Option<PreparedSync>,
integrity: Option<crate::ledger::IntegrityMonitor>,
channel: Option<Channel>,
gate: Option<Gate>,
}
@@ -117,6 +118,7 @@ impl<V: Ed25519Verifier> ClientSync<V> {
verifier,
helper_identity: HelperIdentity::Root,
pending: None,
integrity: None,
channel: None,
gate: None,
}
@@ -127,6 +129,11 @@ impl<V: Ed25519Verifier> ClientSync<V> {
self.helper_identity = identity;
self
}
/// Use with one protected SQLite connection retained for this instance.
pub fn with_periodic_idle_integrity_checks(mut self) -> Self {
self.integrity = Some(crate::ledger::IntegrityMonitor::default());
self
}
/// Invoke on every authenticated/fenced ServerHello or transport loss.
pub fn close(&mut self) {
self.gate = None;
@@ -606,7 +613,17 @@ impl<V: Ed25519Verifier> ClientSync<V> {
channel.connection_generation > 0 && !channel.boot_id.is_nil(),
"UNAUTHENTICATED_CHANNEL"
);
initialize(db)?;
if let Some(integrity) = &mut self.integrity {
let work_pending: bool = db.query_row(
"SELECT EXISTS(SELECT 1 FROM job_attempts WHERE state NOT IN ('succeeded','failed','cancelled','expired','closed_unknown'))",
[],
|row| row.get(0),
)?;
integrity.check(db, work_pending, "JOURNAL_INTEGRITY")?;
initialize_schema(db)?;
} else {
initialize(db)?;
}
let tx = db.transaction_with_behavior(TransactionBehavior::Immediate)?;
let meta = metadata(&tx)?;
ensure!(meta.fault.is_none(), "SYNC_MANUAL_RECOVERY_REQUIRED");
@@ -1019,9 +1036,9 @@ impl<V: Ed25519Verifier> ClientSync<V> {
} else {
for d in decision.attempts.iter().filter(|d| d.unlock) {
let grant = d.grant_id.context("UNLOCK_GRANT_MISSING")?;
let (agent,generation,job,attempt,previous):(String,u64,String,String,Option<Vec<u8>>)=tx.query_row(
"SELECT agent,generation,job,attempt,COALESCE((SELECT outcome FROM execution_evidence WHERE execution_evidence.grant_id=consumed.grant_id),result) FROM consumed WHERE grant_id=?1",
[grant.to_string()],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?,r.get(4)?)))?;
let (agent,generation,job,attempt,previous,start):(String,u64,String,String,Option<Vec<u8>>,Option<Vec<u8>>)=tx.query_row(
"SELECT c.agent,c.generation,c.job,c.attempt,COALESCE(e.outcome,c.result),e.start FROM consumed c LEFT JOIN execution_evidence e USING(grant_id) WHERE c.grant_id=?1",
[grant.to_string()],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?,r.get(4)?,r.get(5)?)))?;
ensure!(
agent == self.binding.agent_id.to_string()
&& generation == self.binding.enrollment_generation
@@ -1034,13 +1051,61 @@ impl<V: Ed25519Verifier> ClientSync<V> {
Some(&self.binding),
grant,
)?;
let previous = resolved
.or(previous)
.context("LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISSING")?;
let original = previous.clone();
let prior = canonical_outcome_bytes(&serde_json::from_slice::<ReconciledOutcome>(
&previous,
)?)?;
let previous = resolved.or(previous);
let summary = gate
.request
.attempts
.iter()
.find(|a| a.job_id == job && a.attempt_id == attempt)
.context("LOCAL_UNLOCK_ATTEMPT_MISSING")?;
let closure = d.reason == "OPERATOR_CLOSED_UNKNOWN"
&& d.outcome_state == "closed_unknown"
&& summary.consumption == "consumed"
&& uncertain_agent_outcome(
&summary.state,
&summary.outcome_bytes,
&summary.outcome_sha256,
);
let prior = match &previous {
Some(bytes) => {
let canonical = canonical_outcome_bytes(&serde_json::from_slice(bytes)?)?;
if closure
&& summary.state == "closed_unknown"
&& uncertain_agent_outcome("unknown", &canonical, &hash(&canonical))
{
summary.outcome_bytes.clone()
} else {
canonical
}
}
None => {
ensure!(closure, "LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISSING");
let start: serde_json::Value = serde_json::from_slice(
&start.context("LOCAL_UNLOCK_ORIGINAL_START_MISSING")?,
)?;
let request: crate::auth::Request =
serde_json::from_value(start["request"].clone())?;
let permit: rmm_domain::jobs::ExecutionPermitPayload =
serde_json::from_slice(&request.permit_bytes)?;
let (original, permit) = self.binding.verify_with(
&request,
permit.not_before,
&self.verifier,
)?;
ensure!(
permit.grant_id == grant
&& original.job_id == d.job_id
&& original.attempt_id == d.attempt_id
&& hash(&request.job_bytes) == d.job_payload_hash
&& Some(permit.authorization_id) == d.authorization_id
&& Some(permit.plan_hash.as_str()) == d.plan_hash.as_deref()
&& start["previous_boot_id"].as_str()
== d.original_boot_id.map(|id| id.to_string()).as_deref(),
"LOCAL_UNLOCK_ORIGINAL_START_BINDING"
);
summary.outcome_bytes.clone()
}
};
ensure!(
d.previous_outcome_sha256.as_deref() == Some(hash(&prior).as_str()),
"LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISMATCH"
@@ -1055,7 +1120,8 @@ impl<V: Ed25519Verifier> ClientSync<V> {
terminal(&outcome.state)
&& outcome.state == d.outcome_state
&& canonical_outcome_bytes(&outcome)? == bytes
&& d.outcome_sha256.as_deref() == Some(hash(&bytes).as_str()),
&& d.outcome_sha256.as_deref() == Some(hash(&bytes).as_str())
&& (previous.is_some() || operator_closure(&d.reason, "unknown", &outcome)),
"LOCAL_UNLOCK_CANONICAL_OUTCOME"
);
let preserve = crate::windows_system_nonstart::preserve_original_unlock(
@@ -1065,7 +1131,12 @@ impl<V: Ed25519Verifier> ClientSync<V> {
&bytes,
)?;
if outcome.state == "closed_unknown" {
crate::ledger::retain_closure_original(&tx, grant, &original, &bytes)?;
crate::ledger::retain_closure_original(
&tx,
grant,
previous.as_deref().unwrap_or_default(),
&bytes,
)?;
}
tx.execute(
"UPDATE consumed SET result=?2 WHERE grant_id=?1",
@@ -1093,8 +1164,11 @@ impl<V: Ed25519Verifier> ClientSync<V> {
}
pub fn initialize(db: &Connection) -> Result<()> {
let check: String = db.query_row("PRAGMA integrity_check", [], |r| r.get(0))?;
ensure!(check == "ok", "JOURNAL_INTEGRITY");
crate::ledger::check_integrity(db, "JOURNAL_INTEGRITY")?;
initialize_schema(db)
}
fn initialize_schema(db: &Connection) -> Result<()> {
db.execute_batch("PRAGMA synchronous=FULL;
CREATE TABLE IF NOT EXISTS reconcile_meta(id INTEGER PRIMARY KEY CHECK(id=1),journal_id TEXT NOT NULL,checkpoint INTEGER NOT NULL DEFAULT 0,high_sequence INTEGER NOT NULL DEFAULT 0,generation INTEGER NOT NULL DEFAULT 0,last_decision TEXT,pending_request TEXT,helper_id TEXT,helper_checkpoint INTEGER NOT NULL DEFAULT 0,fault TEXT);
CREATE TABLE IF NOT EXISTS reconcile_requests(id TEXT PRIMARY KEY,request BLOB NOT NULL,envelope BLOB NOT NULL,generation INTEGER NOT NULL,response BLOB);
@@ -1304,6 +1378,32 @@ pub(crate) fn operator_closure(
&& outcome.evidence["closed_by_human"] == true
&& outcome.evidence["retry"] == false
}
/// An interrupted agent can record its own uncertainty while the independently
/// consumed helper grant has no result. Neither witness establishes an effect.
pub(crate) fn uncertain_agent_outcome(state: &str, bytes: &[u8], digest: &str) -> bool {
if bytes.is_empty() || hash(bytes) != digest {
return false;
}
let Ok(outcome) = serde_json::from_slice::<ReconciledOutcome>(bytes) else {
return false;
};
canonical_outcome_bytes(&outcome).is_ok_and(|canonical| canonical == bytes)
&& outcome.state == state
&& !outcome.output_truncated
&& !outcome.reboot_required
&& (operator_closure("OPERATOR_CLOSED_UNKNOWN", "unknown", &outcome)
|| (state == "unknown"
&& matches!(
outcome.error_code.as_str(),
"AGENT_PROCESS_INTERRUPTED" | "HELPER_TRANSPORT_EFFECT_UNKNOWN"
)
&& outcome.exit_code.is_none()
&& outcome.stdout.is_empty()
&& outcome.stderr.is_empty()
&& outcome.evidence["postconditions_passed"] == false))
}
fn optional_uuid(s: &str) -> Result<Option<Uuid>> {
if s.is_empty() {
Ok(None)
@@ -1452,6 +1552,24 @@ fn attempts(db: &Connection, ledger: &Connection, binding: &Binding) -> Result<V
result_envelope = Some(envelope);
}
}
// The signed closure projection can commit before either consumption
// ledger. A fresh signed check may then finish the remaining writes.
if state == "closed_unknown"
&& consumption == "consumed"
&& (canonical.is_empty()
|| uncertain_agent_outcome("unknown", &canonical, &hash(&canonical)))
{
let closed: Vec<u8> = db.query_row(
"SELECT outcome FROM reconcile_outcomes WHERE job=?1 AND attempt=?2",
params![offer.job_id, offer.attempt_id],
|r| r.get(0),
)?;
ensure!(
uncertain_agent_outcome(&state, &closed, &hash(&closed)),
"LOCAL_CLOSURE_OUTCOME_MISMATCH"
);
canonical = closed;
}
if !canonical.is_empty() {
let o: ReconciledOutcome = serde_json::from_slice(&canonical)?;
ensure!(o.state == state, "LOCAL_OUTCOME_STATE_MISMATCH");
@@ -1746,6 +1864,15 @@ fn merge_helper(local: &mut [SyncAttempt], helper: &HelperSnapshot) -> Result<()
"HELPER_ATTEMPT_BINDING"
);
if h.consumption != "unconsumed" {
let unreported = h.state == "unknown"
&& h.consumption == "consumed"
&& h.outcome_bytes.is_empty()
&& h.outcome_sha256.is_empty()
&& h.update_handoff_bytes.is_empty()
&& a.consumption == "consumed"
&& a.grant_id == h.grant_id
&& a.original_boot_id == h.original_boot_id
&& uncertain_agent_outcome(&a.state, &a.outcome_bytes, &a.outcome_sha256);
if !h.update_handoff_bytes.is_empty() {
ensure!(
h.state == "running" && a.state == "running" && a.outcome_bytes.is_empty(),
@@ -1756,7 +1883,7 @@ fn merge_helper(local: &mut [SyncAttempt], helper: &HelperSnapshot) -> Result<()
a.grant_id = h.grant_id.clone();
a.consumption = h.consumption.clone();
ensure!(
a.outcome_sha256 == h.outcome_sha256,
a.outcome_sha256 == h.outcome_sha256 || unreported,
"HELPER_OUTCOME_MISMATCH"
);
if a.original_boot_id.is_empty() {
@@ -466,6 +466,31 @@ struct Fixture {
identity: HelperIdentity,
}
#[test]
fn idle_anchor_checks_the_head_but_abandonment_rejects_damaged_history() {
let (mut f, sync, _, request, mut guard) = denial_case();
let mut sync = sync.with_periodic_idle_integrity_checks();
f.clock.now += Duration::seconds(1);
let capture = f.snapshot(&mut sync, 7);
let raw = f.request(&capture);
let ready = f.decision(&capture, &raw, 7, true);
assert!(f.apply(&mut sync, &ready, &raw).unwrap().ready);
guard.intent_recorded_at = f.clock.now;
f.db.execute(
"UPDATE helper_sync_decisions SET response=X'00' WHERE checkpoint < ?1",
[capture.checkpoint],
).unwrap();
let anchor = sync.no_start_anchor(&f.db, &guard, 7, &f.clock, &f.boot).unwrap();
guard.anchor_sha256 = hash(&serde_json::to_vec(&anchor).unwrap());
let before = denial_ledger_bytes(&f.db);
assert!(
sync.abandon_no_start(&mut f.db, &guard, &anchor, &request, &f.clock, &f.boot)
.is_err()
);
assert_eq!(denial_ledger_bytes(&f.db), before);
}
#[test]
fn checkpoint_repair_verifies_complete_signed_history_and_detects_removed_predecessor() {
let mut f = Fixture::new();
+201
View File
@@ -1485,6 +1485,207 @@ fn persist_original_fixture_result(
fn completion_outcome() -> rmm_agent_core::auth::Outcome {
serde_json::from_slice(&canonical_outcome_bytes(&success()).unwrap()).unwrap()
}
#[test]
fn human_closure_reconciles_agent_interruption_and_a_helper_without_result() {
use rmm_agent_core::helper_reconcile::{self, BootSource, ExecutorLedgerGuard, HelperSync};
struct TestGuard;
impl ExecutorLedgerGuard for TestGuard {
fn check(&self, db: &Connection) -> anyhow::Result<()> {
anyhow::ensure!(
db.is_autocommit(),
"fixture ledger must be outside a transaction"
);
Ok(())
}
}
struct TestBoot(Uuid);
impl BootSource for TestBoot {
fn current_boot(&self) -> anyhow::Result<Uuid> {
Ok(self.0)
}
}
fn map(snapshot: &helper_reconcile::Snapshot) -> HelperSnapshot {
HelperSnapshot {
journal_id: snapshot.journal_id,
checkpoint: snapshot.checkpoint,
previous_decision_id: snapshot.previous_decision_id,
attempts: snapshot
.attempts
.iter()
.map(|a| SyncAttempt {
job_id: a.job_id.to_string(),
attempt_id: a.attempt_id.to_string(),
job_payload_hash: a.job_payload_hash.clone(),
grant_id: a.grant_id.to_string(),
consumption: a.consumption.clone(),
state: a.state.clone(),
revision: a.revision,
original_boot_id: a.original_boot_id.to_string(),
outcome_sha256: a.outcome_sha256.clone(),
outcome_bytes: a.outcome_bytes.clone(),
..Default::default()
})
.collect(),
}
}
let closed = canonical_outcome_bytes(&ReconciledOutcome {
state: "closed_unknown".into(),
stdout: vec![],
stderr: vec![],
exit_code: None,
output_truncated: false,
reboot_required: false,
evidence: serde_json::json!({
"postconditions_passed":false,"effect":"unknown","closed_by_human":true,"retry":false
}),
error_code: "OPERATOR_CLOSED_UNKNOWN".into(),
})
.unwrap();
for interrupt_after_projection in [false, true] {
let mut f = Fixture::new();
reconcile::initialize(&f.db).unwrap();
let (offer, grant, _) = f.add(
"running",
true,
None,
Action::SetServiceState {
unit: "rmm-acceptance.service".into(),
state: "running".into(),
},
);
let grant = grant.unwrap();
let archive = persist_original_fixture_result(
&mut f,
&offer,
&grant,
rmm_agent_core::auth::Outcome::unknown("AGENT_PROCESS_INTERRUPTED"),
);
rmm_agent_core::jobs::ack(&mut f.db, &archive.message_id).unwrap();
let start: Vec<u8> = f
.ledger
.query_row("SELECT start FROM execution_evidence", [], |row| row.get(0))
.unwrap();
let start: serde_json::Value = serde_json::from_slice(&start).unwrap();
let request: Request = serde_json::from_value(start["request"].clone()).unwrap();
let boot = TestBoot(f.channel.boot_id);
let guard = TestGuard;
let mut helper_db = Connection::open_in_memory().unwrap();
rmm_agent_core::ledger::initialize_ledger(&helper_db).unwrap();
rmm_agent_core::ledger::begin_execution(
&mut helper_db,
&grant,
&request,
boot.0,
f.clock.now,
)
.unwrap();
let mut helper = HelperSync::new(f.binding.clone(), &f.clock).unwrap();
let mut client = f.client();
for round in 0..if interrupt_after_projection { 3 } else { 2 } {
let snapshot = map(&helper
.snapshot(
&mut helper_db,
&guard,
f.channel.connection_generation,
&f.clock,
&boot,
)
.unwrap());
assert_eq!(snapshot.attempts[0].state, "unknown");
assert!(snapshot.attempts[0].outcome_bytes.is_empty());
let prepared = client
.prepare(
&mut f.db,
&f.ledger,
f.channel.clone(),
Some(&snapshot),
&f.clock,
)
.unwrap();
let agent_attempt = &prepared.request.attempts[0];
assert!(!agent_attempt.outcome_bytes.is_empty());
let mut d = f.decision(&prepared, round != 0);
d.issued_at = f.clock.now;
if round != 0 {
let attempt = &mut d.attempts[0];
attempt.revision = 3;
attempt.outcome_state = "closed_unknown".into();
attempt.reason = "OPERATOR_CLOSED_UNKNOWN".into();
attempt.unlock = true;
attempt.outcome_sha256 = Some(hash(&closed));
if agent_attempt.state == "unknown" {
attempt.outcome_bytes = closed.clone();
}
}
let signed = f.sign(&d);
client.apply(&mut f.db, &signed, &f.clock).unwrap();
if interrupt_after_projection && round == 1 {
client = f.client();
continue;
}
client.apply_consumption(&mut f.ledger, &f.clock).unwrap();
let raw = prepared.request.encode_to_vec();
let response = helper_reconcile::SignedResponse {
decision_bytes: signed.decision_bytes,
signature: signed.signature,
};
let applied = helper
.apply(&mut helper_db, &guard, &response, &raw, &f.clock, &boot)
.unwrap();
if round == 0 {
assert!(!applied.ready && applied.unlocked_grants.is_empty());
assert!(!client.can_start(&f.channel, false, &f.clock));
} else {
assert!(applied.ready);
assert_eq!(applied.unlocked_grants, vec![grant.grant_id]);
assert!(
helper
.apply(&mut helper_db, &guard, &response, &raw, &f.clock, &boot)
.unwrap()
.unlocked_grants
.is_empty()
);
client.apply_consumption(&mut f.ledger, &f.clock).unwrap();
client.confirm_helper(&map(&applied.snapshot)).unwrap();
assert!(client.can_start(&f.channel, true, &f.clock));
}
}
assert_eq!(
f.db.query_row::<String, _, _>("SELECT state FROM job_attempts", [], |rec| rec.get(0))
.unwrap(),
"closed_unknown"
);
assert_eq!(
f.db.query_row::<Vec<u8>, _, _>("SELECT envelope FROM reconcile_results", [], |rec| {
rec.get(0)
})
.unwrap(),
archive.encode_to_vec()
);
for ledger in [&f.ledger, &helper_db] {
assert_eq!(
ledger
.query_row::<Vec<u8>, _, _>("SELECT result FROM consumed", [], |rec| rec.get(0))
.unwrap(),
closed
);
assert_eq!(
ledger
.query_row::<(Vec<u8>, Vec<u8>), _, _>(
"SELECT original_outcome,closure_outcome FROM unknown_closure_originals",
[],
|rec| Ok((rec.get(0)?, rec.get(1)?))
)
.unwrap(),
(vec![], closed.clone())
);
}
}
}
#[test]
fn crash_after_durable_outcome_before_result_recovers_original_then_signed_sync_closes() {
let mut f = Fixture::new();
+2
View File
@@ -11,3 +11,5 @@ uuid.workspace = true
serde.workspace = true
serde_json.workspace = true
chrono.workspace = true
[dev-dependencies]
tokio = { workspace = true }
+186 -23
View File
@@ -4,12 +4,14 @@ use crate::{
actions::{ServiceUnit, reboot_required},
process,
};
use rmm_domain::inventory_document::DeviceType;
mod disks;
pub mod proxmox;
use rmm_domain::inventory_document::{CpuTimes, DeviceType, firmware_identifier};
use std::{
collections::BTreeMap,
ffi::{CStr, CString},
fs,
io::Read,
io::{BufRead, BufReader, Read},
mem::MaybeUninit,
net::{Ipv4Addr, Ipv6Addr},
path::Path,
@@ -169,6 +171,9 @@ pub struct OperatingSystem {
pub hostname: String,
pub boot_id: String,
pub uptime_seconds: f64,
pub booted_at: Option<chrono::DateTime<chrono::Utc>>,
pub installed_at: Option<chrono::DateTime<chrono::Utc>>,
pub install_date_source: Option<&'static str>,
}
#[derive(Debug, Clone, serde::Serialize)]
pub struct Interface {
@@ -197,6 +202,8 @@ pub struct Dns {
}
#[derive(Debug, Clone, serde::Serialize)]
pub struct Volume {
#[serde(skip)]
pub(crate) devno: String,
pub mount_point: String,
pub filesystem: String,
pub device: String,
@@ -260,6 +267,8 @@ pub struct Hardware {
pub readable_dmi: BTreeMap<String, String>,
pub device_type: Option<DeviceType>,
pub proxmox_ve: bool,
pub manufacturer: Option<String>,
pub bios_serial_number: Option<String>,
}
#[derive(Debug, Clone, serde::Serialize)]
pub struct LinuxInventory {
@@ -270,12 +279,16 @@ pub struct LinuxInventory {
pub routes: Observation<Vec<Route>>,
pub dns: Observation<Dns>,
pub volumes: Observation<Vec<Observation<Volume>>>,
pub disks: Observation<Vec<disks::Disk>>,
pub software: Observation<Vec<SoftwarePackage>>,
pub users: Observation<Vec<String>>,
pub updates: Observation<UpdateStatus>,
pub hardware: Observation<Hardware>,
pub smart: Observation<()>,
}
pub fn collect_native() -> LinuxInventory {
let volumes = volumes();
let disks = disks::collect(&volumes);
LinuxInventory {
os: operating_system(),
cpu: cpu(),
@@ -283,8 +296,10 @@ pub fn collect_native() -> LinuxInventory {
interfaces: interfaces(),
routes: routes(),
dns: dns(),
volumes: volumes(),
volumes,
disks,
software: software(),
users: logged_in_users(),
updates: updates(),
hardware: hardware(),
smart: Observation::unavailable(
@@ -380,6 +395,22 @@ pub fn capabilities() -> BTreeMap<String, Observation<bool>> {
);
result
}
fn installed_at() -> Option<(chrono::DateTime<chrono::Utc>, &'static str)> {
for (path, source) in [
("/var/log/installer", "installer_directory"),
("/", "root_filesystem"),
] {
if let Some(time) = fs::metadata(path)
.ok()
.and_then(|data| data.created().ok())
.filter(|time| *time > SystemTime::UNIX_EPOCH && *time <= SystemTime::now())
{
return Some((time.into(), source));
}
}
None
}
pub fn operating_system() -> Observation<OperatingSystem> {
let release = read("/etc/os-release", 32 * 1024);
let kernel = read("/proc/sys/kernel/osrelease", 4096);
@@ -417,8 +448,21 @@ pub fn operating_system() -> Observation<OperatingSystem> {
.map(|(k, v)| (k.into(), v.trim_matches('"').into()))
})
.collect();
let booted_at = read("/proc/stat", 256 * 1024)
.data
.as_deref()
.and_then(|stat| {
let seconds = stat
.lines()
.find_map(|line| line.strip_prefix("btime "))?
.trim()
.parse()
.ok()?;
chrono::DateTime::from_timestamp(seconds, 0)
});
let installed = installed_at();
Observation::valid(
"linux.os:/etc/os-release,/proc",
"linux.os:/etc/os-release,/proc,filesystem-birth",
OperatingSystem {
release,
kernel: kernel.trim().into(),
@@ -426,12 +470,109 @@ pub fn operating_system() -> Observation<OperatingSystem> {
hostname: hostname.trim().into(),
boot_id,
uptime_seconds,
booted_at,
installed_at: installed.map(|(time, _)| time),
install_date_source: installed.map(|(_, source)| source),
},
)
} else {
Observation::unavailable("linux.os", Quality::Error, "OS_SOURCE_UNAVAILABLE")
}
}
fn logged_in_users() -> Observation<Vec<String>> {
const LOGINCTL: &str = "/usr/bin/loginctl";
let listed = command(
LOGINCTL,
&["--no-legend", "--no-pager", "list-sessions"],
16 * 1024,
);
let Some(rows) = &listed.data else {
return map(listed, |_| None);
};
let mut ids = rows
.lines()
.filter_map(|line| line.split_whitespace().next());
let mut args = vec![
"--no-pager",
"--all",
"--property=Name,Class,State",
"show-session",
"--",
];
args.extend(
ids.by_ref()
.take(64)
.filter(|id| id.bytes().all(|c| c.is_ascii_alphanumeric())),
);
let limited = listed.truncated || ids.next().is_some();
if args.len() == 5 {
return Observation::valid("linux.logind:loginctl", Vec::new());
}
let mut report = map(command(LOGINCTL, &args, 16 * 1024), |text| {
let users: std::collections::BTreeSet<_> = text
.split("\n\n")
.filter_map(|block| {
let fields: BTreeMap<_, _> = block
.lines()
.filter_map(|line| line.split_once('='))
.collect();
let class = *fields.get("Class")?;
let state = *fields.get("State")?;
let name = *fields.get("Name")?;
(matches!(
class,
"user" | "user-early" | "user-light" | "user-early-light"
) && matches!(state, "online" | "active")
&& !name.is_empty()
&& name.len() <= 255)
.then(|| name.to_owned())
})
.collect();
Some(users.into_iter().collect())
});
report.source_id = "linux.logind:loginctl".into();
if limited && report.data.is_some() {
report = report.partial("SESSION_LIMIT");
report.truncated = true;
}
report
}
fn aggregate_jiffies(stat: &str) -> Option<Vec<u64>> {
stat.lines()
.find(|line| line.starts_with("cpu "))
.and_then(|line| {
line.split_whitespace()
.skip(1)
.map(str::parse)
.collect::<Result<_, _>>()
.ok()
})
}
fn linux_cpu_times(jiffies: &[u64]) -> Option<CpuTimes> {
// Linux already includes guest and guest_nice in user and nice.
let total = jiffies
.iter()
.take(8)
.try_fold(0_u64, |total, ticks| total.checked_add(*ticks))?;
let idle = jiffies
.get(3)?
.checked_add(jiffies.get(4).copied().unwrap_or_default())?;
Some(CpuTimes { total, idle })
}
pub fn cpu_times() -> Option<CpuTimes> {
let mut line = String::new();
BufReader::new(fs::File::open("/proc/stat").ok()?.take(4096))
.read_line(&mut line)
.ok()?;
if !line.ends_with('\n') {
return None;
}
linux_cpu_times(&aggregate_jiffies(&line)?)
}
pub fn cpu() -> Observation<Cpu> {
let info = read("/proc/cpuinfo", 1024 * 1024);
let stat = read("/proc/stat", 256 * 1024);
@@ -460,15 +601,7 @@ pub fn cpu() -> Observation<Cpu> {
.filter(|(k, _)| k.trim() == "model name")
.map(|(_, v)| v.trim().to_owned())
});
let aggregate_jiffies: Option<Vec<u64>> =
stat.lines().find(|l| l.starts_with("cpu ")).and_then(|l| {
l.split_whitespace()
.skip(1)
.map(str::parse)
.collect::<Result<Vec<_>, _>>()
.ok()
});
match aggregate_jiffies {
match aggregate_jiffies(&stat) {
Some(aggregate_jiffies) if logical_processors > 0 => Observation::valid(
"linux.cpu:/proc/cpuinfo,/proc/stat",
Cpu {
@@ -556,11 +689,12 @@ pub fn interfaces() -> Observation<Vec<Interface>> {
}
libc::AF_INET6 => {
let addr = &*item.ifa_addr.cast::<libc::sockaddr_in6>();
addresses.push(format!(
"{}%{}",
Ipv6Addr::from(addr.sin6_addr.s6_addr),
addr.sin6_scope_id
));
let ip = Ipv6Addr::from(addr.sin6_addr.s6_addr);
addresses.push(if addr.sin6_scope_id == 0 {
ip.to_string()
} else {
format!("{ip}%{}", addr.sin6_scope_id)
});
}
_ => {}
}
@@ -811,7 +945,7 @@ pub fn volumes() -> Observation<Vec<Observation<Volume>>> {
));
continue;
};
observations.push(volume(&mount, right[0], right[1]));
observations.push(volume(&mount, left[2], right[0], right[1]));
}
Some(observations)
})
@@ -829,7 +963,7 @@ impl Observation<Vec<Observation<Volume>>> {
self
}
}
fn volume(mount: &str, filesystem: &str, device: &str) -> Observation<Volume> {
fn volume(mount: &str, devno: &str, filesystem: &str, device: &str) -> Observation<Volume> {
// Remote filesystems can hang statvfs indefinitely; explicitly do not probe.
const LOCAL: &[&str] = &[
"ext2", "ext3", "ext4", "xfs", "btrfs", "tmpfs", "overlay", "vfat", "exfat", "f2fs", "zfs",
@@ -873,6 +1007,7 @@ fn volume(mount: &str, filesystem: &str, device: &str) -> Observation<Volume> {
Observation::valid(
&source,
Volume {
devno: devno.to_owned(),
mount_point: mount.into(),
filesystem: filesystem.into(),
device: mount_unescape(device).unwrap_or_else(|| device.into()),
@@ -1147,6 +1282,10 @@ fn machine_type(proxmox_ve: bool) -> Option<DeviceType> {
virtualization_type(&output)
}
pub fn proxmox_ve() -> bool {
Path::new("/usr/bin/pveversion").is_file() && Path::new("/usr/bin/pvedaemon").is_file()
}
pub fn hardware() -> Observation<Hardware> {
const SOURCE: &str = "linux.dmi:/sys/class/dmi/id;systemd-detect-virt;pve-manager";
let mut readable_dmi = BTreeMap::new();
@@ -1164,10 +1303,22 @@ pub fn hardware() -> Observation<Hardware> {
readable_dmi.insert(field.into(), value);
}
}
let proxmox_ve =
Path::new("/usr/bin/pveversion").is_file() && Path::new("/usr/bin/pvedaemon").is_file();
let proxmox_ve = proxmox_ve();
let device_type = machine_type(proxmox_ve);
if readable_dmi.is_empty() && device_type.is_none() {
let manufacturer = readable_dmi
.get("sys_vendor")
.and_then(|value| firmware_identifier(value));
let serial_credential = std::env::var_os("CREDENTIALS_DIRECTORY")
.map(|directory| std::path::PathBuf::from(directory).join("rmm-bios-serial"));
let bios_serial_number = [
Some("/sys/class/dmi/id/product_serial"),
Some("/proc/device-tree/serial-number"),
serial_credential.as_deref().and_then(Path::to_str),
]
.into_iter()
.flatten()
.find_map(|path| sys_text(path).and_then(|value| firmware_identifier(&value)));
if readable_dmi.is_empty() && device_type.is_none() && bios_serial_number.is_none() {
Observation::unavailable(SOURCE, Quality::Unsupported, "HARDWARE_UNAVAILABLE")
} else {
Observation::valid(
@@ -1176,6 +1327,8 @@ pub fn hardware() -> Observation<Hardware> {
readable_dmi,
device_type,
proxmox_ve,
manufacturer,
bios_serial_number,
},
)
.partial("NO_SMART_OR_ENCRYPTION_PROOF")
@@ -1400,6 +1553,16 @@ mod tests {
assert_eq!(request.since_filter, "@100.123457");
assert_eq!(request.until_filter, "@101.987654");
}
#[test]
fn cpu_time_excludes_double_counted_guest_ticks_and_io_wait() {
let before = linux_cpu_times(&[10, 0, 5, 60, 10, 0, 0, 0, 4, 0]).unwrap();
let after = linux_cpu_times(&[30, 0, 5, 120, 30, 0, 0, 0, 20, 0]).unwrap();
assert_eq!(after.usage_since(before), Some(2_000));
assert_eq!(after.usage_since(after), None);
assert_eq!(before.usage_since(after), None);
assert!(linux_cpu_times(&[10, 0, 0]).is_none());
}
#[test]
fn machine_types_distinguish_proxmox_guests_and_probe_failures() {
assert_eq!(machine_type(true), Some(DeviceType::Hypervisor));
+427
View File
@@ -0,0 +1,427 @@
//! Disk capacity is a hardware measurement. Filesystem usage can only be
//! attributed where the block topology and mount identity agree.
use super::{Observation, Quality, Volume, command};
use std::{
collections::{BTreeMap, BTreeSet},
fs,
os::unix::fs::{FileTypeExt, MetadataExt},
path::Path,
};
const SOURCE: &str = "linux.sysfs:block;mountinfo;statvfs";
#[derive(Debug, Clone, serde::Serialize)]
pub struct Disk {
name: String,
kind: &'static str,
model: Option<String>,
members: BTreeSet<String>,
total_bytes: u64,
used_bytes: Option<u64>,
available_bytes: Option<u64>,
}
impl Disk {
fn new(name: String, kind: &'static str, total_bytes: u64) -> Self {
Self {
name,
kind,
model: None,
members: BTreeSet::new(),
total_bytes,
used_bytes: None,
available_bytes: None,
}
}
}
struct Node {
parents: Vec<String>,
disk: bool,
stacked: bool,
label: String,
}
type Nodes = BTreeMap<String, Node>;
type Rows = BTreeMap<String, Disk>;
fn attr(path: impl AsRef<Path>) -> Option<String> {
fs::read_to_string(path)
.ok()
.map(|s| s.trim().to_owned())
.filter(|s| !s.is_empty() && s.len() <= 256)
}
fn devno(dev: u64) -> String {
format!("{}:{}", libc::major(dev), libc::minor(dev))
}
fn block_id(path: &str) -> Option<String> {
if !path.starts_with("/dev/") {
return None;
}
let m = fs::metadata(path).ok()?;
m.file_type().is_block_device().then(|| devno(m.rdev()))
}
fn topology() -> std::io::Result<(Nodes, BTreeMap<String, String>, Rows)> {
let mut nodes = Nodes::new();
let mut ids = BTreeMap::new();
let mut rows = Rows::new();
for entry in fs::read_dir("/sys/class/block")? {
let entry = entry?;
let path = entry.path();
let name = entry.file_name().to_string_lossy().into_owned();
let Some(id) = attr(path.join("dev")) else {
continue;
};
let Ok(real) = path.canonicalize() else {
continue;
};
let part = path.join("partition").exists();
let parents = if part {
real.parent()
.and_then(Path::file_name)
.map(|n| n.to_string_lossy().into_owned())
.into_iter()
.collect()
} else {
fs::read_dir(path.join("slaves"))
.into_iter()
.flatten()
.flatten()
.map(|e| e.file_name().to_string_lossy().into_owned())
.collect::<Vec<_>>()
};
// virtio/SCSI/Xen disks in guests have real bus paths too. Exclude
// synthetic block devices, optical drives and eMMC boot areas.
let disk = !part
&& parents.is_empty()
&& !real.starts_with("/sys/devices/virtual/block")
&& attr(path.join("device/type")).is_none_or(|t| t == "0" || t == "14")
&& !name.starts_with("sr")
&& !(name.starts_with("mmcblk") && name.contains("boot"));
if disk {
if let Some(size) = attr(path.join("size"))
.and_then(|s| s.parse::<u64>().ok())
.and_then(|s| s.checked_mul(512))
.filter(|n| *n > 0)
{
let mut row = Disk::new(name.clone(), "disk", size);
row.model =
attr(path.join("device/model")).or_else(|| attr(path.join("device/name")));
rows.insert(name.clone(), row);
}
}
ids.insert(id, name.clone());
nodes.insert(
name.clone(),
Node {
stacked: !part && !parents.is_empty(),
disk,
parents,
label: attr(path.join("dm/name")).unwrap_or(name),
},
);
}
Ok((nodes, ids, rows))
}
fn owners(name: &str, nodes: &Nodes) -> (BTreeSet<String>, bool) {
let mut pending = vec![name];
let mut visited = BTreeSet::new();
let mut disks = BTreeSet::new();
let mut stacked = false;
while let Some(n) = pending.pop() {
if !visited.insert(n) {
continue;
}
if let Some(node) = nodes.get(n) {
if node.disk {
disks.insert(n.to_owned());
}
stacked |= node.stacked;
pending.extend(node.parents.iter().map(String::as_str));
}
}
(disks, stacked)
}
fn project(
nodes: &Nodes,
ids: &BTreeMap<String, String>,
rows: &mut Rows,
volumes: &[&Volume],
btrfs: &BTreeMap<String, (String, BTreeSet<String>, bool)>,
) {
let mut seen = BTreeSet::new();
let mut invalid = BTreeSet::new();
for v in volumes {
if v.total_bytes == 0 || v.available_bytes > v.free_bytes || v.free_bytes > v.total_bytes {
continue;
}
let (members, key, group) = if v.filesystem == "btrfs" {
// Subvolumes can have different anonymous st_dev numbers. Group by
// the filesystem UUID and never allocate replicated chunks per disk.
let Some((uuid, members, single)) = block_id(&v.device).and_then(|id| btrfs.get(&id))
else {
continue;
};
(
members.clone(),
format!("btrfs:{uuid}"),
(!single).then(|| format!("Btrfs {uuid}")),
)
} else {
let Some(name) = ids.get(&v.devno) else {
continue;
};
let (members, stacked) = owners(name, nodes);
if members.is_empty() {
continue;
}
(
members,
v.devno.clone(),
stacked.then(|| nodes[name].label.clone()),
)
};
if !seen.insert(key.clone()) {
continue;
}
let used = v.total_bytes - v.free_bytes;
if let Some(name) = group {
let mut row = Disk::new(name, "group", v.total_bytes);
row.members = members;
row.used_bytes = Some(used);
row.available_bytes = Some(v.available_bytes);
rows.insert(format!("group:{key}"), row);
} else if members.len() == 1 {
let disk = members.first().unwrap();
if invalid.contains(disk) {
continue;
}
let Some(row) = rows.get_mut(disk) else {
continue;
};
let u = row.used_bytes.unwrap_or(0).checked_add(used);
let a = row
.available_bytes
.unwrap_or(0)
.checked_add(v.available_bytes);
if u.zip(a)
.and_then(|(u, a)| u.checked_add(a))
.is_some_and(|n| n <= row.total_bytes)
{
row.used_bytes = u;
row.available_bytes = a;
} else {
invalid.insert(disk.clone());
row.used_bytes = None;
row.available_bytes = None;
}
}
}
}
fn btrfs_members(
nodes: &Nodes,
ids: &BTreeMap<String, String>,
) -> BTreeMap<String, (String, BTreeSet<String>, bool)> {
let mut result = BTreeMap::new();
for fs in fs::read_dir("/sys/fs/btrfs")
.into_iter()
.flatten()
.flatten()
{
let mut members = BTreeSet::new();
let mut devices = Vec::new();
let mut direct = true;
for dev in fs::read_dir(fs.path().join("devices"))
.into_iter()
.flatten()
.flatten()
{
if let Some(id) = attr(dev.path().join("dev")) {
if let Some(name) = ids.get(&id) {
let (disks, stacked) = owners(name, nodes);
direct &= !stacked && disks.len() == 1;
members.extend(disks);
} else {
direct = false
}
devices.push(id);
} else {
direct = false
}
}
let single = direct && devices.len() == 1;
for id in devices {
result.insert(
id,
(
fs.file_name().to_string_lossy().into_owned(),
members.clone(),
single,
),
);
}
}
result
}
#[cfg(test)]
mod tests {
use super::*;
fn volume(id: &str, total: u64, free: u64, available: u64) -> Volume {
Volume {
devno: id.into(),
mount_point: "/".into(),
filesystem: "ext4".into(),
device: "".into(),
total_bytes: total,
free_bytes: free,
available_bytes: available,
available_percent: None,
total_inodes: 0,
free_inodes: 0,
available_inodes: 0,
available_inode_percent: None,
}
}
#[test]
fn partitions_count_once_pools_stay_separate_and_unmounted_is_unknown() {
let mut nodes = Nodes::new();
let mut rows = Rows::new();
for name in ["sda", "sdb", "vdc"] {
nodes.insert(
name.into(),
Node {
disk: true,
stacked: false,
parents: vec![],
label: name.into(),
},
);
rows.insert(name.into(), Disk::new(name.into(), "disk", 1000));
}
for (name, parents, stacked) in [
("sda1", vec!["sda"], false),
("sda2", vec!["sda"], false),
("md0", vec!["sda", "sdb"], true),
] {
nodes.insert(
name.into(),
Node {
disk: false,
stacked,
parents: parents.into_iter().map(String::from).collect(),
label: name.into(),
},
);
}
let ids = [("8:1", "sda1"), ("8:2", "sda2"), ("9:0", "md0")]
.map(|(a, b)| (a.into(), b.into()))
.into();
let a = volume("8:1", 100, 40, 30);
let b = volume("8:2", 200, 100, 80);
let raid = volume("9:0", 500, 250, 240);
project(
&nodes,
&ids,
&mut rows,
&[&a, &a, &b, &raid, &raid],
&BTreeMap::new(),
);
assert_eq!(
(rows["sda"].used_bytes, rows["sda"].available_bytes),
(Some(160), Some(110))
);
assert_eq!(rows["sdb"].used_bytes, None);
assert_eq!(rows["vdc"].available_bytes, None);
assert_eq!(
rows["group:9:0"].members,
["sda".into(), "sdb".into()].into()
);
assert_eq!(rows["group:9:0"].used_bytes, Some(250));
assert_eq!(rows.len(), 4);
}
}
fn zfs(rows: &mut Rows, nodes: &Nodes, ids: &BTreeMap<String, String>, volumes: &[&Volume]) {
let pools: BTreeSet<_> = volumes
.iter()
.filter(|v| v.filesystem == "zfs")
.filter_map(|v| v.device.split('/').next())
.collect();
if pools.is_empty() {
return;
}
// A pool's root dataset includes its children; don't sum dataset free space.
let usage = command(
"/usr/sbin/zfs",
&[
"list",
"-Hp",
"-t",
"filesystem",
"-o",
"name,used,available",
],
32768,
);
let topology = command("/usr/sbin/zpool", &["status", "-LP"], 32768);
let mut pool = "";
let mut members: BTreeMap<&str, BTreeSet<String>> = BTreeMap::new();
for line in topology
.data
.as_deref()
.unwrap_or("")
.lines()
.map(str::trim)
{
if let Some(p) = line.strip_prefix("pool: ") {
pool = p;
continue;
}
if let Some(name) = line
.split_whitespace()
.next()
.and_then(block_id)
.and_then(|id| ids.get(&id))
{
members
.entry(pool)
.or_default()
.extend(owners(name, nodes).0);
}
}
for p in pools {
let measure = usage
.data
.as_deref()
.unwrap_or("")
.lines()
.find_map(|line| {
let mut cols = line.split('\t');
if cols.next() != Some(p) {
return None;
}
let used = cols.next()?.parse::<u64>().ok()?;
let free = cols.next()?.parse::<u64>().ok()?;
Some((used, free, used.checked_add(free)?))
});
let mut row = Disk::new(format!("ZFS {p}"), "group", measure.map_or(0, |m| m.2));
row.used_bytes = measure.map(|m| m.0);
row.available_bytes = measure.map(|m| m.1);
row.members = members.remove(p).unwrap_or_default();
rows.insert(format!("group:zfs:{p}"), row);
}
}
pub(super) fn collect(volumes: &Observation<Vec<Observation<Volume>>>) -> Observation<Vec<Disk>> {
let Ok((nodes, ids, mut rows)) = topology() else {
return Observation::unavailable(SOURCE, Quality::Unknown, "BLOCK_TOPOLOGY_UNAVAILABLE");
};
let mounted: Vec<_> = volumes
.data
.iter()
.flatten()
.filter_map(|o| o.data.as_ref())
// An over-mounted or replaced mountpoint may now refer to another device.
.filter(|v| fs::metadata(&v.mount_point).is_ok_and(|m| devno(m.dev()) == v.devno))
.collect();
let btrfs = btrfs_members(&nodes, &ids);
project(&nodes, &ids, &mut rows, &mounted, &btrfs);
zfs(&mut rows, &nodes, &ids, &mounted);
let mut result: Vec<_> = rows.into_values().collect();
result.sort_by(|a, b| (a.kind, &a.name).cmp(&(b.kind, &b.name)));
Observation::valid(SOURCE, result).partial("FILESYSTEM_COVERAGE_PARTIAL")
}
+574
View File
@@ -0,0 +1,574 @@
//! Read-only Proxmox VE observation: hosted guests, scheduled vzdump jobs and recent
//! backup runs. Uses a dedicated PVEAuditor API token supplied by the installer as a
//! systemd credential. No writes, no root, no shell; the transport is injected so the
//! orchestration and parsing stay testable without a Proxmox host.
use super::{Observation, Quality};
use serde::Serialize;
use serde_json::Value;
use std::collections::{BTreeMap, BTreeSet};
pub const SOURCE: &str = "proxmox.api:/api2/json";
pub const OBSERVATION_MAX_BYTES: usize = rmm_domain::inventory_document::PROXMOX_OBSERVATION_MAX_BYTES;
pub const MAX_GUESTS: usize = 512;
pub const MAX_JOBS: usize = 64;
pub const MAX_RUNS: usize = 64;
const MAX_LOG_LINES: usize = 4000;
/// Weekly schedules must still show their previous due time.
const SCHEDULE_LOOKBACK_SECONDS: i64 = 8 * 24 * 3600;
const SCHEDULE_ITERATIONS: usize = 100;
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct Proxmox {
pub node: String,
pub cluster: Option<String>,
pub version: Option<String>,
pub guests: Vec<Guest>,
pub backup_jobs: Vec<BackupJob>,
pub backup_runs: Vec<BackupRun>,
/// Local guests that no backup job selects (Proxmox's own analysis).
pub not_backed_up: Vec<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct Guest {
pub vmid: u32,
pub name: String,
pub kind: &'static str,
pub status: String,
pub template: bool,
pub cpus: Option<u32>,
pub memory_bytes: Option<u64>,
pub disk_bytes: Option<u64>,
pub uptime_seconds: Option<u64>,
pub pool: Option<String>,
pub macs: Vec<String>,
pub smbios_uuid: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct BackupJob {
pub id: String,
pub enabled: bool,
pub schedule: Option<String>,
pub storage: Option<String>,
pub mode: Option<String>,
pub selection: &'static str,
/// Local guests this job backs up, resolved from all/vmid/pool minus exclude.
pub covered_vmids: Vec<u32>,
pub next_run: Option<i64>,
/// Most recent scheduled time at or before the observation time.
pub last_expected_run: Option<i64>,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct BackupRun {
pub upid: String,
pub started_at: i64,
pub ended_at: Option<i64>,
/// ok | warning | error | running
pub status: &'static str,
pub message: Option<String>,
pub guests: Vec<GuestResult>,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct GuestResult {
pub vmid: u32,
/// ok | error
pub result: &'static str,
}
/// Bounded GET of one `/api2/json` path, returning the `data` member.
pub trait Api {
fn get(&self, path: &str) -> impl std::future::Future<Output = anyhow::Result<Value>> + Send;
}
pub fn not_configured() -> Observation<Proxmox> {
Observation::unavailable(SOURCE, Quality::Unsupported, "PROXMOX_NOT_CONFIGURED")
}
pub fn failed(code: &str) -> Observation<Proxmox> {
Observation::unavailable(SOURCE, Quality::Error, code)
}
/// The token secret never appears in errors, logs or observations.
pub fn token_header(credential: &str) -> Option<String> {
let token = credential.trim();
let (id, secret) = token.split_once('=')?;
let (user, name) = id.split_once('!')?;
let ok = |s: &str| !s.is_empty() && s.len() <= 128
&& s.bytes().all(|b| b.is_ascii_alphanumeric() || b"-_.@".contains(&b));
(ok(user) && user.contains('@') && ok(name) && ok(secret))
.then(|| format!("PVEAPIToken={token}"))
}
pub async fn collect(api: &impl Api, now: i64) -> Observation<Proxmox> {
match collect_data(api, now).await {
Ok((data, truncated)) => {
let mut observation = Observation::valid(SOURCE, data);
if truncated {
observation = observation.partial("PROXMOX_LIMIT");
observation.truncated = true;
}
fit(observation)
}
Err(error) => {
let code = error.to_string();
failed(if code.starts_with("PROXMOX_") && code.len() <= 64 { &code } else { "PROXMOX_API_ERROR" })
}
}
}
async fn collect_data(api: &impl Api, now: i64) -> anyhow::Result<(Proxmox, bool)> {
let status = api.get("/cluster/status").await?;
let rows = status.as_array().ok_or_else(|| anyhow::anyhow!("PROXMOX_API_SHAPE"))?;
let node = rows.iter()
.find(|r| r["type"] == "node" && flag(&r["local"]) == Some(true))
.and_then(|r| r["name"].as_str())
.filter(|n| clean(n, 64))
.ok_or_else(|| anyhow::anyhow!("PROXMOX_LOCAL_NODE_UNKNOWN"))?
.to_owned();
let cluster = rows.iter().find(|r| r["type"] == "cluster")
.and_then(|r| r["name"].as_str()).filter(|n| clean(n, 64)).map(str::to_owned);
let version = api.get("/version").await.ok()
.and_then(|v| v["version"].as_str().filter(|s| clean(s, 32)).map(str::to_owned));
let mut truncated = false;
let resources = api.get("/cluster/resources?type=vm").await?;
let mut guests = Vec::new();
for row in resources.as_array().into_iter().flatten() {
if row["node"].as_str() != Some(node.as_str()) {
continue;
}
let Some(mut guest) = guest(row) else { continue };
if guests.len() == MAX_GUESTS {
truncated = true;
break;
}
let path = format!("/nodes/{node}/{}/{}/config", guest.kind, guest.vmid);
if let Ok(config) = api.get(&path).await {
guest.macs = macs(&config);
guest.smbios_uuid = config["smbios1"].as_str().and_then(smbios_uuid);
} else {
truncated = true;
}
guests.push(guest);
}
guests.sort_by_key(|g| g.vmid);
let jobs = api.get("/cluster/backup").await?;
let mut backup_jobs = Vec::new();
for row in jobs.as_array().into_iter().flatten() {
if row["type"].as_str().is_some_and(|t| t != "vzdump") {
continue;
}
if backup_jobs.len() == MAX_JOBS {
truncated = true;
break;
}
let Some(mut job) = backup_job(row, &node, &guests) else { continue };
if job.enabled && let Some(schedule) = &job.schedule {
job.last_expected_run = last_expected_run(api, schedule, now).await;
}
backup_jobs.push(job);
}
let local: BTreeSet<u32> = guests.iter().map(|g| g.vmid).collect();
let not_backed_up = api.get("/cluster/backup-info/not-backed-up").await.ok()
.and_then(|v| v.as_array().cloned()).unwrap_or_default().iter()
.filter_map(|r| vmid(&r["vmid"])).filter(|id| local.contains(id))
.collect::<BTreeSet<_>>().into_iter().collect();
let since = now - SCHEDULE_LOOKBACK_SECONDS;
let tasks = api.get(&format!(
"/nodes/{node}/tasks?typefilter=vzdump&source=all&since={since}&limit={MAX_RUNS}"
)).await?;
let mut backup_runs = Vec::new();
for row in tasks.as_array().into_iter().flatten() {
let Some(mut run) = backup_run(row) else { continue };
if backup_runs.len() == MAX_RUNS {
truncated = true;
break;
}
let encoded = percent_encode(&run.upid);
match api.get(&format!("/nodes/{node}/tasks/{encoded}/log?start=0&limit={MAX_LOG_LINES}")).await {
Ok(log) => {
let lines: Vec<&str> = log.as_array().into_iter().flatten()
.filter_map(|l| l["t"].as_str()).collect();
run.guests = guest_results(&lines);
if run.guests.is_empty() {
// A single-guest task names its guest in the UPID.
if let (Some(id), "ok" | "warning") = (upid_vmid(&run.upid), run.status) {
run.guests.push(GuestResult { vmid: id, result: "ok" });
}
}
if lines.len() >= MAX_LOG_LINES {
truncated = true;
}
}
Err(_) => truncated = true,
}
backup_runs.push(run);
}
backup_runs.sort_by(|a, b| b.started_at.cmp(&a.started_at).then_with(|| a.upid.cmp(&b.upid)));
Ok((Proxmox { node, cluster, version, guests, backup_jobs, backup_runs, not_backed_up }, truncated))
}
/// The scheduler's own calendar evaluation, so no local schedule parser can disagree.
/// The API returns at most 100 occurrences, so frequent schedules use shorter windows.
async fn last_expected_run(api: &impl Api, schedule: &str, now: i64) -> Option<i64> {
for lookback in [SCHEDULE_LOOKBACK_SECONDS, 86_400, 7_200, 600] {
let path = format!(
"/cluster/jobs/schedule-analyze?schedule={}&starttime={}&iterations={SCHEDULE_ITERATIONS}",
percent_encode(schedule), now - lookback
);
let runs: Vec<i64> = api.get(&path).await.ok()?.as_array()?.iter()
.filter_map(|r| r["timestamp"].as_i64()).collect();
if runs.len() < SCHEDULE_ITERATIONS || runs.iter().any(|t| *t > now) {
return runs.into_iter().filter(|t| *t <= now).max();
}
}
None
}
/// Keep the newest information when the scope exceeds its wire budget.
fn fit(mut observation: Observation<Proxmox>) -> Observation<Proxmox> {
let size = |o: &Observation<Proxmox>| serde_json::to_vec(o).map(|b| b.len()).unwrap_or(usize::MAX);
while size(&observation) > OBSERVATION_MAX_BYTES - 1024 {
let Some(data) = observation.data.as_mut() else { break };
if data.backup_runs.len() > 8 {
data.backup_runs.pop();
} else if !data.guests.is_empty() {
data.guests.pop();
} else {
return failed("WIRE_SCOPE_LIMIT");
}
observation.truncated = true;
observation = observation.partial("PROXMOX_LIMIT");
}
observation
}
fn guest(row: &Value) -> Option<Guest> {
let kind = match row["type"].as_str()? {
"qemu" => "qemu",
"lxc" => "lxc",
_ => return None,
};
let vmid = vmid(&row["vmid"])?;
let name = row["name"].as_str().filter(|n| clean(n, 128)).unwrap_or("").to_owned();
Some(Guest {
vmid,
name: if name.is_empty() { format!("{kind}-{vmid}") } else { name },
kind,
status: row["status"].as_str().filter(|s| clean(s, 32)).unwrap_or("unknown").to_owned(),
template: flag(&row["template"]) == Some(true),
cpus: row["maxcpu"].as_f64().filter(|c| *c >= 0.0 && *c < 65536.0).map(|c| c.ceil() as u32),
memory_bytes: row["maxmem"].as_u64(),
disk_bytes: row["maxdisk"].as_u64(),
uptime_seconds: row["uptime"].as_u64(),
pool: row["pool"].as_str().filter(|p| clean(p, 64)).map(str::to_owned),
macs: Vec::new(),
smbios_uuid: None,
})
}
fn backup_job(row: &Value, node: &str, guests: &[Guest]) -> Option<BackupJob> {
let id = row["id"].as_str().filter(|s| clean(s, 64))?.to_owned();
let text = |key: &str, limit| row[key].as_str().filter(|s| clean(s, limit)).map(str::to_owned);
let list = |key: &str| -> BTreeSet<u32> {
row[key].as_str().unwrap_or("").split([',', ' ', ';'])
.filter_map(|s| s.trim().parse().ok()).collect()
};
let exclude = list("exclude");
let pool = text("pool", 64);
let vmids = list("vmid");
let all = flag(&row["all"]) == Some(true);
let selection = if all { "all" } else if pool.is_some() { "pool" } else { "vmid" };
let node_matches = row["node"].as_str().is_none_or(|n| n == node);
let covered_vmids = guests.iter()
.filter(|g| node_matches && !g.template)
.filter(|g| match selection {
"all" => !exclude.contains(&g.vmid),
"pool" => g.pool == pool,
_ => vmids.contains(&g.vmid),
})
.map(|g| g.vmid).collect();
Some(BackupJob {
id,
enabled: flag(&row["enabled"]).unwrap_or(true),
schedule: text("schedule", 128).or_else(|| {
// Pre-7.1 format: days of week plus start time.
let time = text("starttime", 16)?;
Some(text("dow", 64).map_or(time.clone(), |dow| format!("{dow} {time}")))
}),
storage: text("storage", 64),
mode: text("mode", 16),
selection,
covered_vmids,
next_run: row["next-run"].as_i64(),
last_expected_run: None,
})
}
fn backup_run(row: &Value) -> Option<BackupRun> {
let upid = row["upid"].as_str().filter(|s| clean(s, 160))?.to_owned();
let started_at = row["starttime"].as_i64()?;
let ended_at = row["endtime"].as_i64();
let raw = row["status"].as_str();
let (status, message) = match (ended_at, raw) {
(None, _) | (_, None) => ("running", None),
(Some(_), Some("OK")) => ("ok", None),
(Some(_), Some(s)) if s.starts_with("WARNINGS") => ("warning", Some(s)),
(Some(_), Some(s)) => ("error", Some(s)),
};
Some(BackupRun {
upid,
started_at,
ended_at,
status,
message: message.map(|m| truncate(m, 256)),
guests: Vec::new(),
})
}
/// vzdump task log lines; the last statement per guest wins.
pub fn guest_results(lines: &[&str]) -> Vec<GuestResult> {
let mut results = BTreeMap::new();
for line in lines {
let line = line.trim();
if let Some(rest) = line.strip_prefix("INFO: Finished Backup of VM ")
&& let Some(id) = leading_number(rest)
{
results.insert(id, "ok");
} else if let Some(rest) = line.strip_prefix("ERROR: Backup of VM ")
&& let Some(id) = leading_number(rest)
{
results.insert(id, "error");
}
}
results.into_iter().map(|(vmid, result)| GuestResult { vmid, result }).collect()
}
/// `netN` values: qemu `virtio=AA:..,bridge=..`; lxc `name=eth0,hwaddr=AA:..`.
pub fn macs(config: &Value) -> Vec<String> {
let mut found = BTreeSet::new();
if let Some(map) = config.as_object() {
for (key, value) in map {
let is_net = key.strip_prefix("net").is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()));
let Some(value) = value.as_str().filter(|_| is_net) else { continue };
for part in value.split(',') {
if let Some((_, candidate)) = part.split_once('=')
&& let Some(mac) = normalize_mac(candidate)
{
found.insert(mac);
}
}
}
}
found.into_iter().take(32).collect()
}
pub fn smbios_uuid(value: &str) -> Option<String> {
value.split(',').find_map(|part| part.strip_prefix("uuid="))
.and_then(|u| uuid::Uuid::parse_str(u.trim()).ok())
.filter(|u| !u.is_nil())
.map(|u| u.hyphenated().to_string())
}
pub fn normalize_mac(value: &str) -> Option<String> {
let parts: Vec<&str> = value.trim().split([':', '-']).collect();
(parts.len() == 6 && parts.iter().all(|p| p.len() == 2 && p.bytes().all(|b| b.is_ascii_hexdigit())))
.then(|| parts.join(":").to_ascii_lowercase())
.filter(|m| m != "00:00:00:00:00:00")
}
/// UPID:node:pid:pstart:starttime:type:id:user: — `id` is the vmid for single-guest runs.
fn upid_vmid(upid: &str) -> Option<u32> {
upid.split(':').nth(6).and_then(|id| id.parse().ok())
}
fn vmid(value: &Value) -> Option<u32> {
value.as_u64().or_else(|| value.as_str()?.parse().ok())
.filter(|id| (100..1_000_000_000).contains(id)).map(|id| id as u32)
}
fn flag(value: &Value) -> Option<bool> {
match value {
Value::Bool(b) => Some(*b),
Value::Number(n) => n.as_u64().map(|n| n != 0),
Value::String(s) => match s.as_str() { "1" | "true" => Some(true), "0" | "false" => Some(false), _ => None },
_ => None,
}
}
fn leading_number(text: &str) -> Option<u32> {
let digits: String = text.chars().take_while(char::is_ascii_digit).collect();
digits.parse().ok()
}
fn clean(s: &str, limit: usize) -> bool {
!s.trim().is_empty() && s.len() <= limit && !s.chars().any(char::is_control)
}
fn truncate(s: &str, limit: usize) -> String {
let s: String = s.chars().filter(|c| !c.is_control()).collect();
if s.len() <= limit {
return s;
}
let mut end = limit;
while !s.is_char_boundary(end) {
end -= 1;
}
s[..end].to_owned()
}
pub fn percent_encode(value: &str) -> String {
value.bytes().map(|b| if b.is_ascii_alphanumeric() || b"-_.~".contains(&b) {
(b as char).to_string()
} else {
format!("%{b:02X}")
}).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use std::collections::HashMap;
struct Fixture(HashMap<String, Value>);
impl Api for Fixture {
async fn get(&self, path: &str) -> anyhow::Result<Value> {
let key = path.split('?').next().unwrap();
self.0.get(path).or_else(|| self.0.get(key)).cloned()
.ok_or_else(|| anyhow::anyhow!("PROXMOX_HTTP_404"))
}
}
fn block<F: std::future::Future>(future: F) -> F::Output {
tokio::runtime::Builder::new_current_thread().build().unwrap().block_on(future)
}
const NOW: i64 = 1_763_640_000;
const UPID_JOB: &str = "UPID:pve:00001234:00ABCDEF:691EE8A0:vzdump::root@pam:";
const UPID_ONE: &str = "UPID:pve:00001235:00ABCDF0:691EE000:vzdump:101:root@pam:";
fn fixture() -> Fixture {
let mut m = HashMap::new();
m.insert("/cluster/status".into(), json!([
{"type":"cluster","name":"lab"},
{"type":"node","name":"pve","local":1},
{"type":"node","name":"pve2","local":0}]));
m.insert("/version".into(), json!({"version":"9.2.4"}));
m.insert("/cluster/resources".into(), json!([
{"type":"qemu","vmid":100,"name":"debian","node":"pve","status":"running","maxcpu":2,"maxmem":1073741824u64,"maxdisk":10737418240u64,"uptime":42,"template":0},
{"type":"lxc","vmid":101,"name":"alpine","node":"pve","status":"stopped","maxcpu":1,"maxmem":134217728,"pool":"lab"},
{"type":"qemu","vmid":9000,"name":"tpl","node":"pve","status":"stopped","template":1},
{"type":"qemu","vmid":200,"name":"elsewhere","node":"pve2","status":"running"}]));
m.insert("/nodes/pve/qemu/100/config".into(), json!({
"net0":"virtio=BC:24:11:AA:BB:01,bridge=vmbr0,firewall=1",
"net1":"e1000=bc-24-11-aa-bb-02,bridge=vmbr1",
"smbios1":"uuid=6E1A2B3C-0000-4000-8000-0000000000AA",
"netmask":"virtio=BC:24:11:FF:FF:FF"}));
m.insert("/nodes/pve/lxc/101/config".into(), json!({
"net0":"name=eth0,bridge=vmbr0,hwaddr=BC:24:11:AA:BB:03,ip=dhcp,type=veth"}));
m.insert("/nodes/pve/qemu/9000/config".into(), json!({}));
m.insert("/cluster/backup".into(), json!([
{"id":"backup-a","type":"vzdump","schedule":"*/2:00","storage":"local","mode":"snapshot","vmid":"100,200","enabled":1,"next-run":NOW+3600},
{"id":"backup-all","type":"vzdump","schedule":"sat 03:00","all":1,"exclude":"100","enabled":0},
{"id":"backup-pool","type":"vzdump","dow":"mon,tue","starttime":"02:30","pool":"lab"},
{"id":"backup-other-node","type":"vzdump","schedule":"daily","all":1,"node":"pve2"}]));
m.insert("/cluster/jobs/schedule-analyze".into(), json!([
{"timestamp":NOW-7200},{"timestamp":NOW-1},{"timestamp":NOW+7199}]));
m.insert("/cluster/backup-info/not-backed-up".into(), json!([{"vmid":101},{"vmid":200}]));
m.insert("/nodes/pve/tasks".into(), json!([
{"upid":UPID_JOB,"starttime":NOW-600,"endtime":NOW-500,"status":"job errors"},
{"upid":UPID_ONE,"starttime":NOW-9000,"endtime":NOW-8900,"status":"OK"},
{"upid":"UPID:pve:1:2:3:vzdump::root@pam:","starttime":NOW-10}]));
m.insert(format!("/nodes/pve/tasks/{}/log", percent_encode(UPID_JOB)), json!([
{"n":1,"t":"INFO: starting new backup job: vzdump 100 101 --storage local"},
{"n":2,"t":"INFO: Starting Backup of VM 100 (qemu)"},
{"n":3,"t":"INFO: Finished Backup of VM 100 (00:00:12)"},
{"n":4,"t":"ERROR: Backup of VM 101 failed - storage full"}]));
m.insert(format!("/nodes/pve/tasks/{}/log", percent_encode(UPID_ONE)), json!([
{"n":1,"t":"INFO: Backup job finished successfully"}]));
m.insert("/nodes/pve/tasks/UPID%3Apve%3A1%3A2%3A3%3Avzdump%3A%3Aroot%40pam%3A/log".into(), json!([]));
Fixture(m)
}
#[test]
fn collects_local_guests_jobs_and_runs() {
let observation = block(collect(&fixture(), NOW));
assert_eq!(observation.quality, Quality::Valid);
let data = observation.data.unwrap();
assert_eq!(data.node, "pve");
assert_eq!(data.cluster.as_deref(), Some("lab"));
assert_eq!(data.guests.iter().map(|g| g.vmid).collect::<Vec<_>>(), [100, 101, 9000]);
let vm = &data.guests[0];
assert_eq!(vm.macs, ["bc:24:11:aa:bb:01", "bc:24:11:aa:bb:02"]);
assert_eq!(vm.smbios_uuid.as_deref(), Some("6e1a2b3c-0000-4000-8000-0000000000aa"));
assert_eq!(data.guests[1].macs, ["bc:24:11:aa:bb:03"]);
let jobs: BTreeMap<_, _> = data.backup_jobs.iter().map(|j| (j.id.as_str(), j)).collect();
assert_eq!(jobs["backup-a"].covered_vmids, [100]);
assert_eq!(jobs["backup-a"].last_expected_run, Some(NOW - 1));
assert!(!jobs["backup-all"].enabled);
assert_eq!(jobs["backup-all"].covered_vmids, [101]);
assert_eq!(jobs["backup-all"].last_expected_run, None);
assert_eq!(jobs["backup-pool"].schedule.as_deref(), Some("mon,tue 02:30"));
assert_eq!(jobs["backup-pool"].covered_vmids, [101]);
assert!(jobs["backup-other-node"].covered_vmids.is_empty());
assert_eq!(data.not_backed_up, [101]);
assert_eq!(data.backup_runs[0].status, "running");
assert_eq!(data.backup_runs[1].status, "error");
assert_eq!(data.backup_runs[1].guests,
[GuestResult { vmid: 100, result: "ok" }, GuestResult { vmid: 101, result: "error" }]);
assert_eq!(data.backup_runs[2].guests, [GuestResult { vmid: 101, result: "ok" }]);
}
#[test]
fn agent_document_passes_the_server_validator() {
let observation = block(collect(&fixture(), NOW));
let data = serde_json::to_value(observation.data.unwrap()).unwrap();
rmm_domain::inventory_document::proxmox::validate(&data).unwrap();
assert!(rmm_domain::inventory_document::proxmox::backup_status(&data).is_some());
}
#[test]
fn missing_local_node_or_api_is_an_error_observation() {
let observation = block(collect(&Fixture(HashMap::new()), NOW));
assert_eq!(observation.quality, Quality::Error);
assert_eq!(observation.error_code.as_deref(), Some("PROXMOX_HTTP_404"));
assert!(observation.data.is_none());
assert_eq!(not_configured().error_code.as_deref(), Some("PROXMOX_NOT_CONFIGURED"));
}
#[test]
fn oversized_observation_keeps_newest_runs_and_marks_truncation() {
let mut data = block(collect(&fixture(), NOW)).data.unwrap();
let guest = data.guests[0].clone();
data.guests = (0..512).map(|i| Guest { vmid: 100 + i, name: "x".repeat(120), ..guest.clone() }).collect();
let observation = fit(Observation::valid(SOURCE, data));
assert!(observation.truncated);
assert!(serde_json::to_vec(&observation).unwrap().len() <= OBSERVATION_MAX_BYTES);
assert_eq!(observation.data.unwrap().backup_runs.len(), 3);
}
#[test]
fn token_header_rejects_malformed_credentials() {
assert_eq!(token_header("rmm-agent@pve!inventory=0f3c-11\n").as_deref(),
Some("PVEAPIToken=rmm-agent@pve!inventory=0f3c-11"));
for bad in ["", "rmm-agent@pve=abc", "rmm-agent!x=abc", "a@pve!x=a b", "a@pve!x=a\r\nX: y"] {
assert!(token_header(bad).is_none(), "{bad}");
}
}
#[test]
fn mac_and_uuid_parsing_is_strict() {
assert_eq!(normalize_mac("BC:24:11:0a:0B:0c").as_deref(), Some("bc:24:11:0a:0b:0c"));
assert!(normalize_mac("bc:24:11:0a:0b").is_none());
assert!(normalize_mac("00:00:00:00:00:00").is_none());
assert!(smbios_uuid("manufacturer=QEMU").is_none());
assert!(smbios_uuid("uuid=00000000-0000-0000-0000-000000000000").is_none());
}
}
+2 -1
View File
@@ -27,7 +27,8 @@ pub fn apt_planning_metadata() -> Result<Vec<u8>> {
Ok(output.stdout)
}
/// Read-only local collection. No shell, package manager or hypervisor access.
/// Read-only local collection. No shell or package manager; Proxmox VE is read only
/// through its local API with a PVEAuditor token (`inventory::proxmox`).
pub fn collect() -> Result<Inventory> {
let os = fs::read_to_string("/etc/os-release").context("read OS release")?;
let hostname = fs::read_to_string("/proc/sys/kernel/hostname")?;
+44 -1
View File
@@ -114,7 +114,7 @@ pub(crate) fn run_controlled(
let state = unsafe {
libc::waitid(
libc::P_PID,
child.id(),
child.id() as libc::id_t,
info.as_mut_ptr(),
libc::WEXITED | libc::WNOHANG | libc::WNOWAIT,
)
@@ -232,6 +232,7 @@ fn drain(
}
Ok(false)
}
#[cfg(not(target_os = "freebsd"))]
fn group_has_no_live_members(pgid: u32) -> bool {
let Ok(entries) = std::fs::read_dir("/proc") else {
return false;
@@ -265,6 +266,48 @@ fn group_has_no_live_members(pgid: u32) -> bool {
true
}
#[cfg(target_os = "freebsd")]
fn group_has_no_live_members(pgid: u32) -> bool {
// FreeBSD has no Linux /proc/<pid>/stat. Ask the kernel for this
// process group while its original leader's PID is still reserved.
use std::{mem, ptr};
let mib = [
libc::CTL_KERN,
libc::KERN_PROC,
libc::KERN_PROC_PGRP,
pgid as i32,
];
let mut size = 0;
let query = |out, n: &mut usize| unsafe {
libc::sysctl(mib.as_ptr(), mib.len() as _, out, n, ptr::null(), 0)
};
let unit = mem::size_of::<libc::kinfo_proc>();
if query(ptr::null_mut(), &mut size) != 0 || size == 0 || size > 16 * 1024 * 1024 {
return false;
}
let mut rows = vec![mem::MaybeUninit::<libc::kinfo_proc>::zeroed(); size.div_ceil(unit) + 1];
size = rows.len() * unit;
if query(rows.as_mut_ptr().cast(), &mut size) != 0
|| !size.is_multiple_of(unit)
|| size / unit > rows.len()
{
return false;
}
let mut leader = false;
for row in &rows[..size / unit] {
// A successful MIB read returned this complete aligned record.
let row = unsafe { row.assume_init_ref() };
if row.ki_structsize as usize != unit
|| row.ki_pgid != pgid as i32
|| row.ki_stat != libc::SZOMB
{
return false;
}
leader |= row.ki_pid == pgid as i32;
}
leader
}
#[cfg(test)]
mod tests {
use super::*;
+62 -11
View File
@@ -1,8 +1,20 @@
//! Additive JSON observation mapping inside the existing snapshot payload. This
//! does not introduce a protobuf tag, a fabricated receive time or a new scope.
use anyhow::Result;
//! does not introduce a protobuf tag or a fabricated receive time.
use anyhow::{Result, ensure};
use rmm_domain::inventory_document::LINUX_SOFTWARE_OBSERVATION_MAX_BYTES;
use serde_json::{Value, json};
fn truncate_rows(observation: &mut Value) -> bool {
let Some(rows) = observation["data"].as_array_mut().filter(|v| !v.is_empty()) else {
return false;
};
rows.truncate(rows.len() / 2);
observation["truncated"] = json!(true);
observation["completeness"] = json!("partial");
observation["error_code"] = json!("WIRE_SCOPE_LIMIT");
true
}
pub fn observations(inventory: &crate::inventory::LinuxInventory) -> Result<Vec<Value>> {
let value = serde_json::to_value(inventory)?;
let mut result = Vec::new();
@@ -14,13 +26,16 @@ pub fn observations(inventory: &crate::inventory::LinuxInventory) -> Result<Vec<
observation["scope"] = json!(format!("linux.{name}"));
// Bound each scope independently. Never drop unsupported/error metadata
// or imply that absent/truncated rows were deleted from a device.
while serde_json::to_vec(&observation)?.len() > 32768 {
observation["truncated"] = json!(true);
observation["completeness"] = json!("partial");
observation["error_code"] = json!("WIRE_SCOPE_LIMIT");
if let Some(rows) = observation["data"].as_array_mut().filter(|v| !v.is_empty()) {
rows.truncate(rows.len() / 2);
} else {
let max_bytes = if name == "software" {
LINUX_SOFTWARE_OBSERVATION_MAX_BYTES
} else {
32768
};
while serde_json::to_vec(&observation)?.len() > max_bytes {
if !truncate_rows(&mut observation) {
observation["truncated"] = json!(true);
observation["completeness"] = json!("partial");
observation["error_code"] = json!("WIRE_SCOPE_LIMIT");
observation["data"] = Value::Null;
observation["quality"] = json!("unknown");
break;
@@ -42,6 +57,20 @@ pub fn snapshot(extended: bool) -> Result<Value> {
Ok(base)
}
pub fn encode_snapshot(mut inventory: Value) -> Result<Vec<u8>> {
let mut bytes = serde_json::to_vec(&inventory)?;
// Preserve the other scopes when a large package list fills the snapshot.
while bytes.len() > 512 * 1024 {
let software = inventory
.get_mut("observations")
.and_then(Value::as_array_mut)
.and_then(|scopes| scopes.iter_mut().find(|scope| scope["scope"] == "linux.software"));
ensure!(software.is_some_and(truncate_rows), "INVENTORY_WIRE_LIMIT");
bytes = serde_json::to_vec(&inventory)?;
}
Ok(bytes)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -49,7 +78,7 @@ mod tests {
fn native_wire_scopes_preserve_quality_time_and_unsupported() {
let native = crate::inventory::collect_native();
let mapped = observations(&native).unwrap();
assert_eq!(mapped.len(), 11);
assert_eq!(mapped.len(), 13);
let smart = mapped.iter().find(|v| v["scope"] == "linux.smart").unwrap();
assert_eq!(smart["quality"], "unsupported");
assert!(smart["data"].is_null());
@@ -61,7 +90,12 @@ mod tests {
);
assert!(!observation["source_id"].as_str().unwrap().is_empty());
assert!(observation.get("received_at").is_none());
assert!(serde_json::to_vec(observation).unwrap().len() <= 32768);
let limit = if observation["scope"] == "linux.software" {
LINUX_SOFTWARE_OBSERVATION_MAX_BYTES
} else {
32768
};
assert!(serde_json::to_vec(observation).unwrap().len() <= limit);
}
let base = snapshot(false).unwrap();
assert!(serde_json::from_value::<rmm_domain::Inventory>(base).is_ok());
@@ -87,4 +121,21 @@ mod tests {
assert_eq!(software["truncated"], true);
assert_eq!(software["completeness"], "partial");
}
#[test]
fn snapshot_limit_shrinks_software_and_preserves_other_scopes() {
let network = json!({"scope":"linux.interfaces","data":"x".repeat(280 * 1024)});
let data = vec!["x".repeat(1024); 240];
let snapshot = json!({"observations":[
network,
{"scope":"linux.software","completeness":"complete","data":data},
]});
let bytes = encode_snapshot(snapshot).unwrap();
let bounded: Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(bounded["observations"][0], network);
assert_eq!(bounded["observations"][1]["completeness"], "partial");
assert_eq!(
bounded["observations"][1]["data"].as_array().unwrap().len(),
120
);
}
}
+24
View File
@@ -0,0 +1,24 @@
[package]
name = "rmm-opnsense-jobs"
version.workspace = true
edition.workspace = true
license.workspace = true
[target.'cfg(target_os = "freebsd")'.dependencies]
anyhow.workspace = true
chrono.workspace = true
hex.workspace = true
libc = "0.2"
prost.workspace = true
rmm-agent-core = { path = "../agent-core" }
rmm-agent-linux = { path = "../agent-linux" }
rmm-domain = { path = "../domain" }
rmm-protocol = { path = "../protocol" }
rusqlite.workspace = true
serde_json.workspace = true
serde.workspace = true
sha1 = "0.10"
uuid.workspace = true
[target.'cfg(target_os = "freebsd")'.dev-dependencies]
ed25519-dalek = "=2.2.0"
+20
View File
@@ -0,0 +1,20 @@
#[cfg(target_os = "freebsd")]
mod native;
#[cfg(target_os = "freebsd")]
mod root;
#[cfg(target_os = "freebsd")]
fn main() -> anyhow::Result<()> {
match std::env::args().nth(1).as_deref() {
Some("--root-helper") => root::serve(),
Some("--root-capabilities") => {
println!("{}", serde_json::to_string(&root::caps()?)?);
Ok(())
}
_ => native::main(),
}
}
#[cfg(not(target_os = "freebsd"))]
fn main() {
eprintln!("FreeBSD required");
std::process::exit(1);
}
+496
View File
@@ -0,0 +1,496 @@
//! One worker per authenticated Python HTTPS channel, never a network listener.
//! Executes approved service-account POSIX scripts or delegates approved root
//! scripts to the independently synchronized local helper.
use anyhow::{Context, Result, bail, ensure};
use chrono::Utc;
use prost::Message;
use rmm_agent_core::{
auth::{Binding, Outcome},
freebsd_clock::NativeClock,
jobs, ledger,
reconcile::{self, Channel, ClientSync, SyncClock},
};
use rmm_agent_linux::actions::{Interpreter, ScriptRequest, run_script_controlled};
use rmm_domain::jobs::{Action, JobPayload, hash};
use rmm_protocol::*;
use rusqlite::{Connection, params};
use std::{
env, fs,
io::{self, BufRead, Read, Write},
os::{
fd::AsRawFd,
unix::fs::{MetadataExt, OpenOptionsExt},
},
path::Path,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
thread::JoinHandle,
time::{Duration, Instant},
};
use uuid::Uuid;
#[cfg(test)]
mod tests;
fn supported(job: &JobPayload) -> bool {
job.execution_identity == "agent_service"
&& job.action.opnsense_service_script()
&& job
.action
.valid_job_limits(job.max_runtime_seconds, job.max_output_bytes)
}
struct Running {
offer: JobOffer,
grant: rmm_domain::jobs::ExecutionPermitPayload,
cancel: Arc<AtomicBool>,
task: JoinHandle<Outcome>,
}
struct Worker {
binding: Binding,
channel: Channel,
sync: ClientSync,
clock: NativeClock,
db: Connection,
ledger: Connection,
sync_due: Instant,
permit_after: Instant,
root_enabled: bool,
pending_sync: Option<SyncRequest>,
running: Option<Running>,
}
impl Drop for Worker {
fn drop(&mut self) {
if let Some(r) = self.running.take() {
r.cancel.store(true, Ordering::Release);
let _ = r.task.join();
}
}
}
impl Worker {
fn envelope(&self, payload: Payload) -> Result<Envelope> {
let seq = self.db.query_row(
"UPDATE counter SET seq=seq+1 WHERE id=1 RETURNING seq",
[],
|r| r.get(0),
)?;
Ok(Envelope {
protocol_major: 1,
message_id: Uuid::new_v4().to_string(),
agent_id: self.binding.agent_id.to_string(),
enrollment_generation: self.binding.enrollment_generation,
connection_generation: self.channel.connection_generation,
boot_id: self.channel.boot_id.to_string(),
sequence: seq,
observed_at: Utc::now().timestamp_millis(),
payload: Some(payload),
..Default::default()
})
}
fn result(&mut self, offer: &JobOffer, grant: &str, outcome: Outcome) -> Result<()> {
let mut e = self.envelope(Payload::JobResult(Default::default()))?;
let tx = self.db.transaction()?;
let rev = jobs::transition(&tx, offer, &outcome.state, grant)?;
e.payload = Some(Payload::JobResult(jobs::result(offer, rev, grant, outcome)));
jobs::event(&tx, &e)?;
tx.commit()?;
self.sync_due = Instant::now();
Ok(())
}
fn recover(&mut self) -> Result<()> {
for (offer, grant) in jobs::interrupted(&self.db, &self.ledger, &self.binding)? {
if grant.is_empty() {
jobs::interrupted_request(&self.db, &offer)?;
} else {
let saved =
jobs::recover_interrupted_outcome(&self.ledger, &self.binding, &offer, &grant)?;
self.result(
&offer,
&grant,
saved.unwrap_or_else(|| Outcome::unknown("AGENT_PROCESS_INTERRUPTED")),
)?;
}
}
Ok(())
}
fn next(&mut self) -> Result<Option<Envelope>> {
if self.running.as_ref().is_some_and(|r| r.task.is_finished()) {
let r = self.running.take().unwrap();
let o = r
.task
.join()
.unwrap_or_else(|_| Outcome::unknown("WORKER_INTERRUPTED"));
ledger::persist_outcome(&self.ledger, &r.grant, &o)?;
self.result(&r.offer, &r.grant.grant_id.to_string(), o.clone())?;
ledger::finish(&self.ledger, &r.grant, &o)?;
}
if let Some(mut e) = jobs::events(&self.db)?.into_iter().next() {
e.connection_generation = self.channel.connection_generation;
return Ok(Some(e));
}
// A refused or unqualified clock closes the effect gate, not telemetry.
if self.running.is_some() {
return Ok(None);
}
if Instant::now() >= self.sync_due {
self.sync_due = Instant::now() + Duration::from_secs(40);
if self.clock.now().is_err() {
self.sync.close();
return Ok(None);
}
let snapshot = if self.root_enabled {
Some(crate::root::snapshot(self.channel.connection_generation)?)
} else {
None
};
let p = self.sync.prepare(
&mut self.db,
&self.ledger,
self.channel.clone(),
snapshot.as_ref(),
&self.clock,
)?;
self.pending_sync = Some(p.request);
return Ok(Some(p.envelope));
}
if self.sync.can_start(&self.channel, false, &self.clock)
&& Instant::now() >= self.permit_after
&& let Some(offer) = jobs::next(&self.db)?
{
let job = self.binding.job(&offer.payload_bytes)?;
if self.clock.now()? >= job.start_before {
self.result(
&offer,
"",
jobs::no_start("expired", "START_DEADLINE_EXPIRED"),
)?;
return self.next();
}
let root = self.root_enabled && crate::root::supported(&job);
ensure!(supported(&job) || root, "UNSUPPORTED_JOB");
if !self.sync.can_start(&self.channel, root, &self.clock) {
self.sync_due = Instant::now();
return Ok(None);
}
jobs::reserve_request(&self.db, &offer)?;
return self
.envelope(Payload::ExecutionPermitRequest(ExecutionPermitRequest {
job_id: offer.job_id,
attempt_id: offer.attempt_id,
job_payload_hash: offer.job_payload_hash,
assignment_version: self.binding.assignment_version,
}))
.map(Some);
}
Ok(None)
}
fn receive(&mut self, e: Envelope) -> Result<()> {
ensure!(
valid_envelope(
&e,
&self.binding.agent_id.to_string(),
self.channel.connection_generation
) && e.enrollment_generation == self.binding.enrollment_generation,
"WRONG_CHANNEL"
);
match e.payload.context("EMPTY_REPLY")? {
Payload::Ack(a) => jobs::ack(&mut self.db, &a.message_id)?,
Payload::SyncResponse(s) => {
let q = self.pending_sync.take().context("UNEXPECTED_SYNC")?;
let applied = self.sync.apply(&mut self.db, &s, &self.clock)?;
self.sync.apply_consumption(&mut self.ledger, &self.clock)?;
if applied.helper_instruction.is_some() {
ensure!(self.root_enabled, "UNEXPECTED_HELPER");
let snapshot = crate::root::apply(&q, &s)?;
if self.sync.can_start(&self.channel, false, &self.clock) {
self.sync.confirm_helper(&snapshot)?;
}
}
}
Payload::JobOffer(o) => {
let receipt = self.envelope(Payload::JobAccepted(JobAccepted {
job_id: o.job_id.clone(),
attempt_id: o.attempt_id.clone(),
job_payload_hash: o.job_payload_hash.clone(),
journal_proof: hash(&o.encode_to_vec()),
}))?;
let tx = self.db.transaction()?;
let fresh = jobs::accept(&tx, &self.binding, &o, |j| {
supported(j) || (self.root_enabled && crate::root::supported(j))
})?;
jobs::event(&tx, &receipt)?;
tx.commit()?;
if fresh {
self.sync_due = Instant::now();
}
}
Payload::ExecutionPermit(p) => self.permit(p)?,
Payload::CancelRequest(c) => {
if let Some(r) = &self.running
&& r.offer.job_id == c.job_id
&& r.offer.attempt_id == c.attempt_id
{
r.cancel.store(true, Ordering::Release);
}
let offer:Option<Vec<u8>>=rusqlite::OptionalExtension::optional(self.db.query_row(
"UPDATE job_attempts SET cancel_requested=1 WHERE job=?1 AND attempt=?2 AND state='accepted' RETURNING offer",
params![c.job_id,c.attempt_id],|r|r.get(0)))?;
if let Some(bytes) = offer {
self.result(
&JobOffer::decode(bytes.as_slice())?,
"",
jobs::no_start("cancelled", "CANCELLED_BEFORE_START"),
)?;
}
}
_ => bail!("UNEXPECTED_JOB_MESSAGE"),
}
Ok(())
}
fn permit(&mut self, p: ExecutionPermit) -> Result<()> {
let offer = jobs::requesting(&self.db)?.context("UNSOLICITED_PERMIT")?;
if !p.denial_code.is_empty() {
jobs::denied_request(&self.db, &offer, &p)?;
self.sync.close();
self.sync_due = Instant::now();
self.permit_after = Instant::now() + Duration::from_secs(30);
return Ok(());
}
let req = jobs::permit_request(&offer, &p)?;
let root = self.binding.job(&offer.payload_bytes)?.execution_identity == "root";
let verify = || -> Result<_> {
ensure!(
self.sync.can_start(&self.channel, root, &self.clock),
"SYNC_REQUIRED"
);
let (low, high) = self.clock.bounds()?;
self.binding.verify(&req, high)?;
self.binding.verify(&req, low)
};
// The signed not_before is the reservation instant. Wait within the
// qualified uncertainty interval before checking both ends at start.
let began = Instant::now();
let (job, grant) = loop {
match verify() {
Ok(x) => break x,
Err(err)
if err.to_string() == "PERMIT_TIME"
&& began.elapsed() < Duration::from_secs(6) =>
{
std::thread::sleep(Duration::from_millis(50));
}
Err(e) => return Err(e),
}
};
ensure!(
supported(&job) || (self.root_enabled && crate::root::supported(&job)),
"UNSUPPORTED_JOB"
);
let Action::RunScript {
script_utf8,
arguments,
expected_stdout,
..
} = &job.action
else {
bail!("SCRIPT_REQUIRED")
};
let script = ScriptRequest::new(
script_utf8.as_bytes().to_vec(),
Interpreter::PosixSh,
arguments.clone(),
Duration::from_secs(job.max_runtime_seconds.into()),
job.max_output_bytes as usize,
)?;
ledger::begin_execution(
&mut self.ledger,
&grant,
&req,
self.channel.boot_id,
self.clock.now()?,
)?;
let id = grant.grant_id.to_string();
let mut running = self.envelope(Payload::JobStateChanged(Default::default()))?;
let tx = self.db.transaction()?;
let revision = jobs::transition(&tx, &offer, "running", &id)?;
running.payload = Some(Payload::JobStateChanged(JobStateChanged {
job_id: offer.job_id.clone(),
attempt_id: offer.attempt_id.clone(),
grant_id: id.clone(),
revision,
state: "running".into(),
}));
jobs::event(&tx, &running)?;
tx.commit()?;
let cancel = Arc::new(AtomicBool::new(false));
let flag = cancel.clone();
let deadline = self.sync.validity_deadline().context("SYNC_LEASE")?;
let binding = self.binding.clone();
let generation = self.channel.connection_generation;
let expected = expected_stdout.clone();
let seconds = job.max_runtime_seconds;
let digest = hash(script_utf8.as_bytes());
let clock = self.clock.clone();
let task = std::thread::spawn(move || {
if root {
return crate::root::execute(generation, req, &flag, seconds)
.unwrap_or_else(|_| Outcome::unknown("HELPER_TRANSPORT_EFFECT_UNKNOWN"));
}
let mut first = true;
let result = run_script_controlled(&script, || {
ensure!(!flag.load(Ordering::Acquire), "CANCEL_OR_FENCE");
if first {
let (lo, hi) = clock.bounds()?;
ensure!(hi < deadline, "SYNC_EXPIRED");
binding.verify(&req, lo)?;
binding.verify(&req, hi)?;
first = false;
} else {
clock.now()?;
}
Ok(())
});
match result {
Ok(o) => {
let ok = o.code == Some(0)
&& !o.timed_out
&& !o.interrupted
&& expected
.as_ref()
.is_none_or(|v| !o.truncated && o.stdout == v.as_bytes());
Outcome {
state: if ok {
"succeeded"
} else if o.interrupted || o.timed_out || o.code.is_none() {
"unknown"
} else {
"failed"
}
.into(),
stdout: o.stdout,
stderr: o.stderr,
output_truncated: o.truncated,
reboot_required: false,
exit_code: o.code,
evidence: serde_json::json!({"postconditions_passed":ok,"effective_uid":unsafe{libc::geteuid()},"script_sha256":digest}),
error_code: if ok { "" } else { "PROBE_POSTCONDITION_FAILED" }.into(),
}
}
Err(_) => Outcome::unknown("PROBE_EFFECT_UNCERTAIN"),
}
});
self.running = Some(Running {
offer,
grant,
task,
cancel,
});
Ok(())
}
}
pub fn main() -> Result<()> {
let args = env::args().skip(1).collect::<Vec<_>>();
ensure!(args.len() == 4, "bound channel required");
let uid = unsafe { libc::geteuid() };
ensure!(
uid != 0 && uid == unsafe { libc::getuid() },
"SERVICE_ACCOUNT_REQUIRED"
);
unsafe { libc::umask(0o077) };
let state = Path::new("/var/db/discovery-agent");
let m = fs::symlink_metadata(state)?;
ensure!(
m.is_dir() && m.uid() == uid && m.mode() & 0o077 == 0,
"PRIVATE_STATE_REQUIRED"
);
let lock = fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(false)
.mode(0o600)
.custom_flags(libc::O_NOFOLLOW)
.open(state.join("job-worker.lock"))?;
ensure!(
unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_EX | libc::LOCK_NB) } == 0,
"ACTIVE_JOB_WORKER"
);
let config = Path::new("/usr/local/etc/discovery-agent/job-binding.json");
for path in [config.parent().unwrap(), config] {
let m = fs::symlink_metadata(path)?;
ensure!(
!m.is_symlink() && m.uid() == 0 && m.mode() & 0o022 == 0,
"PROTECTED_BINDING_REQUIRED"
);
}
let binding: Binding = serde_json::from_slice(&fs::read(config)?)?;
ensure!(
args[0] == binding.agent_id.to_string() && args[1] == binding.asset_id.to_string(),
"WRONG_DEVICE"
);
let channel = Channel {
boot_id: args[2].parse()?,
connection_generation: args[3].parse()?,
};
ensure!(
channel.connection_generation > 0 && !channel.boot_id.is_nil(),
"CHANNEL_REQUIRED"
);
let db = Connection::open(state.join("channel.sqlite"))?;
ledger::bounded_journal_capacity(&db)?;
db.execute_batch("PRAGMA synchronous=FULL;
CREATE TABLE IF NOT EXISTS counter(id INTEGER PRIMARY KEY CHECK(id=1),seq INTEGER NOT NULL);
INSERT OR IGNORE INTO counter SELECT id,n FROM sequence;
UPDATE counter SET seq=max(seq,(SELECT n FROM sequence WHERE id=1)) WHERE id=1;
CREATE TRIGGER IF NOT EXISTS telemetry_sequence AFTER UPDATE ON sequence WHEN NEW.n>(SELECT seq FROM counter WHERE id=1)
BEGIN UPDATE counter SET seq=NEW.n WHERE id=1; END;
CREATE TRIGGER IF NOT EXISTS job_sequence AFTER UPDATE ON counter WHEN NEW.seq>(SELECT n FROM sequence WHERE id=1)
BEGIN UPDATE sequence SET n=NEW.seq WHERE id=1; END;
CREATE TABLE IF NOT EXISTS job_attempts(job TEXT NOT NULL,attempt TEXT NOT NULL,offer BLOB NOT NULL,state TEXT NOT NULL,
revision INTEGER NOT NULL DEFAULT 0,grant_id TEXT NOT NULL DEFAULT '',cancel_requested INTEGER NOT NULL DEFAULT 0,PRIMARY KEY(job,attempt));
CREATE TABLE IF NOT EXISTS job_events(id TEXT PRIMARY KEY,payload BLOB NOT NULL);")?;
reconcile::initialize(&db)?;
let ledger = Connection::open(state.join("consumption.sqlite"))?;
ledger::initialize_ledger(&ledger)?;
let mut w = Worker {
sync: ClientSync::new(binding.clone()),
binding,
db,
ledger,
channel,
clock: NativeClock::default(),
sync_due: Instant::now(),
permit_after: Instant::now(),
pending_sync: None,
running: None,
root_enabled: env::var_os("DISCOVERY_ROOT_SESSION").is_some(),
};
w.recover()?;
let mut input = io::stdin().lock();
println!("ready");
io::stdout().flush()?;
loop {
let mut line = String::new();
let n = (&mut input).take(2_100_000).read_line(&mut line)?;
if n == 0 {
return Ok(());
}
ensure!(line.ends_with('\n'), "FRAME_LIMIT");
let line = line.trim_end();
if line == "next" {
println!(
"{}",
w.next()?
.map(|e| hex::encode(e.encode_to_vec()))
.unwrap_or_default()
);
} else {
let data = hex::decode(line)?;
w.receive(Envelope::decode(data.as_slice())?)?;
println!("ok");
}
io::stdout().flush()?;
}
}
@@ -0,0 +1,148 @@
use super::*;
use chrono::TimeDelta;
use ed25519_dalek::{Signer, SigningKey};
use serde_json::json;
/// Completely isolated identity, signing key, journal and grant. Never contacts
/// the gateway or reads a device key; run the fixture under the guest service uid.
#[test]
fn native_probe_requires_signature_and_redelivery_does_not_rerun() -> Result<()> {
ensure!(
unsafe { libc::geteuid() } != 0,
"run as unprivileged test account"
);
let key = SigningKey::from_bytes(&[31; 32]);
let public = key.verifying_key().to_bytes();
let b = Binding {
tenant_id: Uuid::new_v4(),
asset_id: Uuid::new_v4(),
agent_id: Uuid::new_v4(),
assignment_version: 1,
enrollment_generation: 1,
public_key_hex: hex::encode(public),
};
let db = Connection::open_in_memory()?;
db.execute_batch("CREATE TABLE counter(id INTEGER PRIMARY KEY,seq INTEGER);INSERT INTO counter VALUES(1,0);
CREATE TABLE job_attempts(job TEXT,attempt TEXT,offer BLOB,state TEXT,revision INTEGER DEFAULT 0,grant_id TEXT DEFAULT '',cancel_requested INTEGER DEFAULT 0,PRIMARY KEY(job,attempt));
CREATE TABLE job_events(id TEXT PRIMARY KEY,payload BLOB);")?;
reconcile::initialize(&db)?;
let ledger = Connection::open_in_memory()?;
ledger::initialize_ledger(&ledger)?;
let mut w = Worker {
channel: Channel {
boot_id: Uuid::new_v4(),
connection_generation: 2,
},
sync: ClientSync::new(b.clone()),
binding: b.clone(),
db,
ledger,
sync_due: Instant::now(),
permit_after: Instant::now(),
clock: NativeClock::default(),
pending_sync: None,
running: None,
root_enabled: false,
};
let now = w.clock.now()?;
let job = JobPayload {
schema_version: 1,
tenant_id: b.tenant_id,
asset_id: b.asset_id,
agent_id: b.agent_id,
enrollment_generation: 1,
assignment_version: 1,
job_id: Uuid::new_v4(),
attempt_id: Uuid::new_v4(),
execution_identity: "agent_service".into(),
max_runtime_seconds: 10,
max_output_bytes: 1024,
not_before: now - TimeDelta::minutes(1),
start_before: now + TimeDelta::minutes(2),
action: Action::RunScript {
script_utf8: "uname -s\n".into(),
expected_stdout: Some("FreeBSD\n".into()),
arguments: vec![],
interpreter: "posix_sh".into(),
windows_execution_profile: None,
privileged_script_profile: None,
},
};
let bytes = serde_json::to_vec(&job)?;
let offer = JobOffer {
job_id: job.job_id.to_string(),
attempt_id: job.attempt_id.to_string(),
plan_hash: hash(b"fixture"),
authorization_id: Uuid::new_v4().to_string(),
job_payload_hash: hash(&bytes),
payload_bytes: bytes,
};
w.receive(w.envelope(Payload::JobOffer(offer.clone()))?)?;
let event = w.next()?.unwrap();
ensure!(matches!(event.payload, Some(Payload::JobAccepted(_))));
jobs::ack(&mut w.db, &event.message_id)?;
let Some(Payload::SyncRequest(r)) = w.next()?.unwrap().payload else {
bail!("expected signed sync")
};
let decision = json!({"domain":"rmm.reconciliation-decision","schema_version":1,"decision_id":Uuid::new_v4(),
"request_id":r.request_id,"request_hash":hash(&r.encode_to_vec()),"tenant_id":b.tenant_id,
"asset_id":b.asset_id,"agent_id":b.agent_id,"enrollment_generation":1,"assignment_version":1,
"connection_generation":2,"boot_id":w.channel.boot_id,"journal_id":r.journal_id,"checkpoint":r.checkpoint,
"helper_journal_id":null,"helper_checkpoint":0,"issued_at":now-TimeDelta::seconds(1),"valid_until":now+TimeDelta::seconds(59),
"ready":true,"helper_ready":false,"reason":"CONSISTENT","key_id":hash(&public),"acknowledged_message_ids":[],
"attempts":[{"job_id":job.job_id,"attempt_id":job.attempt_id,"job_payload_hash":offer.job_payload_hash,
"action":"run_script","execution_identity":"agent_service","revision":0,"outcome_state":"accepted","outcome_bytes":[],
"unlock":false,"reason":"ACCEPTED_NOT_STARTED"}]});
let raw = serde_json::to_vec(&decision)?;
w.receive(w.envelope(Payload::SyncResponse(SyncResponse {
signature: key.sign(&raw).to_bytes().to_vec(),
decision_bytes: raw,
}))?)?;
ensure!(matches!(
w.next()?.unwrap().payload,
Some(Payload::ExecutionPermitRequest(_))
));
let raw = serde_json::to_vec(
&json!({"domain":"rmm.execution-permit","schema_version":1,"grant_id":Uuid::new_v4(),
"authorization_id":offer.authorization_id,"tenant_id":b.tenant_id,"asset_id":b.asset_id,"agent_id":b.agent_id,
"enrollment_generation":1,"assignment_version":1,"job_id":job.job_id,"attempt_id":job.attempt_id,
"plan_hash":offer.plan_hash,"job_payload_hash":offer.job_payload_hash,"action":"run_script",
"execution_identity":"agent_service","not_before":now-TimeDelta::seconds(6),"start_before":now+TimeDelta::seconds(50),
"max_runtime_seconds":10,"key_id":hash(&public)}),
)?;
let permit = ExecutionPermit {
signature: key.sign(&raw).to_bytes().to_vec(),
payload_bytes: raw,
..Default::default()
};
let mut invalid = permit.clone();
invalid.signature[0] ^= 1;
ensure!(w.permit(invalid).is_err());
let count =
|db: &Connection| db.query_row("SELECT count(*) FROM consumed", [], |r| r.get::<_, i64>(0));
ensure!(count(&w.ledger)? == 0);
w.permit(permit.clone())?;
let end = Instant::now() + Duration::from_secs(10);
while w.running.is_some() {
ensure!(Instant::now() < end, "execution timeout");
w.next()?;
std::thread::yield_now();
}
w.receive(w.envelope(Payload::JobOffer(offer))?)?;
ensure!(w.permit(permit).is_err() && count(&w.ledger)? == 1);
let results = jobs::events(&w.db)?
.into_iter()
.filter_map(|e| match e.payload {
Some(Payload::JobResult(r)) => Some(r),
_ => None,
})
.collect::<Vec<_>>();
ensure!(
results.len() == 1
&& results[0].state == "succeeded"
&& results[0].exit_code == Some(0)
&& results[0].stdout == b"FreeBSD\n",
"native output mismatch"
);
Ok(())
}
+543
View File
@@ -0,0 +1,543 @@
//! FreeBSD peer credentials, boot identity and executor lock around the same
//! independent HelperSync and consumption ledger used by the Linux root helper.
use anyhow::{Context, Result, bail, ensure};
use prost::Message;
use rmm_agent_core::{
auth::{Binding, Outcome, Request},
freebsd_clock::NativeClock,
helper_reconcile::{self as h, BootSource, ExecutorLedgerGuard},
ledger,
reconcile::{HelperSnapshot, SyncClock},
};
use rmm_agent_linux::actions::{Interpreter, ScriptRequest, run_script_controlled};
use rmm_domain::jobs::{Action, FULL_MAINTENANCE_CAPABILITY, JobPayload, hash};
use rmm_protocol::{SyncAttempt, SyncRequest, SyncResponse};
use rusqlite::{Connection, OptionalExtension};
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use sha1::{Digest, Sha1};
use std::{
fs::{self, File, OpenOptions, Permissions},
io::{Read, Write},
os::{
fd::AsRawFd,
unix::{
fs::{FileTypeExt, MetadataExt, OpenOptionsExt, PermissionsExt},
net::{UnixListener, UnixStream},
},
},
path::{Path, PathBuf},
process::Command,
sync::atomic::{AtomicBool, Ordering},
time::Duration,
};
use uuid::Uuid;
const DIR: &str = "/var/db/discovery-root-jobs";
const SOCK: &str = "/var/run/discovery-root-jobs/control.sock";
pub const POLICY: &str = "/usr/local/etc/discovery-agent/root-policy.json";
const BINDING: &str = "/usr/local/etc/discovery-agent/job-binding.json";
const LIMIT: usize = 8 * 1024 * 1024;
#[derive(Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct Policy {
schema_version: u8,
profile: String,
agent_uid: u32,
agent_gid: u32,
binding_sha256: String,
helper_binary_sha256: String,
}
fn protected(path: &Path) -> Result<()> {
for p in path.ancestors() {
let m = fs::symlink_metadata(p)?;
ensure!(
m.uid() == 0
&& m.mode() & 0o022 == 0
&& !m.is_symlink()
&& (if p == path {
m.is_file() || m.is_dir()
} else {
m.is_dir()
}),
"UNPROTECTED_ROOT_PATH"
);
}
Ok(())
}
fn config() -> Result<(Policy, Binding, String)> {
protected(Path::new(POLICY))?;
protected(Path::new(BINDING))?;
let raw = fs::read(POLICY)?;
ensure!(raw.len() <= 8192, "POLICY_SIZE");
let p: Policy = serde_json::from_slice(&raw)?;
let bytes = fs::read(BINDING)?;
ensure!(
p.schema_version == 1
&& p.profile == "full_maintenance_v1"
&& p.agent_uid > 0
&& p.agent_gid > 0
&& p.binding_sha256 == hash(&bytes)
&& rmm_domain::agent_update::digest(&p.helper_binary_sha256),
"POLICY_INVALID"
);
Ok((p, serde_json::from_slice(&bytes)?, hash(&raw)))
}
struct Boot(Uuid);
impl BootSource for Boot {
fn current_boot(&self) -> Result<Uuid> {
// Same UUIDv5 derivation as the already enrolled Python agent. Obtain
// boottime independently, never trust a value supplied on the socket.
let r = Command::new("/sbin/sysctl")
.args(["-n", "kern.boottime"])
.env_clear()
.output()?;
ensure!(r.status.success(), "BOOT_UNAVAILABLE");
let s = std::str::from_utf8(&r.stdout)?
.split_once('}')
.context("BOOT_FORMAT")?
.0;
let mut sha = Sha1::new();
sha.update(self.0.as_bytes());
sha.update(s.as_bytes());
let mut b: [u8; 16] = sha.finalize()[..16].try_into()?;
b[6] = (b[6] & 15) | 0x50;
b[8] = (b[8] & 63) | 0x80;
Ok(Uuid::from_bytes(b))
}
}
struct Guard {
_lock: File,
db_file: File,
}
impl ExecutorLedgerGuard for Guard {
fn check(&self, db: &Connection) -> Result<()> {
let path: String = db.query_row(
"SELECT file FROM pragma_database_list WHERE name='main'",
[],
|r| r.get(0),
)?;
ensure!(
Path::new(&path) == Path::new(DIR).join("consumption.sqlite"),
"WRONG_HELPER_LEDGER"
);
let m = fs::symlink_metadata(path)?;
let f = self.db_file.metadata()?;
ensure!(
m.uid() == 0
&& m.mode() & 0o077 == 0
&& m.is_file()
&& (m.dev(), m.ino()) == (f.dev(), f.ino()),
"HELPER_LEDGER_REPLACED"
);
Ok(())
}
}
fn peer(s: &UnixStream) -> Result<u32> {
let (mut uid, mut gid) = (u32::MAX, u32::MAX);
ensure!(
unsafe { libc::getpeereid(s.as_raw_fd(), &mut uid, &mut gid) } == 0,
"PEER_IDENTITY"
);
Ok(uid)
}
fn frame<T: Serialize>(s: &mut UnixStream, value: &T) -> Result<()> {
let bytes = serde_json::to_vec(value)?;
ensure!(bytes.len() <= LIMIT, "IPC_SIZE");
s.write_all(&(bytes.len() as u32).to_be_bytes())?;
s.write_all(&bytes)?;
Ok(())
}
fn read<T: DeserializeOwned>(s: &mut UnixStream) -> Result<T> {
let mut len = [0; 4];
s.read_exact(&mut len)?;
let n = u32::from_be_bytes(len) as usize;
ensure!((1..=LIMIT).contains(&n), "IPC_SIZE");
let mut b = vec![0; n];
s.read_exact(&mut b)?;
Ok(serde_json::from_slice(&b)?)
}
#[derive(Serialize, Deserialize)]
enum Op {
Probe,
Snapshot(u64),
Apply(Vec<u8>, h::SignedResponse),
Execute(u64, Request),
}
#[derive(Serialize, Deserialize)]
struct Call {
nonce: Uuid,
policy: String,
image: String,
op: Op,
}
#[derive(Serialize, Deserialize)]
enum Data {
Pins,
Snapshot(h::Snapshot),
Outcome(Outcome),
}
#[derive(Serialize, Deserialize)]
struct Reply {
nonce: Uuid,
data: Data,
}
fn connect(op: Op) -> Result<(UnixStream, Uuid)> {
let (p, _, policy) = config()?;
let s = Path::new(SOCK);
protected(s.parent().unwrap())?;
let m = fs::symlink_metadata(s)?;
ensure!(
m.file_type().is_socket() && m.uid() == 0 && m.gid() == p.agent_gid && m.mode() & 7 == 0,
"HELPER_SOCKET_MODE"
);
let mut s = UnixStream::connect(s)?;
ensure!(peer(&s)? == 0, "HELPER_NOT_ROOT");
s.set_read_timeout(Some(Duration::from_secs(12)))?;
s.set_write_timeout(Some(Duration::from_secs(5)))?;
let nonce = Uuid::new_v4();
frame(
&mut s,
&Call {
nonce,
policy,
image: p.helper_binary_sha256,
op,
},
)?;
Ok((s, nonce))
}
fn call(op: Op) -> Result<Data> {
let (mut s, n) = connect(op)?;
let r: Reply = read(&mut s)?;
ensure!(r.nonce == n, "HELPER_CHALLENGE");
Ok(r.data)
}
pub fn caps() -> Result<Vec<String>> {
let (p, _, policy) = config()?;
ensure!(matches!(call(Op::Probe)?, Data::Pins), "HELPER_PROBE");
Ok(vec![
"opnsense.root.posix/1".into(),
"helper.root.v1".into(),
"root:run_script".into(),
FULL_MAINTENANCE_CAPABILITY.into(),
format!("helper.binary.sha256:{}", p.helper_binary_sha256),
format!("helper.policy.sha256:{policy}"),
])
}
fn mapped(data: Data) -> Result<HelperSnapshot> {
let Data::Snapshot(s) = data else {
bail!("HELPER_SNAPSHOT")
};
Ok(HelperSnapshot {
journal_id: s.journal_id,
checkpoint: s.checkpoint,
previous_decision_id: s.previous_decision_id,
attempts: s
.attempts
.into_iter()
.map(|a| SyncAttempt {
job_id: a.job_id.to_string(),
attempt_id: a.attempt_id.to_string(),
job_payload_hash: a.job_payload_hash,
grant_id: a.grant_id.to_string(),
consumption: a.consumption,
state: a.state,
revision: a.revision,
original_boot_id: a.original_boot_id.to_string(),
outcome_sha256: a.outcome_sha256,
outcome_bytes: a.outcome_bytes,
..Default::default()
})
.collect(),
})
}
pub fn snapshot(g: u64) -> Result<HelperSnapshot> {
mapped(call(Op::Snapshot(g))?)
}
pub fn apply(q: &SyncRequest, s: &SyncResponse) -> Result<HelperSnapshot> {
mapped(call(Op::Apply(
q.encode_to_vec(),
h::SignedResponse {
decision_bytes: s.decision_bytes.clone(),
signature: s.signature.clone(),
},
))?)
}
pub fn execute(g: u64, req: Request, cancel: &AtomicBool, seconds: u32) -> Result<Outcome> {
let (mut s, n) = connect(Op::Execute(g, req))?;
let started = std::time::Instant::now();
// Poll only for response availability. On cancellation dropping the stream
// fences the root executor, which records the actual result in its own DB.
loop {
ensure!(
!cancel.load(Ordering::Acquire)
&& started.elapsed() < Duration::from_secs(u64::from(seconds) + 12),
"CALLER_FENCED"
);
let mut fd = libc::pollfd {
fd: s.as_raw_fd(),
events: libc::POLLIN,
revents: 0,
};
let r = unsafe { libc::poll(&mut fd, 1, 100) };
if r < 0 {
return Err(std::io::Error::last_os_error().into());
}
if r != 0 {
break;
}
}
let r: Reply = read(&mut s)?;
ensure!(r.nonce == n, "HELPER_CHALLENGE");
let Data::Outcome(o) = r.data else {
bail!("HELPER_RESULT")
};
Ok(o)
}
pub fn supported(j: &JobPayload) -> bool {
j.execution_identity == "root"
&& j.action.opnsense_root_script()
&& j.action
.valid_job_limits(j.max_runtime_seconds, j.max_output_bytes)
}
struct Helper {
db: Connection,
guard: Guard,
b: Binding,
clock: NativeClock,
boot: Boot,
sync: h::HelperSync,
p: Policy,
policy: String,
image: PathBuf,
}
impl Helper {
fn invoke(&mut self, s: &mut UnixStream) -> Result<()> {
ensure!(peer(s)? == self.p.agent_uid, "AGENT_PEER_DENIED");
s.set_read_timeout(Some(Duration::from_secs(5)))?;
s.set_write_timeout(Some(Duration::from_secs(5)))?;
let Call {
nonce,
policy,
image,
op,
} = read(s)?;
ensure!(
!nonce.is_nil() && policy == self.policy && image == self.p.helper_binary_sha256,
"PIN_MISMATCH"
);
let data = match op {
Op::Probe => Data::Pins,
Op::Snapshot(g) => Data::Snapshot(self.sync.snapshot(
&mut self.db,
&self.guard,
g,
&self.clock,
&self.boot,
)?),
Op::Apply(q, r) => Data::Snapshot(
self.sync
.apply(&mut self.db, &self.guard, &r, &q, &self.clock, &self.boot)?
.snapshot,
),
Op::Execute(g, req) => {
let (lo, hi) = self.clock.bounds()?;
let (job, grant) = self.b.verify(&req, lo)?;
self.b.verify(&req, hi)?;
ensure!(supported(&job), "ROOT_PROFILE_REQUIRED");
self.guard.check(&self.db)?;
let prior:Option<Option<Vec<u8>>>=self.db.query_row("SELECT COALESCE(result,(SELECT outcome FROM execution_evidence e WHERE e.grant_id=c.grant_id)) FROM consumed c WHERE grant_id=?1",
[grant.grant_id.to_string()],|r|r.get(0)).optional()?;
if let Some(old) = prior {
Data::Outcome(match old {
Some(bytes) => serde_json::from_slice(&bytes)?,
None => Outcome::unknown("PREVIOUS_EFFECT_UNRESOLVED"),
})
} else {
let Action::RunScript {
script_utf8,
arguments,
expected_stdout,
..
} = &job.action
else {
bail!("SCRIPT_REQUIRED")
};
let script = ScriptRequest::new(
script_utf8.as_bytes().to_vec(),
Interpreter::PosixSh,
arguments.clone(),
Duration::from_secs(job.max_runtime_seconds.into()),
job.max_output_bytes as usize,
)?;
ensure!(
self.sync
.can_start(&self.db, &self.guard, g, &self.clock, &self.boot),
"HELPER_SYNC_REQUIRED"
);
let until = self.sync.validity_deadline().context("HELPER_LEASE")?;
ensure!(
hash(&fs::read(&self.image)?) == image && config()?.2 == policy,
"HELPER_POLICY_CHANGED"
);
ledger::begin_execution(
&mut self.db,
&grant,
&req,
self.boot.current_boot()?,
self.clock.now()?,
)?;
let mut first = true;
let res = run_script_controlled(&script, || {
let (lo, hi) = self.clock.bounds()?;
if first {
ensure!(hi < until, "HELPER_LEASE_EXPIRED");
self.b.verify(&req, lo)?;
self.b.verify(&req, hi)?;
first = false;
}
let mut byte = 0u8;
let rc = unsafe {
libc::recv(
s.as_raw_fd(),
(&mut byte as *mut u8).cast(),
1,
libc::MSG_PEEK | libc::MSG_DONTWAIT,
)
};
ensure!(
rc < 0
&& std::io::Error::last_os_error().kind()
== std::io::ErrorKind::WouldBlock,
"CALLER_FENCED"
);
Ok(())
});
let out = match res {
Err(_) => Outcome::unknown("HELPER_SCRIPT_EFFECT_UNKNOWN"),
Ok(o) => {
let unknown = o.interrupted || o.timed_out || o.code.is_none();
let ok = !unknown
&& o.code == Some(0)
&& expected_stdout
.as_ref()
.is_none_or(|v| !o.truncated && o.stdout == v.as_bytes());
Outcome {
state: if unknown {
"unknown"
} else if ok {
"succeeded"
} else {
"failed"
}
.into(),
stdout: o.stdout,
stderr: o.stderr,
output_truncated: o.truncated,
reboot_required: false,
exit_code: o.code,
evidence: serde_json::json!({
"postconditions_passed":ok,"script_sha256":hash(script_utf8.as_bytes()),"execution_uid":unsafe{libc::geteuid()},
"privileged_script_profile":"full_maintenance_v1","output_encoding":"raw_native_bytes","termination_does_not_prove_absence_of_effects":unknown}),
error_code: if unknown {
"HELPER_SCRIPT_EFFECT_UNKNOWN"
} else if ok {
""
} else {
"SCRIPT_POSTCONDITION_FAILED"
}
.into(),
}
}
};
ledger::finish(&self.db, &grant, &out)?;
Data::Outcome(out)
}
}
};
frame(s, &Reply { nonce, data })
}
}
pub fn serve() -> Result<()> {
ensure!(
unsafe { libc::geteuid() } == 0 && unsafe { libc::getuid() } == 0,
"ROOT_REQUIRED"
);
unsafe { libc::umask(0o077) };
let (p, b, policy) = config()?;
let image = std::env::current_exe()?;
protected(&image)?;
ensure!(
hash(&fs::read(&image)?) == p.helper_binary_sha256,
"RUNNING_IMAGE_PIN"
);
let directory = Path::new(DIR);
protected(directory)?;
ensure!(
fs::metadata(directory)?.mode() & 0o077 == 0,
"ROOT_STATE_NOT_PRIVATE"
);
let open = |name: &str| {
OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.mode(0o600)
.custom_flags(libc::O_NOFOLLOW)
.open(directory.join(name))
};
let lock = open("executor.lock")?;
ensure!(
unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_EX | libc::LOCK_NB) } == 0,
"HELPER_BUSY"
);
let guard = Guard {
_lock: lock,
db_file: open("consumption.sqlite")?,
};
let db = Connection::open(directory.join("consumption.sqlite"))?;
guard.check(&db)?;
ledger::initialize_ledger(&db)?;
let clock = NativeClock::default();
let sync = h::HelperSync::new(b.clone(), &clock)?;
let boot = Boot(b.agent_id);
let path = Path::new(SOCK);
protected(path.parent().unwrap())?;
if let Ok(m) = fs::symlink_metadata(path) {
ensure!(
m.uid() == 0 && m.file_type().is_socket(),
"UNEXPECTED_SOCKET"
);
fs::remove_file(path)?;
}
let listener = UnixListener::bind(path)?;
std::os::unix::fs::chown(path, Some(0), Some(p.agent_gid))?;
fs::set_permissions(path, Permissions::from_mode(0o660))?;
let mut helper = Helper {
db,
guard,
b,
p,
policy,
image,
clock,
boot,
sync,
};
for s in listener.incoming() {
if let Err(e) = helper.invoke(&mut s?) {
let code = e.to_string();
eprintln!(
"root helper: {}",
if code.len() < 80 && code.bytes().all(|v| v.is_ascii_uppercase() || v == b'_') {
&code
} else {
"REJECTED"
}
);
}
}
Ok(())
}
+1 -1
View File
@@ -52,7 +52,7 @@ rcgen = { version = "=0.14.10", default-features = false, features = ["crypto",
cc = "=1.11.1"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61.2", features = ["Win32_Foundation", "Win32_Security", "Win32_Security_Authorization", "Win32_Storage_FileSystem", "Win32_System_IO", "Win32_System_ApplicationInstallationAndServicing", "Win32_System_EventLog", "Win32_System_Diagnostics_ToolHelp", "Win32_System_JobObjects", "Win32_System_Pipes", "Win32_System_SystemInformation", "Win32_System_Registry", "Win32_System_Memory", "Win32_System_Services", "Win32_System_Threading", "Win32_System_Shutdown"] }
windows-sys = { version = "0.61.2", features = ["Win32_Foundation", "Win32_NetworkManagement_IpHelper", "Win32_NetworkManagement_Ndis", "Win32_Networking_WinSock", "Win32_Security", "Win32_Security_Authorization", "Win32_Storage_FileSystem", "Win32_System_IO", "Win32_System_ApplicationInstallationAndServicing", "Win32_System_EventLog", "Win32_System_Diagnostics_ToolHelp", "Win32_System_JobObjects", "Win32_System_Pipes", "Win32_System_RemoteDesktop", "Win32_System_SystemInformation", "Win32_System_Registry", "Win32_System_Memory", "Win32_System_Services", "Win32_System_Threading", "Win32_System_Shutdown"] }
[[example]]
name = "windows_reviewed_recovery"
+6
View File
@@ -2,6 +2,8 @@ use std::{env, path::PathBuf, process::Command};
fn main() {
println!("cargo:rerun-if-changed=native/w32time.idl");
println!("cargo:rerun-if-changed=native/clock_rpc.c");
println!("cargo:rerun-if-changed=native/clock_local.c");
println!("cargo:rerun-if-changed=native/clock_status.h");
println!("cargo:rerun-if-changed=native/clock_endpoints.c");
if env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("windows") {
return;
@@ -52,10 +54,12 @@ fn main() {
.include(&out)
.file(out.join("w32time_c.c"))
.file("native/clock_rpc.c")
.file("native/clock_local.c")
.file("native/clock_endpoints.c")
.compile("rmm_clock_rpc");
println!("cargo:rustc-link-lib=rpcrt4");
println!("cargo:rustc-link-lib=advapi32");
println!("cargo:rustc-link-lib=secur32");
return;
}
assert_eq!(
@@ -89,8 +93,10 @@ fn main() {
.include(&out)
.file(out.join("w32time_c.c"))
.file("native/clock_rpc.c")
.file("native/clock_local.c")
.file("native/clock_endpoints.c")
.compile("rmm_clock_rpc");
println!("cargo:rustc-link-lib=rpcrt4");
println!("cargo:rustc-link-lib=advapi32");
println!("cargo:rustc-link-lib=secur32");
}
@@ -185,6 +185,7 @@ mod native {
Payload::Heartbeat(Heartbeat {
instance_id: Uuid::new_v4().to_string(),
uptime_seconds: 1,
system_usage: None,
}),
)?;
let acknowledgement = send(client, &heartbeat).await?;
+288
View File
@@ -0,0 +1,288 @@
/* MS-W32T 2.2.17 / opnum 6 and MS-RPCE connection-oriented packet privacy.
* Keep the status RPC on the same local pipe handle whose server PID we verify
* against the Service Control Manager, before exchanging any auth tokens.
* SSPI performs the NTLM exchange and protects the RPC headers and payloads.
*/
#define _WIN32_WINNT 0x0a00
#define SECURITY_WIN32
#include <windows.h>
#include <rpc.h>
#include <security.h>
#include <string.h>
#include "clock_status.h"
enum {
RPC_HEADER = 16, CALL_HEADER = 24, SECURITY_TRAILER = 8, SIGNATURE = 16,
FIRST_LAST = 3, HEADER_SIGN = 4, FRAME_LIMIT = 4280, TOKEN_LIMIT = 16384,
STATUS_LAYOUT = 120, STATUS_FIXED_WIRE = 112, STATUS_SOURCE_PTR = 60,
};
enum { REQUEST = 0, RESPONSE = 2, BIND = 11, BIND_ACK = 12, AUTH3 = 16 };
enum { BIND_CALL_ID = 1, STATUS_CALL_ID = 2, AUTH_CONTEXT_ID = 1, STATUS_OPNUM = 6 };
static DWORD w32time_pid(void) {
SC_HANDLE manager = OpenSCManagerW(NULL, NULL, SC_MANAGER_CONNECT);
if (!manager) return 0;
SC_HANDLE service = OpenServiceW(manager, L"W32Time", SERVICE_QUERY_STATUS);
SERVICE_STATUS_PROCESS status = {0};
DWORD needed = 0;
BOOL ok = service && QueryServiceStatusEx(service, SC_STATUS_PROCESS_INFO,
(BYTE *)&status, sizeof status, &needed);
if (service) CloseServiceHandle(service);
CloseServiceHandle(manager);
return ok && status.dwCurrentState == SERVICE_RUNNING ? status.dwProcessId : 0;
}
static BOOL is_w32time(HANDLE pipe, DWORD expected_pid) {
ULONG peer_pid = 0;
return expected_pid != 0 && GetNamedPipeServerProcessId(pipe, &peer_pid)
&& peer_pid == expected_pid && w32time_pid() == expected_pid;
}
static DWORD pipe_io(HANDLE pipe, BYTE *buffer, DWORD capacity, BOOL writing,
DWORD *count, ULONGLONG deadline) {
ULONGLONG now = GetTickCount64();
if (now >= deadline) return ERROR_TIMEOUT;
OVERLAPPED io = {0};
io.hEvent = CreateEventW(NULL, TRUE, FALSE, NULL);
if (!io.hEvent) return GetLastError();
BOOL ok = writing ? WriteFile(pipe, buffer, capacity, NULL, &io)
: ReadFile(pipe, buffer, capacity, NULL, &io);
DWORD error = ok ? ERROR_SUCCESS : GetLastError();
if (error == ERROR_IO_PENDING) {
if (WaitForSingleObject(io.hEvent, (DWORD)(deadline - now)) != WAIT_OBJECT_0) {
CancelIoEx(pipe, &io);
/* Finish cancellation before releasing the stack buffer. */
GetOverlappedResult(pipe, &io, count, TRUE);
error = ERROR_TIMEOUT;
} else {
error = GetOverlappedResult(pipe, &io, count, FALSE)
? ERROR_SUCCESS : GetLastError();
}
} else if (ok) {
error = GetOverlappedResult(pipe, &io, count, FALSE)
? ERROR_SUCCESS : GetLastError();
}
CloseHandle(io.hEvent);
return error;
}
static void put16(BYTE *p, uint16_t v) { memcpy(p, &v, sizeof v); }
static void put32(BYTE *p, uint32_t v) { memcpy(p, &v, sizeof v); }
static uint16_t get16(const BYTE *p) { uint16_t v; memcpy(&v, p, sizeof v); return v; }
static uint32_t get32(const BYTE *p) { uint32_t v; memcpy(&v, p, sizeof v); return v; }
static uint64_t get64(const BYTE *p) { uint64_t v; memcpy(&v, p, sizeof v); return v; }
static int64_t get_i64(const BYTE *p) { int64_t v; memcpy(&v, p, sizeof v); return v; }
static DWORD align4(DWORD offset) { return (offset + 3) & ~3u; }
static void write_header(BYTE *p, BYTE type, BYTE flags, WORD length, WORD auth_length, DWORD call) {
memset(p, 0, RPC_HEADER);
p[0] = 5;
p[2] = type;
p[3] = flags;
p[4] = 0x10; /* little-endian NDR */
put16(p + 8, length);
put16(p + 10, auth_length);
put32(p + 12, call);
}
static void write_security_trailer(BYTE *p) {
memset(p, 0, SECURITY_TRAILER);
p[0] = RPC_C_AUTHN_WINNT;
p[1] = RPC_C_AUTHN_LEVEL_PKT_PRIVACY;
put32(p + 4, AUTH_CONTEXT_ID);
}
static BOOL valid_security_trailer(const BYTE *p) {
return p[0] == RPC_C_AUTHN_WINNT && p[1] == RPC_C_AUTHN_LEVEL_PKT_PRIVACY
&& p[3] == 0 && get32(p + 4) == AUTH_CONTEXT_ID;
}
static DWORD send_frame(HANDLE pipe, BYTE *frame, ULONGLONG deadline) {
DWORD length = get16(frame + 8);
DWORD count = 0;
DWORD error = pipe_io(pipe, frame, length, TRUE, &count, deadline);
return !error && count != length ? ERROR_INVALID_DATA : error;
}
static DWORD read_frame(HANDLE pipe, BYTE *frame, DWORD *length, BYTE type, DWORD call,
ULONGLONG deadline) {
DWORD error = pipe_io(pipe, frame, FRAME_LIMIT, FALSE, length, deadline);
if (error) return error;
if (*length < RPC_HEADER || frame[0] != 5 || frame[1] != 0 || frame[2] != type
|| (frame[3] & FIRST_LAST) != FIRST_LAST || get32(frame + 4) != 0x10
|| get16(frame + 8) != *length || get32(frame + 12) != call) {
return ERROR_INVALID_DATA;
}
return ERROR_SUCCESS;
}
static DWORD bind_status_rpc(HANDLE pipe, CredHandle *credentials, CtxtHandle *context,
ULONGLONG deadline) {
static const BYTE interface_v4_1[20] = {0x84,0xd8,0xb6,0x8f,0x88,0x23,0xd0,0x11,
0x8c,0x35,0,0xc0,0x4f,0xda,0x27,0x95,4,0,1,0};
static const BYTE ndr32_v2[20] = {0x04,0x5d,0x88,0x8a,0xeb,0x1c,0xc9,0x11,
0x9f,0xe8,0x08,0,0x2b,0x10,0x48,0x60,2};
const ULONG required = ISC_REQ_USE_DCE_STYLE | ISC_REQ_CONFIDENTIALITY
| ISC_REQ_INTEGRITY | ISC_REQ_REPLAY_DETECT | ISC_REQ_SEQUENCE_DETECT;
ULONG attributes = 0;
BYTE out[TOKEN_LIMIT] = {0};
BYTE reply[FRAME_LIMIT] = {0};
SecBuffer token = {sizeof out - 80, SECBUFFER_TOKEN, out + 80};
SecBufferDesc output = {SECBUFFER_VERSION, 1, &token};
SECURITY_STATUS status = InitializeSecurityContextW(credentials, NULL, L"", required, 0,
SECURITY_NATIVE_DREP, NULL, 0, context, &output, &attributes, NULL);
if (status != SEC_I_CONTINUE_NEEDED) return status ? (DWORD)status : ERROR_INVALID_DATA;
if (!token.cbBuffer || token.cbBuffer > sizeof out - 80) return ERROR_INVALID_DATA;
write_header(out, BIND, FIRST_LAST | HEADER_SIGN, (WORD)(80 + token.cbBuffer),
(WORD)token.cbBuffer, BIND_CALL_ID);
put16(out + 16, FRAME_LIMIT);
put16(out + 18, FRAME_LIMIT);
out[24] = 1; /* one bind context */
out[30] = 1; /* one transfer syntax */
memcpy(out + 32, interface_v4_1, sizeof interface_v4_1);
memcpy(out + 52, ndr32_v2, sizeof ndr32_v2);
write_security_trailer(out + 72);
DWORD error = send_frame(pipe, out, deadline);
if (error) return error;
DWORD length = 0;
error = read_frame(pipe, reply, &length, BIND_ACK, BIND_CALL_ID, deadline);
if (error) return error;
DWORD challenge_length = get16(reply + 10);
if (reply[3] != (FIRST_LAST | HEADER_SIGN) || length < 56
|| !challenge_length || challenge_length > length - 56) return ERROR_INVALID_DATA;
DWORD verdict = align4(26 + get16(reply + 24));
DWORD trailer = length - SECURITY_TRAILER - challenge_length;
if (verdict > trailer || trailer - verdict < 28 || reply[verdict] != 1
|| get16(reply + verdict + 4) != 0 || get16(reply + verdict + 6) != 0
|| memcmp(reply + verdict + 8, ndr32_v2, sizeof ndr32_v2)
|| !valid_security_trailer(reply + trailer)) return ERROR_INVALID_DATA;
SecBuffer challenge = {challenge_length, SECBUFFER_TOKEN, reply + trailer + SECURITY_TRAILER};
SecBufferDesc input = {SECBUFFER_VERSION, 1, &challenge};
token.pvBuffer = out + 28;
token.cbBuffer = sizeof out - 28;
status = InitializeSecurityContextW(credentials, context, L"", required, 0,
SECURITY_NATIVE_DREP, &input, 0, context, &output, &attributes, NULL);
if (status == SEC_I_COMPLETE_NEEDED) status = CompleteAuthToken(context, &output);
if (status) return (DWORD)status;
if (!token.cbBuffer || token.cbBuffer > sizeof out - 28
|| !(attributes & ISC_RET_CONFIDENTIALITY) || !(attributes & ISC_RET_INTEGRITY)) {
return ERROR_INVALID_DATA;
}
write_header(out, AUTH3, FIRST_LAST, (WORD)(28 + token.cbBuffer),
(WORD)token.cbBuffer, BIND_CALL_ID);
memset(out + RPC_HEADER, 0, 4);
write_security_trailer(out + 20);
return send_frame(pipe, out, deadline);
}
static DWORD decode_status(const BYTE *body, DWORD length, RmmClockStatus *out) {
/* Unique NDR32 status pointer, x64 MS-W32T layout ID, no auxiliary entries. */
if (length < STATUS_FIXED_WIRE + 4 || !get32(body) || get32(body + 8) != STATUS_LAYOUT
|| get32(body + 104) || get32(body + 108)) return RPC_X_BAD_STUB_DATA;
DWORD result_offset = STATUS_FIXED_WIRE;
if (get32(body + STATUS_SOURCE_PTR)) {
/* Skip the optional conformant UTF-16 string without allocating or logging it. */
if (length - result_offset < 16) return RPC_X_BAD_STUB_DATA;
DWORD maximum = get32(body + result_offset);
DWORD offset = get32(body + result_offset + 4);
DWORD count = get32(body + result_offset + 8);
if (offset || !count || count > maximum || count > (length - result_offset - 16) / 2)
return RPC_X_BAD_STUB_DATA;
result_offset += 12 + 2 * count;
if (get16(body + result_offset - 2)) return RPC_X_BAD_STUB_DATA;
result_offset = align4(result_offset);
}
if (length - 4 != result_offset) return RPC_X_BAD_STUB_DATA;
DWORD result = get32(body + result_offset);
if (result) return result;
*out = (RmmClockStatus){
.leap = get32(body + 12), .stratum = get32(body + 16), .size = STATUS_LAYOUT,
.last_sync = get64(body + 32), .root_delay = get_i64(body + 40),
.root_dispersion = get64(body + 48), .phase = get_i64(body + 64),
.state = get32(body + 72), .flags = get32(body + 76),
.last_result = get32(body + 88), .sync_age = get64(body + 96),
};
return ERROR_SUCCESS;
}
static DWORD request_status(HANDLE pipe, CtxtHandle *context, ULONGLONG deadline,
DWORD expected_pid, RmmClockStatus *out) {
SecPkgContext_Sizes sizes = {0};
SECURITY_STATUS security = QueryContextAttributesW(context, SECPKG_ATTR_SIZES, &sizes);
if (security) return (DWORD)security;
if (sizes.cbSecurityTrailer != SIGNATURE) return ERROR_INVALID_DATA;
BYTE frame[CALL_HEADER + SECURITY_TRAILER + SIGNATURE] = {0};
write_header(frame, REQUEST, FIRST_LAST, sizeof frame, SIGNATURE, STATUS_CALL_ID);
put16(frame + 22, STATUS_OPNUM); /* only the read-only operation has an encoding */
write_security_trailer(frame + CALL_HEADER);
SecBuffer parts[] = {
{CALL_HEADER, SECBUFFER_DATA | SECBUFFER_READONLY_WITH_CHECKSUM, frame},
{0, SECBUFFER_DATA, frame + CALL_HEADER},
{SECURITY_TRAILER, SECBUFFER_DATA | SECBUFFER_READONLY_WITH_CHECKSUM, frame + CALL_HEADER},
{SIGNATURE, SECBUFFER_TOKEN, frame + CALL_HEADER + SECURITY_TRAILER},
};
SecBufferDesc message = {SECBUFFER_VERSION, 4, parts};
security = EncryptMessage(context, 0, &message, 0);
if (security) return (DWORD)security;
if (parts[3].cbBuffer != SIGNATURE) return ERROR_INVALID_DATA;
DWORD error = send_frame(pipe, frame, deadline);
if (error) return error;
BYTE reply[FRAME_LIMIT] = {0};
DWORD length = 0;
error = read_frame(pipe, reply, &length, RESPONSE, STATUS_CALL_ID, deadline);
if (error) return error;
if (reply[3] != FIRST_LAST || get16(reply + 10) != SIGNATURE || get16(reply + 20) != 0
|| length < sizeof frame) return ERROR_INVALID_DATA;
DWORD trailer = length - SECURITY_TRAILER - SIGNATURE;
BYTE padding = reply[trailer + 2];
if (padding > 15 || padding > trailer - CALL_HEADER
|| !valid_security_trailer(reply + trailer)) return ERROR_INVALID_DATA;
parts[0].pvBuffer = reply;
parts[1].pvBuffer = reply + CALL_HEADER;
parts[1].cbBuffer = trailer - CALL_HEADER;
parts[2].pvBuffer = reply + trailer;
parts[3].pvBuffer = reply + trailer + SECURITY_TRAILER;
ULONG qop = 0;
security = DecryptMessage(context, &message, 0, &qop);
if (security) return (DWORD)security;
if (qop != 0) return ERROR_INVALID_DATA;
if (!is_w32time(pipe, expected_pid)) return ERROR_ACCESS_DENIED;
return decode_status(reply + CALL_HEADER, trailer - CALL_HEADER - padding, out);
}
uint32_t rmm_local_clock_query(RmmClockStatus *out) {
memset(out, 0, sizeof *out);
DWORD expected_pid = w32time_pid();
if (!expected_pid) return ERROR_SERVICE_NOT_ACTIVE;
ULONGLONG deadline = GetTickCount64() + 750;
HANDLE pipe = CreateFileW(L"\\\\.\\pipe\\W32TIME_ALT", FILE_READ_DATA | FILE_WRITE_DATA
| FILE_WRITE_ATTRIBUTES | SYNCHRONIZE, 0, NULL, OPEN_EXISTING,
FILE_FLAG_OVERLAPPED | SECURITY_SQOS_PRESENT | SECURITY_IDENTIFICATION, NULL);
if (pipe == INVALID_HANDLE_VALUE) return GetLastError();
DWORD mode = PIPE_READMODE_MESSAGE;
DWORD error;
if (!is_w32time(pipe, expected_pid)) {
error = ERROR_ACCESS_DENIED;
} else if (!SetNamedPipeHandleState(pipe, &mode, NULL, NULL)) {
error = GetLastError();
} else {
CredHandle credentials;
CtxtHandle context;
SecInvalidateHandle(&context);
error = (DWORD)AcquireCredentialsHandleW(NULL, L"NTLM", SECPKG_CRED_OUTBOUND,
NULL, NULL, NULL, NULL, &credentials, NULL);
if (!error) {
error = bind_status_rpc(pipe, &credentials, &context, deadline);
if (!error) error = request_status(pipe, &context, deadline, expected_pid, out);
if (SecIsValidHandle(&context)) DeleteSecurityContext(&context);
FreeCredentialsHandle(&credentials);
}
}
CloseHandle(pipe);
return error;
}
+1 -9
View File
@@ -9,6 +9,7 @@
#include <stdlib.h>
#include <string.h>
#include "w32time.h"
#include "clock_status.h"
/* Documented public SDK API; older MinGW headers omit its declaration. */
int RPC_ENTRY RpcExceptionFilter(unsigned long exception_code);
/* Older MinGW headers stop at V3. This is the documented x64 V5 SDK layout,
@@ -61,15 +62,6 @@ void __RPC_USER MIDL_user_free(void *memory) {
static void release_allocations(Arena *arena) {
while (arena->head) MIDL_user_free(arena->head + 1);
}
/* Stable numeric ABI, no returned peer/source strings or remotely sized data. */
typedef struct {
uint32_t leap, stratum, state, flags, last_result, size;
uint64_t last_sync, sync_age;
int64_t root_delay;
uint64_t root_dispersion;
int64_t phase;
} RmmClockStatus;
_Static_assert(sizeof(RmmClockStatus) == 64, "Rust numeric clock ABI");
_Static_assert(sizeof(RMM_W32TIME_STATUS) == 120, "MS-W32T x64 structure ABI");
/* Diagnostic-only numeric ABI. No strings, names, pointers or remote arrays
* leave C. A zero native return is meaningful only when return_present is1.
@@ -0,0 +1,17 @@
#ifndef RMM_CLOCK_STATUS_H
#define RMM_CLOCK_STATUS_H
#include <stdint.h>
/* Numeric Windows status only. No remote strings or pointers cross the ABI. */
typedef struct {
uint32_t leap, stratum, state, flags, last_result, size;
uint64_t last_sync, sync_age;
int64_t root_delay;
uint64_t root_dispersion;
int64_t phase;
} RmmClockStatus;
_Static_assert(sizeof(RmmClockStatus) == 64, "Rust numeric clock ABI");
#endif
+167 -25
View File
@@ -1,19 +1,20 @@
//! Read-only trusted clock gate for Windows 11. All MS-W32T time intervals are
//! 100 ns; last_sync alone is a FILETIME (1601 epoch), sync_age is a duration.
//! No W32Time reconfiguration, resync, unauthenticated pipe or server-time anchor.
//! Policy: <=1h successful sync age, SYNC state, stratum1..15, nonalarm leap,
//! <=2s root dispersion + roundtrip/2 + absolute phase + query latency.
//! Require a successful sync within one day, SYNC state, stratum 1..15 and no
//! leap alarm. Bound dispersion, half the round trip, phase, holdover and
//! query latency to two seconds.
//! A >1s wall step or suspend invalidates until a post-discontinuity sync.
use anyhow::{Result, anyhow, ensure};
use chrono::{DateTime, Utc};
use serde::Serialize;
use std::sync::Mutex;
/// Flip only after native account AND service-bound ALPC, positive readiness,
/// wrong-account/wrong-service denial and fail-closed qualification.
pub const ADAPTER_QUALIFIED: bool = false;
const ADAPTER_QUALIFIED: bool = cfg!(windows);
const SECOND: u64 = 10_000_000;
const MAX_AGE: u64 = 3600 * SECOND;
const MAX_AGE: u64 = 24 * 60 * 60 * SECOND;
const MAX_ERROR: u64 = 2 * SECOND;
const CACHE_AGE: u64 = 5 * SECOND;
const RETRY_AGE: u64 = SECOND;
const FILETIME_UNIX: u64 = 116_444_736_000_000_000;
#[derive(Clone, Copy, Debug, Default, Serialize)]
@@ -124,9 +125,11 @@ impl Continuity {
}
fn evaluate(status: &Status, wall: u64, query_ticks: u64) -> Result<u64> {
ensure!(status.size == 120, "CLOCK_UNSUPPORTED_LAYOUT");
// Like the Linux kernel, retain a still-qualified last good sample when
// a poll has no new data. The service's state and age still govern trust.
ensure!(
status.state == 2
&& status.last_result == 0
&& matches!(status.last_result, 0..=2)
&& status.leap <= 2
&& (1..=15).contains(&status.stratum),
"CLOCK_UNSYNCHRONIZED"
@@ -152,6 +155,7 @@ fn evaluate(status: &Status, wall: u64, query_ticks: u64) -> Result<u64> {
.root_dispersion
.checked_add((status.root_delay as u64).div_ceil(2))
.and_then(|v| v.checked_add(status.phase.unsigned_abs()))
.and_then(|v| v.checked_add(holdover_error(status.sync_age)))
.and_then(|v| v.checked_add(query_ticks))
.ok_or_else(|| anyhow!("CLOCK_UNCERTAINTY_OVERFLOW"))?;
ensure!(
@@ -160,8 +164,38 @@ fn evaluate(status: &Status, wall: u64, query_ticks: u64) -> Result<u64> {
);
Ok(bound)
}
fn holdover_error(elapsed: u64) -> u64 {
// NTP's nominal dispersion growth (RFC 5905, PHI = 15 ppm).
elapsed.saturating_mul(15).div_ceil(1_000_000)
}
#[derive(Clone, Copy)]
struct CachedQualification {
sample: Sample,
bound: Option<u64>,
valid_for: u64,
}
impl CachedQualification {
fn bound_at(self, sample: Sample) -> Option<u64> {
let elapsed = sample.interrupt.checked_sub(self.sample.interrupt)?;
let awake = sample.unbiased.checked_sub(self.sample.unbiased)?;
let wall = sample.wall.checked_sub(self.sample.wall)?;
let wall_drift = elapsed.abs_diff(wall);
let bound = self
.bound?
.checked_add(holdover_error(elapsed))?
.checked_add(wall_drift)?;
(elapsed < self.valid_for
&& elapsed.abs_diff(awake) <= SECOND
&& wall_drift <= SECOND
&& bound <= MAX_ERROR)
.then_some(bound)
}
}
pub struct NativeClock {
state: Mutex<Continuity>,
cache: Mutex<Option<CachedQualification>>,
diagnostic: bool,
}
impl Default for NativeClock {
@@ -173,6 +207,7 @@ impl NativeClock {
pub fn new() -> Self {
Self {
state: Mutex::new(Continuity::default()),
cache: Mutex::new(None),
diagnostic: false,
}
}
@@ -192,7 +227,7 @@ impl NativeClock {
transport: if self.diagnostic {
"ncacn_np:W32TIME_ALT:diagnostic_only"
} else {
"ncalrpc:SID19:W32Time:V5_descriptor:mutual_auth"
"local_pipe:W32TIME_ALT:verified_service_pid"
},
status: None,
query_ticks: ticks,
@@ -222,7 +257,7 @@ impl NativeClock {
report.status = Some(raw);
match state.validate(&raw, after, ticks) {
Ok(bound) => {
report.trusted = !self.diagnostic;
report.trusted = ADAPTER_QUALIFIED && !self.diagnostic;
report.reason = if self.diagnostic {
"CLOCK_SERVER_IDENTITY_UNQUALIFIED"
} else {
@@ -235,26 +270,66 @@ impl NativeClock {
}
report
}
pub fn now(&self) -> Result<DateTime<Utc>> {
fn qualified_sample(&self) -> Result<(Sample, u64)> {
ensure!(ADAPTER_QUALIFIED, "CLOCK_ADAPTER_UNQUALIFIED");
let mut cache = self
.cache
.lock()
.map_err(|_| anyhow!("CLOCK_STATE_POISONED"))?;
let current = sample();
let observed = self
.state
.lock()
.map_err(|_| anyhow!("CLOCK_STATE_POISONED"))?
.observe(current);
if observed.is_err() {
*cache = None;
observed?;
}
if let Some(cached) = *cache {
if let Some(bound) = cached.bound_at(current) {
return Ok((current, bound));
}
ensure!(
cached.bound.is_some()
|| current.interrupt.saturating_sub(cached.sample.interrupt) >= RETRY_AGE,
"CLOCK_RETRY_PENDING"
);
}
let report = self.report();
ensure!(report.trusted, "CLOCK_UNTRUSTED: {}", report.reason);
let ticks = report
.sample
.wall
.checked_sub(FILETIME_UNIX)
.ok_or_else(|| anyhow!("CLOCK_TIMESTAMP"))?;
DateTime::from_timestamp(
(ticks / SECOND).try_into()?,
((ticks % SECOND) * 100) as u32,
)
.ok_or_else(|| anyhow!("CLOCK_TIMESTAMP"))
let bound = report.error_bound_ticks.filter(|_| report.trusted);
*cache = Some(CachedQualification {
sample: report.sample,
bound,
valid_for: report.status.map_or(0, |status| {
CACHE_AGE.min(MAX_AGE.saturating_sub(status.sync_age))
}),
});
Ok((
report.sample,
bound.ok_or_else(|| anyhow!("CLOCK_UNTRUSTED: {}", report.reason))?,
))
}
pub fn now(&self) -> Result<DateTime<Utc>> {
utc(self.qualified_sample()?.0.wall)
}
}
fn utc(wall: u64) -> Result<DateTime<Utc>> {
let ticks = wall
.checked_sub(FILETIME_UNIX)
.ok_or_else(|| anyhow!("CLOCK_TIMESTAMP"))?;
DateTime::from_timestamp(
(ticks / SECOND).try_into()?,
((ticks % SECOND) * 100) as u32,
)
.ok_or_else(|| anyhow!("CLOCK_TIMESTAMP"))
}
/// Named-pipe diagnostics can never open the production clock gate.
pub fn named_pipe_diagnostic() -> ClockReport {
NativeClock {
state: Mutex::new(Continuity::default()),
cache: Mutex::new(None),
diagnostic: true,
}
.report()
@@ -376,6 +451,17 @@ impl rmm_agent_core::reconcile::SyncClock for NativeClock {
fn now(&self) -> Result<DateTime<Utc>> {
NativeClock::now(self)
}
fn reading(&self) -> Result<rmm_agent_core::reconcile::ClockReading> {
let (sample, bound) = self.qualified_sample()?;
Ok(rmm_agent_core::reconcile::ClockReading {
own_wall: utc(sample.wall)?,
upper: utc(sample.wall + bound)?,
})
}
fn bounds(&self) -> Result<(DateTime<Utc>, DateTime<Utc>)> {
let (sample, bound) = self.qualified_sample()?;
Ok((utc(sample.wall - bound)?, utc(sample.wall + bound)?))
}
}
#[cfg(windows)]
fn sample() -> Sample {
@@ -401,7 +487,7 @@ fn raw_query(diagnostic: bool) -> (u32, Status) {
#[link(name = "advapi32")]
unsafe extern "C" {
fn rmm_clock_query(status: *mut Status) -> u32;
fn rmm_clock_alpc_query(status: *mut Status) -> u32;
fn rmm_local_clock_query(status: *mut Status) -> u32;
}
let mut status = Status::default();
// SAFETY: numeric C ABI and contained RPC exceptions, fixed local endpoints.
@@ -409,7 +495,7 @@ fn raw_query(diagnostic: bool) -> (u32, Status) {
if diagnostic {
rmm_clock_query(&mut status)
} else {
rmm_clock_alpc_query(&mut status)
rmm_local_clock_query(&mut status)
}
};
(code, status)
@@ -522,8 +608,9 @@ mod tests {
},
)
}
#[cfg(not(windows))]
#[test]
fn production_clock_stays_closed_until_native_adapter_qualification() {
fn production_windows_clock_is_unavailable_on_other_platforms() {
assert_eq!(std::mem::size_of::<StatusRpcTrace>(), 32);
assert_eq!(
NativeClock::new().now().unwrap_err().to_string(),
@@ -559,7 +646,7 @@ mod tests {
Status { state: 1, ..s },
Status { leap: 3, ..s },
Status {
last_result: 2,
last_result: 3,
..s
},
Status { stratum: 16, ..s },
@@ -585,6 +672,61 @@ mod tests {
}
}
#[test]
fn missed_poll_keeps_a_bounded_last_good_sample() {
let (s, v) = good();
let age = 2 * 3600 * SECOND;
for last_result in [1, 2] {
let held = Status {
last_result,
last_sync: v.wall - age,
sync_age: age,
..s
};
assert!(evaluate(&held, v.wall, 100).is_ok());
}
let old = Status {
last_result: 2,
last_sync: v.wall - MAX_AGE - 1,
sync_age: MAX_AGE + 1,
..s
};
assert!(evaluate(&old, v.wall, 100).is_err());
}
#[test]
fn cached_bound_expires_and_rejects_wall_step_and_suspend() {
let (_, mut sample) = good();
let cache = CachedQualification {
sample,
bound: Some(SECOND),
valid_for: CACHE_AGE,
};
sample.wall += SECOND;
sample.interrupt += SECOND;
sample.unbiased += SECOND;
assert_eq!(cache.bound_at(sample), Some(SECOND + 150));
assert_eq!(
cache.bound_at(Sample {
wall: sample.wall + 2 * SECOND,
..sample
}),
None
);
assert_eq!(
cache.bound_at(Sample {
unbiased: sample.unbiased - 2 * SECOND,
..sample
}),
None
);
let elapsed = CACHE_AGE;
let expired = Sample {
wall: cache.sample.wall + elapsed,
interrupt: cache.sample.interrupt + elapsed,
unbiased: cache.sample.unbiased + elapsed,
};
assert_eq!(cache.bound_at(expired), None);
}
#[test]
fn suspend_closes_until_actual_post_resume_sync() {
let (s, v) = good();
let mut c = Continuity::default();
+42 -22
View File
@@ -899,13 +899,15 @@ impl SystemHelper {
let pin = settings.pin()?;
ipc::publish_self_query_rights(&settings.agent.service_sid, &settings.helper.service_sid)?;
let (guard, db) = LedgerGuard::acquire(&settings.helper.service_sid, &settings.binding)?;
let clock = if settings.windows_update_only() || settings.managed_scripts() {
let role = rmm_time_attestation::Role::SystemHelper;
let clock = if settings.managed_scripts() {
RuntimeClock::local(role)
} else if settings.windows_update_only() {
RuntimeClock::enrolled_windows_update_helper(settings.binding.clone(), settings.helper.clone())
} else { RuntimeClock::new(rmm_time_attestation::Role::SystemHelper) }.for_generation(0);
} else {
RuntimeClock::new(role)
}.for_generation(0);
let boot = WindowsBoot::capture()?;
let sync = HelperSync::with_verifier(settings.binding.clone(), DalekVerifier, &clock)
.ok()
.map(|s| s.with_helper_identity(HelperIdentity::System));
Ok(Self {
settings,
pin,
@@ -914,7 +916,8 @@ impl SystemHelper {
db,
clock,
boot,
sync,
// The initial connection creates the reconciler using its own qualified time.
sync: None,
last_diagnostic: None,
last_apply_diagnostic: None,
unsettled_msi: None,
@@ -1008,14 +1011,17 @@ impl SystemHelper {
}
fn sync(&mut self) -> Result<&mut HelperSync<DalekVerifier>> {
if self.sync.is_none() {
self.sync = Some(
HelperSync::with_verifier(
self.settings.binding.clone(),
DalekVerifier,
&self.clock,
)?
.with_helper_identity(HelperIdentity::System),
);
let sync = HelperSync::with_verifier(
self.settings.binding.clone(),
DalekVerifier,
&self.clock,
)?
.with_helper_identity(HelperIdentity::System);
self.sync = Some(if self.settings.managed_scripts() {
sync.with_periodic_idle_integrity_checks()
} else {
sync
});
}
self.sync.as_mut().context("SYSTEM_SYNC_CLOSED")
}
@@ -1290,6 +1296,16 @@ impl SystemHelper {
}
ensure!(self.settings.enable_effects, "SYSTEM_EFFECTS_DISABLED");
verified_window(&self.settings.binding, request, &self.clock)?;
if self.settings.managed_scripts() {
ledger::check_integrity(&self.db, "ROOT_LEDGER_INTEGRITY")?;
rmm_agent_core::helper_reconcile::verify_checkpoint_repair_history_with(
&self.db,
&self.settings.binding,
None,
HelperIdentity::System,
&DalekVerifier,
)?;
}
self.sync()?;
ensure!(
self.sync.as_ref().unwrap().can_start(
@@ -1795,14 +1811,18 @@ unsafe extern "system" fn service_main(_: u32, _: *mut *mut u16) {
let result = (|| -> Result<()> {
let mut helper = SystemHelper::open()?;
let server = ipc::Server::bind(helper.pin().clone())?;
let time_server = ipc::Server::bind_time(helper.pin().clone())?;
let update_server=crate::update_control::server_pin(&helper.settings.binding,&helper.pin.helper)?
.map(crate::update_control_ipc::Server::bind).transpose()?;
let time_pin = helper.pin().clone();
let time_clock = helper.clock.source();
let time_worker = std::thread::Builder::new()
.name("rmm-helper-time-control".into())
.spawn(move || time_control(time_server, time_pin, time_clock))?;
let time_worker = if time_clock.uses_local_clock() {
None
} else {
let time_pin = helper.pin().clone();
let time_server = ipc::Server::bind_time(time_pin.clone())?;
Some(std::thread::Builder::new()
.name("rmm-helper-time-control".into())
.spawn(move || time_control(time_server, time_pin, time_clock))?)
};
scm_status(SERVICE_RUNNING, 0);
let mut next_update_unknown=Instant::now();
while !STOP.load(Ordering::SeqCst) {
@@ -1879,9 +1899,9 @@ unsafe extern "system" fn service_main(_: u32, _: *mut *mut u16) {
}
// The time lane has no ledger/job operation and its receive wait is
// bounded. Shutdown closes its private evidence before service exit.
time_worker
.join()
.map_err(|_| anyhow::anyhow!("SYSTEM_TIME_WORKER_FAILED"))??;
if let Some(worker) = time_worker {
worker.join().map_err(|_| anyhow::anyhow!("SYSTEM_TIME_WORKER_FAILED"))??;
}
Ok(())
})();
scm_status(SERVICE_STOPPED, if result.is_ok() { 0 } else { 1066 });
+90 -42
View File
@@ -2,7 +2,9 @@
//! uninstall commands; registry entries are evidence, not verified MSI state.
use anyhow::{Result, ensure};
use chrono::Utc;
use rmm_domain::inventory_document::DeviceType;
use rmm_domain::inventory_document::{
CpuTimes, DeviceType, SOFTWARE_OBSERVATION_MAX_BYTES, firmware_identifier,
};
use serde_json::{Value, json};
use std::{
mem::{size_of, zeroed},
@@ -10,8 +12,12 @@ use std::{
time::{Duration, Instant},
};
use windows_sys::Win32::{
Foundation::{ERROR_FILE_NOT_FOUND, ERROR_NO_MORE_ITEMS, ERROR_SUCCESS},
System::{Registry::*, SystemInformation::*},
Foundation::{ERROR_FILE_NOT_FOUND, ERROR_NO_MORE_ITEMS, ERROR_SUCCESS, FILETIME},
System::{
Registry::*,
SystemInformation::*,
Threading::{GetActiveProcessorGroupCount, GetSystemTimes},
},
};
struct Registry(HKEY);
@@ -116,13 +122,77 @@ fn classify_machine(
(!manufacturer.is_empty() && !product.is_empty()).then_some(DeviceType::Physical)
}
fn machine_type() -> Option<DeviceType> {
fn machine_type(manufacturer: Option<&str>, product: Option<&str>) -> Option<DeviceType> {
let hyper_v_host = open(
HKEY_LOCAL_MACHINE,
r"SYSTEM\CurrentControlSet\Services\vmms",
KEY_WOW64_64KEY,
)
.is_ok();
classify_machine(
hyper_v_host,
non_microsoft_hypervisor(),
manufacturer,
product,
)
}
fn memory_status() -> Result<MEMORYSTATUSEX> {
let mut memory: MEMORYSTATUSEX = unsafe { zeroed() };
memory.dwLength = size_of::<MEMORYSTATUSEX>() as u32;
ensure!(
unsafe { GlobalMemoryStatusEx(&mut memory) } != 0
&& memory.ullTotalPhys > 0
&& memory.ullAvailPhys <= memory.ullTotalPhys
&& memory.dwMemoryLoad <= 100,
"NATIVE_HARDWARE_QUERY"
);
Ok(memory)
}
pub fn physical_memory() -> Option<(u64, u64)> {
let memory = memory_status().ok()?;
Some((memory.ullAvailPhys, memory.ullTotalPhys))
}
pub fn cpu_times() -> Option<CpuTimes> {
// GetSystemTimes on a multi-group machine only reports the caller's group.
if unsafe { GetActiveProcessorGroupCount() } != 1 {
return None;
}
let (mut idle, mut kernel, mut user) = (
FILETIME::default(),
FILETIME::default(),
FILETIME::default(),
);
if unsafe { GetSystemTimes(&mut idle, &mut kernel, &mut user) } == 0 {
return None;
}
let ticks =
|time: FILETIME| (u64::from(time.dwHighDateTime) << 32) | u64::from(time.dwLowDateTime);
// Windows includes idle time in the kernel counter.
Some(CpuTimes {
total: ticks(kernel).checked_add(ticks(user))?,
idle: ticks(idle),
})
}
fn hardware() -> Result<Value> {
let mut system: SYSTEM_INFO = unsafe { zeroed() };
unsafe {
GetNativeSystemInfo(&mut system);
}
ensure!(system.dwNumberOfProcessors > 0, "NATIVE_HARDWARE_QUERY");
let memory = memory_status()?;
let cpu_model = open(
HKEY_LOCAL_MACHINE,
r"HARDWARE\DESCRIPTION\System\CentralProcessor\0",
KEY_WOW64_64KEY,
)
.ok()
.and_then(|key| text(key.0, "ProcessorNameString").ok().flatten())
.map(|name| name.trim().to_owned())
.filter(|name| !name.is_empty());
let bios = open(
HKEY_LOCAL_MACHINE,
r"HARDWARE\DESCRIPTION\System\BIOS",
@@ -133,48 +203,24 @@ fn machine_type() -> Option<DeviceType> {
bios.as_ref()
.and_then(|key| text(key.0, name).ok().flatten())
};
classify_machine(
hyper_v_host,
non_microsoft_hypervisor(),
read("SystemManufacturer").as_deref(),
read("SystemProductName").as_deref(),
)
}
fn hardware() -> Result<Value> {
let mut system: SYSTEM_INFO = unsafe { zeroed() };
unsafe {
GetNativeSystemInfo(&mut system);
}
let mut memory: MEMORYSTATUSEX = unsafe { zeroed() };
memory.dwLength = size_of::<MEMORYSTATUSEX>() as u32;
ensure!(
unsafe { GlobalMemoryStatusEx(&mut memory) } != 0
&& system.dwNumberOfProcessors > 0
&& memory.ullTotalPhys > 0
&& memory.ullAvailPhys <= memory.ullTotalPhys
&& memory.dwMemoryLoad <= 100,
"NATIVE_HARDWARE_QUERY"
);
let cpu_model = open(
HKEY_LOCAL_MACHINE,
r"HARDWARE\DESCRIPTION\System\CentralProcessor\0",
KEY_WOW64_64KEY,
)
.ok()
.and_then(|key| text(key.0, "ProcessorNameString").ok().flatten())
.map(|name| name.trim().to_owned())
.filter(|name| !name.is_empty());
let manufacturer = read("SystemManufacturer").and_then(|value| firmware_identifier(&value));
let product = read("SystemProductName");
let bios_serial_number = read("SystemSerialNumber")
.and_then(|value| firmware_identifier(&value))
.or_else(crate::inventory_sources::bios_serial);
Ok(
json!({"cpu_model":cpu_model, "logical_processors_visible":system.dwNumberOfProcessors,
"memory_total_bytes":memory.ullTotalPhys,"memory_available_bytes":memory.ullAvailPhys,
"memory_load_percent":memory.dwMemoryLoad,"device_type":machine_type()}),
"memory_load_percent":memory.dwMemoryLoad,
"device_type":machine_type(manufacturer.as_deref(),product.as_deref()),
"manufacturer":manufacturer,"bios_serial_number":bios_serial_number}),
)
}
fn software() -> Result<Value> {
let began = Instant::now();
let mut entries = Vec::new();
let mut entry_bytes = 0;
let mut errors = 0u32;
let mut scanned = 0u32;
let mut truncated = false;
@@ -196,7 +242,7 @@ fn software() -> Result<Value> {
}
};
for index in 0..2048 {
if began.elapsed() > Duration::from_millis(250) || entries.len() >= 64 {
if began.elapsed() > Duration::from_millis(250) {
truncated = true;
break;
}
@@ -238,12 +284,12 @@ fn software() -> Result<Value> {
})();
match entry {
Ok(Some(entry)) => {
entries.push(entry);
if serde_json::to_vec(&entries)?.len() > 12288 {
entries.pop();
entry_bytes += serde_json::to_vec(&entry)?.len() + 1;
if entry_bytes > SOFTWARE_OBSERVATION_MAX_BYTES - 2048 {
truncated = true;
break;
}
entries.push(entry);
}
Ok(None) => {}
Err(_) => errors += 1,
@@ -274,7 +320,7 @@ pub fn observations() -> Vec<Value> {
[
(
"windows.hardware",
"windows.win32:GetNativeSystemInfo+GlobalMemoryStatusEx;registry:CentralProcessor,BIOS,vmms;cpuid",
"windows.win32:GetNativeSystemInfo+GlobalMemoryStatusEx+GetSystemFirmwareTable;registry:CentralProcessor,BIOS,vmms;cpuid",
hardware as Provider,
),
(
@@ -282,6 +328,8 @@ pub fn observations() -> Vec<Value> {
"windows.registry:HKLM.Uninstall:64+32",
software as Provider,
),
("windows.interfaces", "windows.win32:GetAdaptersAddresses", crate::inventory_sources::interfaces as Provider),
("windows.users", "windows.win32:WTSEnumerateSessions", crate::inventory_sources::users as Provider),
]
.into_iter()
.map(|(scope, source, collect)| {
@@ -0,0 +1,266 @@
//! Local read-only inventory providers. Each result is bounded independently.
use anyhow::{Context, Result, ensure};
use rmm_domain::inventory_document::firmware_identifier;
use serde_json::{Value, json};
use std::{
collections::BTreeSet,
ffi::c_void,
mem::size_of,
net::{Ipv4Addr, Ipv6Addr},
ptr, slice,
};
use windows_sys::Win32::{
Foundation::{ERROR_BUFFER_OVERFLOW, ERROR_NO_DATA, ERROR_SUCCESS},
NetworkManagement::IpHelper::*,
Networking::WinSock::{
AF_INET, AF_INET6, AF_UNSPEC, SOCKADDR_IN, SOCKADDR_IN6, SOCKET_ADDRESS,
},
System::{
RemoteDesktop::*,
SystemInformation::{GetSystemFirmwareTable, RSMB},
},
};
pub(crate) fn bios_serial() -> Option<String> {
let length = unsafe { GetSystemFirmwareTable(RSMB, 0, ptr::null_mut(), 0) };
if !(8..=1024 * 1024).contains(&length) {
return None;
}
let mut data = vec![0; length as usize];
if unsafe { GetSystemFirmwareTable(RSMB, 0, data.as_mut_ptr(), length) } != length {
return None;
}
let count = u32::from_le_bytes(data[4..8].try_into().ok()?) as usize;
smbios_serial(data.get(8..)?.get(..count)?)
}
fn smbios_serial(mut table: &[u8]) -> Option<String> {
while let Some(header) = table.get(..4) {
let length = usize::from(header[1]);
if length < 4 || header[0] == 127 {
break;
}
let strings = table.get(length..)?;
let end = strings.windows(2).position(|pair| pair == [0, 0])?;
if header[0] == 1 {
let offset = usize::from(*table.get(..length)?.get(7)?).checked_sub(1)?;
let serial = strings[..end].split(|&c| c == 0).nth(offset)?;
return firmware_identifier(std::str::from_utf8(serial).ok()?);
}
table = strings.get(end + 2..)?;
}
None
}
// Pointer fields returned by IP Helper refer into its caller-provided buffer.
fn at<T>(buffer: &[u64], pointer: *const T) -> Option<&T> {
let offset = (pointer as usize).checked_sub(buffer.as_ptr() as usize)?;
if offset.checked_add(size_of::<T>())? > std::mem::size_of_val(buffer) || !pointer.is_aligned()
{
return None;
}
Some(unsafe { &*pointer })
}
fn wide_name(buffer: &[u64], pointer: *const u16) -> Option<String> {
let mut text = Vec::new();
for i in 0..256 {
let character = *at(buffer, pointer.wrapping_add(i))?;
if character == 0 {
return (!text.is_empty()).then(|| String::from_utf16_lossy(&text));
}
text.push(character);
}
None
}
fn ip_address(buffer: &[u64], address: SOCKET_ADDRESS) -> Option<String> {
let sockaddr = at(buffer, address.lpSockaddr)?;
match sockaddr.sa_family {
AF_INET if address.iSockaddrLength >= size_of::<SOCKADDR_IN>() as i32 => {
let ipv4 = at(buffer, address.lpSockaddr.cast::<SOCKADDR_IN>())?;
Some(Ipv4Addr::from(unsafe { ipv4.sin_addr.S_un.S_addr }.to_ne_bytes()).to_string())
}
AF_INET6 if address.iSockaddrLength >= size_of::<SOCKADDR_IN6>() as i32 => {
let ipv6 = at(buffer, address.lpSockaddr.cast::<SOCKADDR_IN6>())?;
let ip = Ipv6Addr::from(unsafe { ipv6.sin6_addr.u.Byte });
let scope = unsafe { ipv6.Anonymous.sin6_scope_id };
Some(if scope == 0 {
ip.to_string()
} else {
format!("{ip}%{scope}")
})
}
_ => None,
}
}
pub(crate) fn interfaces() -> Result<Value> {
let mut bytes = 15360_u32;
let mut buffer = vec![0_u64; (bytes as usize).div_ceil(8)];
let flags = GAA_FLAG_SKIP_ANYCAST
| GAA_FLAG_SKIP_MULTICAST
| GAA_FLAG_SKIP_DNS_SERVER
| GAA_FLAG_INCLUDE_ALL_INTERFACES;
let mut status = unsafe {
GetAdaptersAddresses(
u32::from(AF_UNSPEC),
flags,
ptr::null(),
buffer.as_mut_ptr().cast(),
&mut bytes,
)
};
if status == ERROR_BUFFER_OVERFLOW && bytes <= 1024 * 1024 {
buffer.resize((bytes as usize).div_ceil(8), 0);
status = unsafe {
GetAdaptersAddresses(
u32::from(AF_UNSPEC),
flags,
ptr::null(),
buffer.as_mut_ptr().cast(),
&mut bytes,
)
};
}
if status == ERROR_NO_DATA {
return Ok(json!({"entries":[],"truncated":false}));
}
ensure!(status == ERROR_SUCCESS, "ADAPTER_ADDRESSES");
let mut current = buffer.as_ptr().cast::<IP_ADAPTER_ADDRESSES_LH>();
let mut entries = Vec::new();
let mut truncated = false;
let mut size = 0;
while !current.is_null() && entries.len() < 128 {
let adapter = at(&buffer, current).context("ADAPTER_POINTER")?;
let mut next = adapter.FirstUnicastAddress.cast_const();
let mut addresses = BTreeSet::new();
for _ in 0..64 {
if next.is_null() {
break;
}
let address = at(&buffer, next).context("ADAPTER_ADDRESS_POINTER")?;
if let Some(ip) = ip_address(&buffer, address.Address) {
addresses.insert(ip);
}
next = address.Next.cast_const();
}
truncated |= !next.is_null();
let name = wide_name(&buffer, adapter.FriendlyName)
.or_else(|| wide_name(&buffer, adapter.Description))
.unwrap_or_else(|| {
format!("Adapter {}", unsafe {
adapter.Anonymous1.Anonymous.IfIndex
})
});
let mac = adapter
.PhysicalAddress
.get(..adapter.PhysicalAddressLength as usize)
.filter(|bytes| !bytes.is_empty())
.map(|bytes| {
bytes
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<Vec<_>>()
.join(":")
});
let entry = json!({"name":name,"addresses":addresses,"mac":mac});
size += serde_json::to_vec(&entry)?.len() + 1;
if size > 30 * 1024 {
truncated = true;
break;
}
entries.push(entry);
current = adapter.Next.cast_const();
}
Ok(json!({"entries":entries,"truncated":truncated || !current.is_null()}))
}
struct WtsMemory(*mut c_void);
impl Drop for WtsMemory {
fn drop(&mut self) {
if !self.0.is_null() {
unsafe { WTSFreeMemory(self.0) };
}
}
}
fn session_text(id: u32, class: WTS_INFO_CLASS) -> Option<String> {
let mut text = ptr::null_mut();
let mut bytes = 0;
if unsafe { WTSQuerySessionInformationW(ptr::null_mut(), id, class, &mut text, &mut bytes) }
== 0
{
return None;
}
let _guard = WtsMemory(text.cast());
if text.is_null() || !(2..=512).contains(&bytes) || bytes % 2 != 0 {
return None;
}
let units = unsafe { slice::from_raw_parts(text, (bytes / 2) as usize) };
String::from_utf16(units.split(|c| *c == 0).next()?).ok()
}
pub(crate) fn users() -> Result<Value> {
let mut list = ptr::null_mut();
let mut count = 0;
ensure!(
unsafe { WTSEnumerateSessionsW(ptr::null_mut(), 0, 1, &mut list, &mut count) } != 0,
"SESSION_QUERY"
);
let _guard = WtsMemory(list.cast());
let mut entries = BTreeSet::new();
let mut errors = 0;
let mut truncated = count > 256;
if count != 0 {
ensure!(!list.is_null(), "SESSION_LIST");
for session in unsafe { slice::from_raw_parts(list, count.min(256) as usize) } {
if session.SessionId == 0
|| ![WTSActive, WTSConnected, WTSDisconnected].contains(&session.State)
{
continue;
}
match session_text(session.SessionId, WTSUserName) {
Some(name) if !name.is_empty() => {
let account = match session_text(session.SessionId, WTSDomainName) {
Some(domain) if !domain.is_empty() => format!("{domain}\\{name}"),
_ => name,
};
if entries.len() >= 64 {
truncated = true;
break;
}
if account.len() <= 255 && !account.chars().any(char::is_control) {
entries.insert(account);
} else {
errors += 1;
}
}
Some(_) => {}
None => errors += 1,
}
}
}
ensure!(
errors == 0 || !entries.is_empty(),
"SESSION_INFORMATION_UNAVAILABLE"
);
Ok(json!({"entries":entries,"read_errors":errors,"truncated":truncated}))
}
#[cfg(test)]
mod tests {
use super::smbios_serial;
#[test]
fn reads_the_system_serial_and_rejects_incomplete_entries() {
assert_eq!(
smbios_serial(b"\x01\x08\x00\x00\x01\x00\x00\x02Vendor\0ABC-123\0\0").as_deref(),
Some("ABC-123")
);
assert_eq!(
smbios_serial(b"\x01\x08\x00\x00\x01\x00\x00\x02Vendor\0ABC-123\0"),
None
);
}
}
@@ -53,6 +53,7 @@ pub fn clock_code(error: &anyhow::Error) -> &'static str {
"TIME_RUNTIME_NO_CONNECTION",
"TIME_LOCAL_WALL_OUTSIDE_BOUND",
"TIME_SOURCE_AGE_AT_CHECK",
"TIME_WINDOWS_LOCAL_CLOCK",
];
error
.chain()
+1
View File
@@ -24,6 +24,7 @@ pub mod helper_system;
pub mod windows_update_setup;
#[cfg(windows)]
pub mod inventory;
mod inventory_sources;
#[cfg(windows)]
pub mod job_binding;
pub mod journal;
+140 -36
View File
@@ -1,6 +1,6 @@
//! Read-only Windows APIs. The kernel's recorded startup time identifies the boot.
use anyhow::{Context, Result, ensure};
use chrono::Utc;
use chrono::{DateTime, Utc};
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
use std::{ffi::c_void, os::windows::ffi::OsStrExt, path::Path, ptr, time::Duration};
@@ -227,28 +227,51 @@ pub fn private_path(path: &Path, directory: bool) -> Result<()> {
}
fn registry_string(name: &str) -> Result<String> {
let key = wide("SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion");
let name = wide(name);
let value_name = wide(name);
let mut bytes = vec![0u16; 2048];
let mut size = (bytes.len() * 2) as u32;
let status = unsafe {
RegGetValueW(
HKEY_LOCAL_MACHINE,
key.as_ptr(),
value_name.as_ptr(),
RRF_RT_REG_SZ,
ptr::null_mut(),
bytes.as_mut_ptr().cast(),
&mut size,
)
};
ensure!(status == 0, "OS_REGISTRY {name} (Win32 {status})");
ensure!(
unsafe {
RegGetValueW(
HKEY_LOCAL_MACHINE,
key.as_ptr(),
name.as_ptr(),
RRF_RT_REG_SZ,
ptr::null_mut(),
bytes.as_mut_ptr().cast(),
&mut size,
)
} == 0,
"OS_REGISTRY"
size >= 2 && size % 2 == 0 && size as usize <= bytes.len() * 2,
"OS_REGISTRY_SIZE {name}"
);
ensure!(
size >= 2 && size as usize <= bytes.len() * 2,
"OS_REGISTRY_SIZE"
);
Ok(String::from_utf16(&bytes[..size as usize / 2 - 1])?)
String::from_utf16(&bytes[..size as usize / 2 - 1])
.with_context(|| format!("OS_REGISTRY_ENCODING {name}"))
}
fn os_inventory(read: impl Fn(&str) -> Result<String>) -> Result<(String, Value)> {
let product = read("ProductName")?;
let build = read("CurrentBuildNumber")?;
// Older Windows releases can omit DisplayVersion. Both version labels are
// optional; the product and build still identify the OS without either.
let version = |name| read(name).ok().filter(|s| !s.trim().is_empty());
let display = version("DisplayVersion");
let release = if display.is_none() {
version("ReleaseId")
} else {
None
};
let os = match (&display, &release) {
(Some(display), _) => format!("{product}; DisplayVersion={display}; Build={build}"),
(None, Some(release)) => format!("{product}; ReleaseId={release}; Build={build}"),
(None, None) => format!("{product}; Build={build}"),
};
let mut data = json!({"product_name":product,"display_version":display,"build":build});
if let Some(release) = release {
data["release_id"] = json!(release);
}
Ok((os, data))
}
pub(crate) fn computer_name(format: COMPUTER_NAME_FORMAT) -> Result<String> {
let mut name = [0u16; 256];
@@ -261,28 +284,61 @@ pub(crate) fn computer_name(format: COMPUTER_NAME_FORMAT) -> Result<String> {
Ok(String::from_utf16(&name[..len as usize])?)
}
fn install_date() -> Option<DateTime<Utc>> {
let key = wide(r"SOFTWARE\Microsoft\Windows NT\CurrentVersion");
let name = wide("InstallDate");
let mut seconds = 0_u32;
let mut bytes = size_of::<u32>() as u32;
let status = unsafe { RegGetValueW(HKEY_LOCAL_MACHINE,key.as_ptr(),name.as_ptr(),RRF_RT_REG_DWORD,
ptr::null_mut(),ptr::from_mut(&mut seconds).cast(),&mut bytes) };
(status == 0 && bytes as usize == size_of::<u32>() && seconds > 0)
.then(|| DateTime::from_timestamp(i64::from(seconds), 0))
.flatten().filter(|time| *time <= Utc::now())
}
pub fn snapshot() -> Result<Value> {
snapshot_with_boot(None)
}
fn disk_space(root: &[u16]) -> Option<(u64, u64)> {
let (mut available, mut total) = (0, 0);
(unsafe { GetDiskFreeSpaceExW(root.as_ptr(), &mut available, &mut total, ptr::null_mut()) } != 0
&& total > 0 && available <= total).then_some((available, total))
}
fn fixed_volumes(system: &str, available: u64, total: u64) -> Vec<Value> {
use windows_sys::Win32::Storage::FileSystem::{GetDriveTypeW, GetLogicalDrives};
const DRIVE_FIXED: u32 = 3;
let mask = unsafe { GetLogicalDrives() };
let mut rows = vec![json!({"name":system,"free_bytes":available,"total_bytes":total})];
for letter in b'A'..=b'Z' {
if mask & (1 << (letter - b'A')) == 0 { continue; }
let name = format!("{}:", char::from(letter));
if name.eq_ignore_ascii_case(system) { continue; }
let root = wide(format!("{name}\\"));
if unsafe { GetDriveTypeW(root.as_ptr()) } != DRIVE_FIXED { continue; }
// Retain an identified but unreadable volume without claiming zero usage.
let sizes = disk_space(&root);
rows.push(json!({"name":name,"free_bytes":sizes.map(|s| s.0),"total_bytes":sizes.map(|s| s.1)}));
}
rows
}
pub fn snapshot_with_boot(booted_at: Option<DateTime<Utc>>) -> Result<Value> {
let started = Utc::now();
let hostname = computer_name(ComputerNameDnsHostname)?;
let volume = wide(std::env::var("SystemDrive").context("SYSTEM_DRIVE")? + "\\");
let mut available = 0;
let mut total = 0;
let mut free = 0;
ensure!(
unsafe { GetDiskFreeSpaceExW(volume.as_ptr(), &mut available, &mut total, &mut free) } != 0,
"VOLUME_QUERY"
);
ensure!(total > 0 && available <= total, "VOLUME_VALUES");
let product = registry_string("ProductName")?;
let display = registry_string("DisplayVersion")?;
let build = registry_string("CurrentBuildNumber")?;
let os = format!("{product}; DisplayVersion={display}; Build={build}");
let system = std::env::var("SystemDrive").context("SYSTEM_DRIVE")?.to_ascii_uppercase();
let (available, total) = disk_space(&wide(format!("{system}\\"))).context("VOLUME_QUERY")?;
let volumes = fixed_volumes(&system, available, total);
let (os, mut os_data) = os_inventory(registry_string)?;
os_data["installed_at"] = json!(install_date());
os_data["booted_at"] = json!(booted_at);
let completed = Utc::now();
let mut snapshot = json!({"os":os,"hostname":hostname,"architecture":std::env::consts::ARCH,"root_total_bytes":total,"root_available_bytes":available,"root_available_inodes":null,
"collection_started_at":started,"collection_completed_at":completed,
"observations":[
{"scope":"windows.os","source_id":"windows.registry:CurrentVersion;GetComputerNameExW","observed_at":completed,"quality":"valid","completeness":"partial","truncated":false,"error_code":null,"data":{"product_name":product,"display_version":display,"build":build}},
{"scope":"windows.volumes","source_id":"windows.win32:GetDiskFreeSpaceExW","observed_at":completed,"quality":"valid","completeness":"partial","truncated":false,"error_code":null,"data":{"system_volume_total_bytes":total,"system_volume_available_bytes":available}},
{"scope":"windows.os","source_id":"windows.registry:CurrentVersion;GetComputerNameExW;Kernel-General/12","observed_at":completed,"quality":"valid","completeness":"partial","truncated":false,"error_code":null,"data":os_data},
{"scope":"windows.volumes","source_id":"windows.win32:GetLogicalDrives;GetDriveTypeW;GetDiskFreeSpaceExW","observed_at":completed,"quality":"valid","completeness":"partial","truncated":false,"error_code":null,"data":{"system_volume_total_bytes":total,"system_volume_available_bytes":available,"drives":volumes}},
{"scope":"windows.inodes","source_id":"windows.win32","observed_at":completed,"quality":"unsupported","completeness":"unknown","truncated":false,"error_code":"NOT_APPLICABLE","data":null}
]});
snapshot["observations"]
@@ -292,7 +348,7 @@ pub fn snapshot() -> Result<Value> {
snapshot["collection_completed_at"] = json!(Utc::now());
Ok(snapshot)
}
pub async fn boot_id() -> Result<String> {
pub async fn boot_identity() -> Result<(String, DateTime<Utc>)> {
// Fixed passive query only. Never interpolates a job or caller-supplied code.
// Avoid Win32_Product: it can cause MSI consistency checks and repairs.
let executable = Path::new(&std::env::var("SystemRoot")?)
@@ -328,11 +384,59 @@ if ($nodes.Count -ne 1) { throw 'BOOT_START_TIME' }
.try_into()?;
bytes[6] = (bytes[6] & 15) | 0x80;
bytes[8] = (bytes[8] & 63) | 0x80;
Ok(uuid::Uuid::from_bytes(bytes).to_string())
let ticks = text.parse::<i64>()?.checked_sub(621_355_968_000_000_000).context("BOOT_EPOCH")?;
let time = DateTime::from_timestamp(ticks.div_euclid(10_000_000), ticks.rem_euclid(10_000_000) as u32 * 100)
.context("BOOT_EPOCH")?;
Ok((uuid::Uuid::from_bytes(bytes).to_string(),time))
}
pub async fn boot_id() -> Result<String> {
boot_identity().await.map(|(id, _)| id)
}
#[cfg(test)]
mod tests {
fn os_fixture(display: Option<&str>, release: Option<&str>) -> (String, serde_json::Value) {
super::os_inventory(|name| {
match name {
"ProductName" => Some("Windows 10 Enterprise LTSC"),
"CurrentBuildNumber" => Some("17763"),
"DisplayVersion" => display,
"ReleaseId" => release,
_ => panic!("unexpected registry value: {name}"),
}
.map(str::to_owned)
.ok_or_else(|| anyhow::anyhow!("missing {name}"))
})
.unwrap()
}
#[test]
fn os_inventory_prefers_display_version() {
let (os, data) = os_fixture(Some("22H2"), Some("2009"));
assert!(os.contains("DisplayVersion=22H2"));
assert_eq!(data["display_version"], "22H2");
assert!(data.get("release_id").is_none());
}
#[test]
fn os_inventory_uses_legacy_release_id_when_display_version_is_missing() {
let (os, data) = os_fixture(None, Some("1809"));
assert_eq!(os, "Windows 10 Enterprise LTSC; ReleaseId=1809; Build=17763");
assert!(data["display_version"].is_null());
assert_eq!(data["release_id"], "1809");
}
#[test]
fn os_inventory_keeps_product_and_build_without_version_labels() {
for display in [None, Some(" ")] {
let (os, data) = os_fixture(display, None);
assert_eq!(os, "Windows 10 Enterprise LTSC; Build=17763");
assert!(data["display_version"].is_null());
assert_eq!(data["build"], "17763");
}
}
#[test]
fn native_inventory_has_real_windows_sources_and_no_fake_inodes() {
let v = super::snapshot().unwrap();
+200 -33
View File
@@ -8,7 +8,7 @@ use rmm_agent_core::{
reconcile::{self, SyncClock},
};
use rmm_agent_windows::certificate_rotation::{ProtectedKeyFiles, RotationStore};
use rmm_agent_windows::{CAPABILITIES, job_binding::readonly_binding, journal::Journal, native};
use rmm_agent_windows::{CAPABILITIES, inventory, job_binding::readonly_binding, journal::Journal, native};
use rmm_domain::jobs::{Action, ExecutionPermitPayload};
use rmm_protocol::*;
use rusqlite::{Connection, OptionalExtension};
@@ -92,6 +92,11 @@ fn rotation_file_id(path: &Path) -> Result<[u32; 3]> {
])
}
impl Config {
fn supports_customer_transfer(&self) -> bool {
cfg!(feature = "system-helper") && self.enable_jobs
&& self.localservice_time_policy.is_some() && self.job_binding_file.is_none()
&& !self.windows_echo_probe_only && !self.windows_update_helper
}
fn allows_script_action(&self, action: &Action, identity: &str) -> bool {
let Action::RunScript { interpreter, windows_execution_profile, .. } = action else {
return false;
@@ -183,6 +188,16 @@ fn protected_read(path: &Path, max: usize) -> Result<Vec<u8>> {
);
Ok(fs::read(path)?)
}
fn transfer_csr(previous: Option<&[u8]>) -> Result<(Vec<u8>, String)> {
let key = match previous {
Some(raw) => KeyPair::from_pem(std::str::from_utf8(raw)?)?,
None => KeyPair::generate()?,
};
let mut parameters = CertificateParams::default();
parameters.distinguished_name.push(DnType::CommonName, "customer-transfer");
let csr = parameters.serialize_request(&key)?.pem()?;
Ok((key.serialize_pem().into_bytes(), csr))
}
async fn enrollment(config: &Config, ca: &[u8]) -> Result<EnrollmentResponse> {
let state = &config.state_directory;
let identity = state.join("identity.json");
@@ -625,12 +640,14 @@ struct Agent {
config: Config,
reported_version: String,
release_job_guard: Option<fs::File>,
transfer_gate: Option<fs::File>,
identity: EnrollmentResponse,
ca: Vec<u8>,
journal: Journal,
certificates: RotationStore,
key_files: ProtectedKeyFiles,
boot: String,
booted_at: chrono::DateTime<Utc>,
instance: String,
started: Instant,
binding: Option<Binding>,
@@ -822,6 +839,10 @@ impl Agent {
) -> Result<()> {
ensure!(self.time_refresh.is_none(), "TIME_REFRESH_ALREADY_RUNNING");
use rmm_agent_windows::signed_clock::{CONFIG_PATH, SamplingMode};
let clock = self.clock.native.source();
if clock.uses_local_clock() {
return Ok(());
}
// Presence opts into sampling only. Each role opens and verifies the
// protected profile and signed response itself before accepting time.
let Some(mode) = SamplingMode::configured(self.config.enable_jobs, Path::new(CONFIG_PATH))?
@@ -843,7 +864,6 @@ impl Agent {
self.binding.as_ref(),
self.helper_lane.clone(),
)? };
let clock = self.clock.native.source();
let stop = Arc::new(AtomicBool::new(false));
let mut tasks = Vec::new();
let own_clock = clock.clone();
@@ -934,7 +954,7 @@ impl Agent {
let _ = time_helper.time_close(generation).await;
}
helper_report.record(&helper_state, "system_helper", &result);
tokio::time::sleep(mode.interval(result.is_ok())).await;
tokio::time::sleep(mode.helper_interval(result.is_ok())).await;
}
let _ = time_helper.time_close(generation).await;
}));
@@ -1274,8 +1294,8 @@ impl Agent {
&& !self.helper_apply_unconfirmed
&& !self.prepared
&& self.running.is_none()
&& !self.journal.connection().query_row("SELECT EXISTS(SELECT 1 FROM job_attempts WHERE state IN ('requesting','running','verifying'))",[],|r|r.get::<_,bool>(0))?
&& self.helper_probe_at.elapsed() >= Duration::from_secs(30)
&& !self.journal.connection().query_row("SELECT EXISTS(SELECT 1 FROM job_attempts WHERE state IN ('requesting','running','verifying'))",[],|r|r.get::<_,bool>(0))?
{
let agent = self.identity.agent_id.clone();
let asset = self.identity.asset_id.clone();
@@ -1560,12 +1580,54 @@ impl Agent {
}
Ok(false)
}
async fn stage_customer_transfer(&mut self) -> Result<bool> {
if !self.config.supports_customer_transfer() || self.running.is_some()
|| self.prepared || self.deferred_sync.is_some() || self.deferred_permit.is_some()
|| self.release_job_guard.is_some() {
return Ok(false);
}
let (Some(binding), Some(ledger)) = (&self.binding, &self.ledger) else {
return Ok(false);
};
if !rmm_agent_core::customer_transfer::idle(self.journal.connection(), ledger)? {
return Ok(false);
}
#[cfg(feature = "system-helper")]
if self.helper_control_busy_for_sync() || self.update_health.is_some() {
return Ok(false);
}
// Keep the release service from replacing the program during the handoff.
let binding = binding.clone();
if !self.reserve_release_gate()? {
return Ok(false);
}
let result = async {
let identity = self.certificates.current(
self.journal.connection(), &self.key_files, Utc::now().timestamp(),
)?;
let client = crate::https_channel::mtls_client(
self.config.connect_address, &self.ca, &identity.certificate_pem, &identity.private_key_pem,
)?;
let base = format!("https://{}:{}", self.config.server_name, self.config.connect_address.port());
rmm_agent_core::customer_transfer::stage(
&client, &base, &self.config.state_directory, &binding, transfer_csr,
).await
}.await;
let gate = self.release_job_guard.take();
if matches!(&result, Ok(true)) {
self.transfer_gate = gate;
}
result
}
fn capabilities(&self) -> Vec<String> {
let mut caps: Vec<String> = CAPABILITIES.iter().map(|c| c.to_string()).collect();
#[cfg(target_env = "gnu")]
caps.push("agent.release.windows.gnu/1".into());
// queue_update_observation emits the passive update status in every profile.
caps.push("agent.update/1".into());
if self.config.supports_customer_transfer() {
caps.push(rmm_domain::customer_transfer::CAPABILITY.into());
}
if self.config.windows_echo_probe_only {
caps.push("windows.echo-probe/1".into());
}
@@ -2420,8 +2482,8 @@ impl Agent {
if self.journal.count()? > 0 || !jobs::events(self.journal.connection())?.is_empty() {
return Ok(false);
}
let data = serde_json::to_vec(&native::snapshot()?)?;
ensure!(data.len() <= 32768, "WINDOWS_INVENTORY_LIMIT");
let data = serde_json::to_vec(&native::snapshot_with_boot(Some(self.booted_at))?)?;
ensure!(data.len() <= 256 * 1024, "WINDOWS_INVENTORY_LIMIT");
let envelope = self.envelope(
generation,
Payload::InventorySnapshot(InventorySnapshot {
@@ -2537,7 +2599,7 @@ impl Agent {
};
self.renew_certificate(admission.as_deref()).await?;
}
self.boot = native::boot_id().await?;
(self.boot, self.booted_at) = native::boot_identity().await?;
let advertised = self.capabilities();
let hello = self.envelope(
0,
@@ -2602,9 +2664,11 @@ impl Agent {
let mut rotation_at = Instant::now();
let mut rotation_after = Duration::from_secs(30);
let mut heartbeat = Instant::now() - Duration::from_secs(60);
let mut previous_cpu = None;
let mut heartbeat_after = Duration::from_secs(30);
let mut tick = tokio::time::interval(Duration::from_millis(20));
tick.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
let active_tick = Duration::from_millis(20);
let idle_tick = Duration::from_millis(250);
let mut next_tick = tokio::time::Instant::now();
let mut readiness_at = Instant::now() - Duration::from_secs(5);
#[cfg(feature = "system-helper")]
let mut update_control_at = Instant::now() - Duration::from_secs(2);
@@ -2615,9 +2679,19 @@ impl Agent {
running.cancel.store(true, Ordering::SeqCst);
}
}
let control_pending = self.running.is_some()
|| self.prepared
|| self.deferred_sync.is_some()
|| self.deferred_permit.is_some()
|| self.helper_control_pending();
#[cfg(feature = "system-helper")]
let control_pending = control_pending || self.helper_probe.is_some();
let cadence = if control_pending { active_tick } else { idle_tick };
next_tick = next_tick.min(tokio::time::Instant::now() + cadence);
tokio::select! {
Some(completion)=self.completed_rx.recv()=>{self.completed(generation,completion)?;needs_sync=true;},
_=tick.tick()=>{
_=tokio::time::sleep_until(next_tick)=>{
next_tick = tokio::time::Instant::now() + cadence;
if !stop.load(Ordering::SeqCst) && self.poll_sync_apply(generation,true).is_err(){self.close_gate();needs_sync=true;}
#[cfg(feature = "system-helper")]
if !stop.load(Ordering::SeqCst) && self.poll_helper(&channel,generation).await.is_err() {self.close_gate();needs_sync=self.sync.is_some();}
@@ -2649,7 +2723,9 @@ impl Agent {
if self.sync.is_some() && !self.prepared && !self.helper_control_busy_for_sync() && rmm_agent_windows::job_readiness::refresh_due(needs_sync,sync_at.elapsed(),self.sync.as_ref().and_then(|s|s.validity_deadline()),self.clock.reading().ok().map(|r|r.upper)) {
if self.start_sync(&channel,generation).await.is_err(){self.close_gate();needs_sync=true;}else{needs_sync=false;}sync_at=Instant::now();
}
if Instant::now() >= permit_after && !needs_sync && !self.prepared && self.ready(generation) && jobs::events(self.journal.connection())?.is_empty() && self.journal.count()?==0 {
if Instant::now() >= permit_after && !needs_sync && !self.prepared && self.config.enable_jobs
&& self.journal.connection().prepare_cached("SELECT EXISTS(SELECT 1 FROM job_attempts WHERE state='accepted' AND cancel_requested=0)")?.query_row([], |row| row.get::<_, bool>(0))?
&& self.ready(generation) && jobs::events(self.journal.connection())?.is_empty() && self.journal.count()?==0 {
// A transiently stale clock skips this tick; it must not end the session.
if let Ok(now)=self.clock.now() && let Some(offer)=jobs::next_at(self.journal.connection(),now)? {
let job=self.binding.as_ref().context("JOB_BINDING_REQUIRED")?.job(&offer.payload_bytes)?;
@@ -2744,6 +2820,11 @@ impl Agent {
if rotation_at.elapsed() >= rotation_after {
rotation_at = Instant::now();
rotation_after = Duration::from_secs(30);
match self.stage_customer_transfer().await {
Ok(true) => return Ok(()),
Ok(false) => (),
Err(_) => eprintln!("CUSTOMER_TRANSFER_PENDING"),
}
let now = Utc::now().timestamp();
if self
.certificates
@@ -2776,9 +2857,19 @@ impl Agent {
// inventory offline. Retry setup at the next heartbeat.
eprintln!("PASSIVE_SIGNED_TIME_UNAVAILABLE");
}
let cpu = inventory::cpu_times();
let cpu_usage_basis_points = cpu.zip(previous_cpu)
.and_then(|(current, previous)| current.usage_since(previous));
previous_cpu = cpu;
let memory = inventory::physical_memory();
let payload = Payload::Heartbeat(Heartbeat {
instance_id: self.instance.clone(),
uptime_seconds: self.started.elapsed().as_secs(),
system_usage: Some(SystemUsage {
cpu_usage_basis_points,
ram_available_bytes: memory.map(|(available, _)| available),
ram_total_bytes: memory.map(|(_, total)| total),
}),
});
let h = self.envelope(generation, payload)?;
channel.send(&h).await?;
@@ -2788,6 +2879,7 @@ impl Agent {
Payload::Heartbeat(Heartbeat {
instance_id: self.instance.clone(),
uptime_seconds: self.started.elapsed().as_secs(),
system_usage: None,
}),
)?;
channel.poll(poll).await?;
@@ -2807,7 +2899,7 @@ impl Agent {
self.replay(&channel, generation).await?;
replay = Instant::now();
}
if !stop.load(Ordering::SeqCst) && inventory.elapsed() >= Duration::from_secs(86400) {
if !stop.load(Ordering::SeqCst) && inventory.elapsed() >= Duration::from_secs(300) {
self.queue_inventory(generation)?;
inventory = Instant::now();
}
@@ -2892,7 +2984,23 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
);
let path = Path::new(&arguments[2]);
ensure!(path.is_absolute(), "ABSOLUTE_CONFIG_REQUIRED");
let config = Config::read(path)?;
let runtime = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.worker_threads(2)
.build()?;
runtime.block_on(async {
let mut gate = None;
loop {
match run_service_binding(path, stop, gate.take()).await? {
Some(held) => gate = Some(held),
None => return Ok(()),
}
}
})
}
async fn run_service_binding(path: &Path, stop: &'static AtomicBool, release_guard: Option<fs::File>) -> Result<Option<fs::File>> {
let mut config = Config::read(path)?;
native::require_local_service()?;
ensure!(
config.state_directory.is_absolute(),
@@ -2908,21 +3016,50 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
let child = child?;
native::private_path(&child.path(), child.file_type()?.is_dir())?;
}
let journal = Journal::open(&config.state_directory)?;
let runtime = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.worker_threads(2)
.build()?;
runtime.block_on(async move {
let identity = enrollment(&config, &ca).await?;
let mut journal=journal;
if rmm_agent_core::customer_transfer::pending(&config.state_directory)? {
#[cfg(feature = "system-helper")]
{
ensure!(config.supports_customer_transfer(), "TRANSFER_CONFIG");
let policy = config.localservice_time_policy.as_deref().context("TRANSFER_POLICY")?;
let signer = rmm_agent_windows::signed_clock::enrolled_transfer_signer(policy)?;
loop {
if stop.load(Ordering::SeqCst) {
return Ok(None);
}
match rmm_agent_core::customer_transfer::activate(&config.state_directory, &signer) {
Ok(()) => break,
Err(error) if error.downcast_ref::<std::io::Error>()
.is_some_and(|e| matches!(e.raw_os_error(), Some(32 | 33))) => {
// Another service can briefly hold a read-only handle.
tokio::time::sleep(Duration::from_secs(2)).await;
}
Err(error) => return Err(error),
}
}
}
#[cfg(not(feature = "system-helper"))]
anyhow::bail!("TRANSFER_CONFIG");
}
drop(release_guard);
let mut journal = Journal::open(&config.state_directory)?;
let identity = enrollment(&config, &ca).await?;
journal.pin_identity(&identity.agent_id)?;
let boot = native::boot_id().await?;
let (boot, booted_at) = native::boot_identity().await?;
let binding=config.job_binding_file.as_deref().map(readonly_binding).transpose()?;
#[cfg(feature = "system-helper")]
let binding = if let Some(policy) = &config.localservice_time_policy {
Some(rmm_agent_windows::signed_clock::enrolled_binding(policy, &identity)?)
} else { binding };
let (binding, installed_tenant) = if let Some(policy) = &config.localservice_time_policy {
let mut enrolled = rmm_agent_windows::signed_clock::enrolled_binding(policy, &identity)?;
let installed = enrolled.tenant_id;
if config.supports_customer_transfer()
&& let Some(target) = rmm_agent_core::customer_transfer::assigned_tenant(
&config.state_directory, &identity, &enrolled.public_key_hex,
)? {
enrolled.tenant_id = target;
// Elevated helpers are bound to the original installation.
config.managed_script_helper = false;
}
(Some(enrolled), Some(installed))
} else { (binding, None) };
if let Some(binding)=&binding {
ensure!(binding.agent_id.to_string()==identity.agent_id && binding.asset_id.to_string()==identity.asset_id,"ENROLLMENT_JOB_BINDING_MISMATCH");
native::private_path(&config.state_directory.join("jobs"),true)?;
@@ -2938,8 +3075,17 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
let clock=JobClock::new(config.enable_jobs);
#[cfg(feature = "system-helper")]
let clock = if let Some(policy) = &config.localservice_time_policy {
let binding = binding.clone().context("JOB_BINDING_REQUIRED")?;
JobClock { enabled: true, native: rmm_agent_windows::signed_clock::RuntimeClock::enrolled_local_service(binding,policy.clone()).for_generation(0) }
use rmm_agent_windows::signed_clock::RuntimeClock;
let source = if config.windows_update_helper {
RuntimeClock::enrolled_local_service(
binding.clone().context("JOB_BINDING_REQUIRED")?,
policy.clone(),
installed_tenant.context("INSTALLED_TENANT_REQUIRED")?,
)
} else {
RuntimeClock::local(rmm_time_attestation::Role::Agent)
};
JobClock { enabled: true, native: source.for_generation(0) }
} else { clock };
let mut config=config;
if config.enable_jobs {
@@ -2955,18 +3101,32 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
ensure!(rmm_agent_windows::echo_probe::protected_scope()? == *scope, "WINDOWS_ECHO_CONFIGURATION_CHANGED");
}
}
let sync=if config.enable_jobs && (!config.windows_echo_probe_only || config.echo_scope.is_some()) {binding.clone().map(|binding|reconcile::ClientSync::new(binding).with_helper_identity(reconcile::HelperIdentity::System))} else {None};
let sync = if config.enable_jobs && (!config.windows_echo_probe_only || config.echo_scope.is_some()) {
binding.clone().map(|binding| {
let sync = reconcile::ClientSync::new(binding)
.with_helper_identity(reconcile::HelperIdentity::System);
if config.localservice_time_policy.is_some() {
sync.with_periodic_idle_integrity_checks()
} else {
sync
}
})
} else {
None
};
let (completed_tx,completed_rx)=tokio::sync::mpsc::channel(1);
let mut agent = Agent {
config,
reported_version: installed_version()?,
release_job_guard: None,
transfer_gate: None,
identity,
ca,
journal,
certificates,
key_files,
boot,
booted_at,
instance: Uuid::new_v4().to_string(),
started: Instant::now(), binding, ledger, _ledger_file:ledger_file, clock, sync,
#[cfg(feature="system-helper")]
@@ -3024,6 +3184,12 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
if let Err(error) = agent.session(stop).await {
record_session_error(&agent.config.state_directory, &error);
}
if let Some(guard) = agent.transfer_gate.take() {
if let Some(worker) = agent.readiness_worker.take() {
let _ = worker.await;
}
return Ok(Some(guard));
}
if start.elapsed() > Duration::from_secs(60) {
backoff = 1;
}
@@ -3042,8 +3208,7 @@ fn service_run(stop: &'static AtomicBool) -> Result<()> {
tokio::time::sleep(Duration::from_millis(250)).await;
}
}
Ok(())
})
Ok(None)
}
pub fn main() -> Result<()> {
let args: Vec<String> = std::env::args().skip(1).collect();
@@ -3063,11 +3228,13 @@ pub fn main() -> Result<()> {
}
if args == ["--native-probe"] {
let runtime = tokio::runtime::Runtime::new()?;
let boot = runtime.block_on(native::boot_id())?;
let inventory = native::snapshot()?;
let (boot, booted_at) = runtime.block_on(native::boot_identity())?;
let inventory = native::snapshot_with_boot(Some(booted_at))?;
let drives = inventory["observations"].as_array().and_then(|observations|
observations.iter().find(|o| o["scope"] == "windows.volumes")).map(|o| &o["data"]["drives"]);
println!(
"{}",
serde_json::json!({"platform":"windows","architecture":std::env::consts::ARCH,"hostname_available":!inventory["hostname"].as_str().unwrap_or("").is_empty(),"system_volume_available":inventory["root_available_bytes"].as_u64().is_some(),"inodes":null,"boot_id":boot,"capabilities":CAPABILITIES,"jobs_supported":false})
serde_json::json!({"platform":"windows","architecture":std::env::consts::ARCH,"hostname_available":!inventory["hostname"].as_str().unwrap_or("").is_empty(),"system_volume_available":inventory["root_available_bytes"].as_u64().is_some(),"fixed_volumes":drives,"inodes":null,"boot_id":boot,"capabilities":CAPABILITIES,"jobs_supported":false})
);
return Ok(());
}
+68 -11
View File
@@ -31,6 +31,15 @@ impl SamplingMode {
(Self::Jobs, false) => Duration::from_millis(500),
}
}
pub fn helper_interval(self, verified: bool) -> std::time::Duration {
match (self, verified) {
// Each refresh verifies both service images over the pipe. A held
// signed time sample remains usable for up to five minutes.
(Self::Jobs, true) => std::time::Duration::from_secs(5),
_ => self.interval(verified),
}
}
}
fn independent_time_key(time_key_hex: &str, job_key_hex: &str) -> Result<[u8; 32]> {
@@ -802,10 +811,7 @@ mod installed {
tenant_id: Uuid,
job_signing_public_key_hex: String,
}
pub fn enrolled_binding(
policy: &Path,
registration: &rmm_protocol::EnrollmentResponse,
) -> Result<rmm_agent_core::auth::Binding> {
fn installed_policy(policy: &Path) -> Result<LocalPolicy> {
crate::job_binding::check_admin_readonly(policy)?;
ensure!(
std::fs::metadata(policy)?.len() <= MAX_CONFIG_BYTES as u64,
@@ -816,6 +822,16 @@ mod installed {
p.schema_version == 1 && !p.tenant_id.is_nil() && digest(&p.job_signing_public_key_hex),
"INSTALLED_JOB_TRUST"
);
Ok(p)
}
pub fn enrolled_transfer_signer(policy: &Path) -> Result<String> {
Ok(installed_policy(policy)?.job_signing_public_key_hex)
}
pub fn enrolled_binding(
policy: &Path,
registration: &rmm_protocol::EnrollmentResponse,
) -> Result<rmm_agent_core::auth::Binding> {
let p = installed_policy(policy)?;
let binding = rmm_agent_core::auth::Binding {
tenant_id: p.tenant_id,
agent_id: Uuid::parse_str(&registration.agent_id)?,
@@ -963,6 +979,7 @@ mod installed {
generation: u64,
binding: &rmm_agent_core::auth::Binding,
policy_file: &Path,
installed_tenant: Uuid,
helper: Option<&ipc::PeerPin>,
) -> Result<Self> {
let mut failure_fence = ConstructorFailureFence { armed: true };
@@ -1002,7 +1019,7 @@ mod installed {
.map(|(id, key)| Ok((id, independent_time_key(&key, &binding.public_key_hex)?)))
.collect::<Result<BTreeMap<_, _>>>()?;
ensure!(
p.tenant_id == binding.tenant_id
p.tenant_id == installed_tenant
&& p.job_signing_public_key_hex == binding.public_key_hex,
"TIME_POLICY_BINDING"
);
@@ -1393,8 +1410,9 @@ mod installed {
#[derive(Clone)]
pub struct RuntimeClock {
role: Role,
native: Option<Arc<crate::clock::NativeClock>>,
enrolled_helper: Option<ipc::PeerPin>,
local_policy: Option<(rmm_agent_core::auth::Binding, std::path::PathBuf)>,
local_policy: Option<(rmm_agent_core::auth::Binding, std::path::PathBuf, Uuid)>,
state: Arc<Mutex<Option<(u64, SignedClock)>>>,
}
#[derive(Clone)]
@@ -1406,22 +1424,33 @@ mod installed {
pub fn new(role: Role) -> Self {
Self {
role,
native: None,
enrolled_helper: None,
local_policy: None,
state: Arc::new(Mutex::new(None)),
}
}
pub fn local(role: Role) -> Self {
let mut clock = Self::new(role);
clock.native = Some(Arc::new(crate::clock::NativeClock::new()));
clock
}
pub fn uses_local_clock(&self) -> bool {
self.native.is_some()
}
pub fn enrolled_local_service(
binding: rmm_agent_core::auth::Binding,
policy: std::path::PathBuf,
installed_tenant: Uuid,
) -> Self {
let mut value = Self::new(Role::Agent);
value.local_policy = Some((binding, policy));
value.local_policy = Some((binding, policy, installed_tenant));
value
}
pub fn enrolled_windows_update_helper(binding: rmm_agent_core::auth::Binding, helper: ipc::PeerPin) -> Self {
let mut value = Self::new(Role::SystemHelper);
value.local_policy = Some((binding, std::path::PathBuf::from(r"C:\ProgramData\RustDiscovery\job-time.json")));
let installed_tenant = binding.tenant_id;
value.local_policy = Some((binding, std::path::PathBuf::from(r"C:\ProgramData\RustDiscovery\job-time.json"), installed_tenant));
value.enrolled_helper = Some(helper);
value
}
@@ -1440,8 +1469,8 @@ mod installed {
ensure!(generation > 0, "TIME_RUNTIME_GENERATION");
self.close()?;
let clock = match &self.local_policy {
Some((binding, policy)) => {
SignedClock::open_local_service(generation, binding, policy, self.enrolled_helper.as_ref()).await?
Some((binding, policy, installed_tenant)) => {
SignedClock::open_local_service(generation, binding, policy, *installed_tenant, self.enrolled_helper.as_ref()).await?
}
None => SignedClock::open(self.role, generation).await?,
};
@@ -1509,6 +1538,10 @@ mod installed {
}
fn bounds(&self) -> Result<(chrono::DateTime<chrono::Utc>, chrono::DateTime<chrono::Utc>)> {
ensure!(self.generation > 0, "TIME_RUNTIME_NO_CONNECTION");
if let Some(native) = &self.source.native {
return rmm_agent_core::reconcile::SyncClock::bounds(native.as_ref())
.context("TIME_WINDOWS_LOCAL_CLOCK");
}
let mut state = self
.source
.state
@@ -1525,6 +1558,10 @@ mod installed {
}
fn reading(&self) -> Result<rmm_agent_core::reconcile::ClockReading> {
ensure!(self.generation > 0, "TIME_RUNTIME_NO_CONNECTION");
if let Some(native) = &self.source.native {
return rmm_agent_core::reconcile::SyncClock::reading(native.as_ref())
.context("TIME_WINDOWS_LOCAL_CLOCK");
}
let mut state = self
.source
.state
@@ -1561,7 +1598,7 @@ mod installed {
#[cfg(all(windows, feature = "system-helper"))]
pub use installed::{
BoundClock, CONFIG_PATH, Comparison, RuntimeClock, SignedClock, UPDATER_CONFIG_PATH,
enrolled_binding,
enrolled_binding, enrolled_transfer_signer,
};
#[cfg(test)]
@@ -1637,6 +1674,26 @@ mod tests {
}
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn helper_sampling_spaces_successes_and_retries_failures_quickly() {
use std::time::Duration;
assert_eq!(
SamplingMode::Jobs.helper_interval(true),
Duration::from_secs(5)
);
assert_eq!(
SamplingMode::Jobs.helper_interval(false),
Duration::from_millis(500)
);
for verified in [false, true] {
assert_eq!(
SamplingMode::Passive.helper_interval(verified),
Duration::from_secs(30)
);
}
}
fn sample(ns: u64) -> Sample {
Sample {
wall_utc_ns: 1_800_000_000_000_000_000 + ns as i64,
+1 -1
View File
@@ -103,7 +103,7 @@ impl Observer {
}
self.cached()
.cloned()
.unwrap_or_else(|| Observation::not_configured(Platform::LinuxX64, chrono::Utc::now()))
.unwrap_or_else(|| Observation::not_configured(Platform::NATIVE_LINUX, chrono::Utc::now()))
}
pub fn catalog_record(&self) -> Option<GuardedRecord> {
self.cached()?;
+9 -7
View File
@@ -15,7 +15,7 @@ use rmm_domain::{
};
use rmm_privileged_helper::{Binding, TrustedClock};
use rmm_updater::{
ActiveRelease, Platform as ReleasePlatform, ReleaseRelation, SignedRelease, TrustedContext,
ActiveRelease, ReleaseRelation, SignedRelease, TrustedContext,
TrustedUpdateKey, UpdateKeyring, verify_release,
};
use serde::Deserialize;
@@ -346,12 +346,12 @@ impl Trust {
.try_into()
.map_err(|_| anyhow::anyhow!("UPDATE_CATALOG_KEY_SIZE"))?;
ensure!(
key.platform == Platform::LinuxX64 && key.public_key_hex != binding.public_key_hex,
key.platform == Platform::NATIVE_LINUX && key.public_key_hex != binding.public_key_hex,
"UPDATE_CATALOG_KEY_SEPARATION"
);
trusted.push(TrustedUpdateKey::new(
public,
ReleasePlatform::LinuxX64,
Platform::NATIVE_LINUX.into(),
key.first_sequence,
key.last_sequence,
key.revoked,
@@ -426,7 +426,7 @@ impl Trust {
},
&self.keys,
&TrustedContext {
platform: ReleasePlatform::LinuxX64,
platform: Platform::NATIVE_LINUX.into(),
now_utc_seconds: now,
keyring_generation: self.generation,
protocol_major: self.protocol,
@@ -522,16 +522,18 @@ async fn request(
async fn fetch(client: &reqwest::Client, address: SocketAddr) -> Result<Vec<u8>> {
let response = client
.get(format!(
"https://{}:{}/agent/v1/releases/current?platform=x86_64-unknown-linux-gnu",
"https://{}:{}/agent/v1/releases/current?platform={}",
crate::https_channel::server_name()?,
address.port()
address.port(),
Platform::NATIVE_LINUX.target_triple()
))
.send()
.await?;
crate::https_channel::bounded_body(response, RESPONSE_LIMIT, "application/json").await
}
#[cfg(test)]
// These fixtures provision the legacy x86-64 manual updater profile.
#[cfg(all(test, target_arch = "x86_64"))]
mod tests {
use super::*;
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
+1 -1
View File
@@ -319,7 +319,7 @@ pub fn context(
let scope = match job.action {
rmm_domain::jobs::Action::UpdateAgent { ref update } => {
match update.manifest().map_err(anyhow::Error::msg)?.platform {
update::Platform::LinuxX64 => "linux.base",
update::Platform::LinuxX64 | update::Platform::LinuxArm64 => "linux.base",
update::Platform::WindowsX64 | update::Platform::WindowsGnuX64 => "windows.base",
}
}
+127 -16
View File
@@ -7,6 +7,7 @@ mod certificate_rotation;
mod helper_sync;
mod https_channel;
mod jobs;
mod proxmox_api;
mod reconcile;
#[cfg(test)]
mod runtime_tests;
@@ -64,6 +65,8 @@ struct Agent<C: reconcile::SyncClock = reconcile::NativeClock> {
rotation_policy: certificate_rotation::RotationPolicy,
catalog: agent_update_catalog::Checker,
update_observer: agent_update_apply::Observer,
transfer_state: Option<PathBuf>,
transfer_gate: Option<fs::File>,
}
#[derive(Default)]
struct EffectFence {
@@ -465,6 +468,11 @@ impl<C: reconcile::SyncClock> Agent<C> {
"heartbeat/1".into(),
"agent.update/1".into(),
];
if self.transfer_state.is_some() {
capabilities.push(rmm_domain::customer_transfer::CAPABILITY.into());
}
#[cfg(target_arch = "aarch64")]
capabilities.push(rmm_domain::agent_update::LINUX_ARM64_RELEASE_CAPABILITY.into());
if self.binding.is_some() {
capabilities.push("rmm.sync.v1".into());
}
@@ -474,30 +482,64 @@ impl<C: reconcile::SyncClock> Agent<C> {
if rmm_agent_linux::actions::Interpreter::PosixSh.available() {
capabilities.push("run_script".into());
}
capabilities.extend(helper_sync::script_capabilities(binding));
if self.helpers.is_some() {
capabilities.extend(helper_sync::script_capabilities(binding));
}
}
capabilities.sort();
capabilities.dedup();
capabilities
}
fn queue_inventory(&mut self) {
async fn stage_customer_transfer(&mut self) -> Result<bool> {
let (Some(dir), Some(binding)) = (&self.transfer_state, &self.binding) else {
return Ok(false);
};
if !self.running_cancels.is_empty() || self.prepared.is_some()
|| !rmm_agent_core::customer_transfer::idle(&self.db, &self.ledger)? {
return Ok(false);
}
let identity = self.certificates.current(Utc::now().timestamp())?;
let client = https_channel::mtls_client(
self.address, &self.ca, &identity.certificate_pem, &identity.private_key_pem,
)?;
let base = format!("https://{}:{}", https_channel::server_name()?, self.address.port());
let guard = fs::File::open(env::var_os("RMM_RELEASE_LOCK").context("RELEASE_LOCK_REQUIRED")?)?;
match fs2::FileExt::try_lock_shared(&guard) {
Ok(()) => (),
Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => return Ok(false),
Err(error) => return Err(error.into()),
}
if rmm_agent_core::customer_transfer::stage(&client, &base, dir, binding, transfer_csr).await? {
self.transfer_gate = Some(guard);
return Ok(true);
}
Ok(false)
}
fn queue_inventory(&mut self) -> bool {
if self.inventory_inflight || !storage::passive_collection_ready(&self.db).unwrap_or(false)
{
return;
return false;
}
self.inventory_inflight = true;
let sender = self.inventory_tx.clone();
let extended = env::var("RMM_EXTENDED_INVENTORY").unwrap_or_else(|_| "1".into()) == "1";
let binding = self.binding.clone();
let catalog = self.catalog.cached(Utc::now());
let runtime = tokio::runtime::Handle::current();
tokio::task::spawn_blocking(move || {
let collected_at = Utc::now();
let result = rmm_agent_linux::wire::snapshot(extended).and_then(|mut inventory| {
if extended {
inventory["observations"]
let proxmox = runtime.block_on(proxmox_api::observation(
rmm_agent_linux::inventory::proxmox_ve(),
));
let scopes = inventory["observations"]
.as_array_mut()
.context("INVENTORY_OBSERVATIONS")?
.push(apt::planning_observation(binding.as_ref()));
.context("INVENTORY_OBSERVATIONS")?;
scopes.push(apt::planning_observation(binding.as_ref()));
if let Some(observation) = proxmox {
scopes.push(proxmox_api::wire(&observation)?);
}
inventory["collection_completed_at"] = serde_json::json!(Utc::now());
}
if let Some(report) = catalog {
@@ -511,12 +553,12 @@ impl<C: reconcile::SyncClock> Agent<C> {
.push(agent_update_catalog::observation(report, Utc::now()));
inventory["collection_completed_at"] = serde_json::json!(Utc::now());
}
let bytes = serde_json::to_vec(&inventory)?;
ensure!(bytes.len() <= 512 * 1024, "INVENTORY_WIRE_LIMIT");
let bytes = rmm_agent_linux::wire::encode_snapshot(inventory)?;
Ok((bytes, Utc::now().timestamp_millis()))
});
let _ = sender.blocking_send(result);
});
true
}
fn queue_update_observation(&mut self, generation: u64) -> Result<()> {
if !storage::passive_collection_ready(&self.db)? {
@@ -647,6 +689,7 @@ impl<C: reconcile::SyncClock> Agent<C> {
let mut inventory_at = Instant::now();
let mut update_at = Instant::now();
let mut heartbeat_at = Instant::now();
let mut previous_cpu = None;
let mut heartbeat_after = Duration::from_millis(rand::random_range(27_000..33_000));
let mut tick = tokio::time::interval(Duration::from_secs(5));
let mut sync_at = Instant::now();
@@ -749,10 +792,10 @@ impl<C: reconcile::SyncClock> Agent<C> {
let decoded=proof.decode().map_err(anyhow::Error::msg)?;
let event:Option<rmm_updater_handoff::Event>=decoded.events.last().map(|raw|serde_json::from_slice(raw)).transpose()?;
if event.is_some_and(|event|matches!(event.evidence,rmm_updater_handoff::Evidence::ServicesStarted{..}|rmm_updater_handoff::Evidence::RollbackServicesStarted{..})) {
if health_inventory_prefix.as_deref()!=Some(proof.head_sha256.as_str()) {
if health_inventory_prefix.as_deref()!=Some(proof.head_sha256.as_str()) && self.queue_inventory() {
// Fixed update health includes one genuinely new
// normal base sample, never a fresh timestamp on old data.
self.queue_inventory();health_inventory_prefix=Some(proof.head_sha256.clone());
health_inventory_prefix=Some(proof.head_sha256.clone());
}
if health_task.is_none() && self.prepared.is_none() {
match self.start_update_health(proof) {Ok(task)=>health_task=task,Err(error)=>{report_closed_gate(&error);self.close_gate();needs_sync=true;}}
@@ -807,11 +850,21 @@ impl<C: reconcile::SyncClock> Agent<C> {
}
if first_heartbeat || heartbeat_at.elapsed() >= heartbeat_after {
first_heartbeat = false;
let cpu = rmm_agent_linux::inventory::cpu_times();
let cpu_usage_basis_points = cpu.zip(previous_cpu)
.and_then(|(current, previous)| current.usage_since(previous));
previous_cpu = cpu;
let memory = rmm_agent_linux::inventory::memory().data;
let heartbeat = self.envelope(
generation,
Payload::Heartbeat(Heartbeat {
instance_id: self.instance.clone(),
uptime_seconds: self.started.elapsed().as_secs(),
system_usage: Some(SystemUsage {
cpu_usage_basis_points,
ram_available_bytes: memory.as_ref().and_then(|m| m.available_bytes),
ram_total_bytes: memory.map(|m| m.total_bytes),
}),
}),
)?;
ws.send(&heartbeat).await?;
@@ -821,6 +874,7 @@ impl<C: reconcile::SyncClock> Agent<C> {
Payload::Heartbeat(Heartbeat {
instance_id: self.instance.clone(),
uptime_seconds: self.started.elapsed().as_secs(),
system_usage: None,
}),
)?;
ws.poll(poll).await?;
@@ -828,12 +882,18 @@ impl<C: reconcile::SyncClock> Agent<C> {
heartbeat_at = Instant::now();
heartbeat_after = Duration::from_millis(rand::random_range(27_000..33_000));
}
if inventory_at.elapsed() >= Duration::from_secs(60) {
self.queue_inventory();
// A simultaneous update report can fill the durable queue. Keep
// inventory due until collection actually starts.
if inventory_at.elapsed() >= Duration::from_secs(60) && self.queue_inventory() {
inventory_at = Instant::now();
}
if rotation_at.elapsed() >= Duration::from_secs(30) {
rotation_at = Instant::now();
match self.stage_customer_transfer().await {
Ok(true) => return Ok(()),
Ok(false) => (),
Err(_) => eprintln!("customer transfer waiting for a valid handoff"),
}
if self
.certificates
.due(self.rotation_policy, Utc::now().timestamp())?
@@ -858,6 +918,27 @@ impl<C: reconcile::SyncClock> Agent<C> {
}
}
fn transfer_csr(previous: Option<&[u8]>) -> Result<(Vec<u8>, String)> {
let key = match previous {
Some(raw) => PKey::private_key_from_pem(raw)?,
None => {
let group = EcGroup::from_curve_name(Nid::X9_62_PRIME256V1)?;
PKey::from_ec_key(EcKey::generate(&group)?)?
}
};
let mut csr = X509Req::builder()?;
let mut name = X509NameBuilder::new()?;
name.append_entry_by_text("CN", "customer-transfer")?;
csr.set_subject_name(&name.build())?;
csr.set_pubkey(&key)?;
csr.sign(&key, MessageDigest::sha256())?;
Ok((key.private_key_to_pem_pkcs8()?, String::from_utf8(csr.build().to_pem()?)?))
}
fn customer_transfer_signer(dir: &Path) -> Result<rmm_privileged_helper::SigningKey> {
rmm_privileged_helper::SigningKey::read(&dir.join("job-signing-key.json"), unsafe { libc::geteuid() })
}
async fn enrollment(dir: &Path, address: std::net::SocketAddr, ca: &[u8]) -> Result<String> {
let identity = dir.join("identity.json");
if identity.exists() {
@@ -935,6 +1016,18 @@ async fn main() -> Result<()> {
PathBuf::from(env::var("RMM_AGENT_STATE").unwrap_or_else(|_| "devstate/agent".into()));
storage::prepare_dir(&dir)?;
let _lock = storage::lock(&dir)?;
let mut release_guard = None;
loop {
if rmm_agent_core::customer_transfer::pending(&dir)? {
let signer = customer_transfer_signer(&dir)?;
rmm_agent_core::customer_transfer::activate(&dir, &signer.public_key_hex)?;
}
drop(release_guard.take());
release_guard = Some(run_agent(&dir).await?);
}
}
async fn run_agent(dir: &Path) -> Result<fs::File> {
let address = env::var("RMM_CONNECT_ADDR")
.unwrap_or_else(|_| "127.0.0.1:18443".into())
.parse()?;
@@ -963,6 +1056,20 @@ async fn main() -> Result<()> {
enrollment_generation: 1, assignment_version: 1, public_key_hex: String::new(),
});
}
let transfer_state = (env::var_os("RMM_JOB_BINDING_FILE").is_none()
&& env::var_os("RMM_ENROLLMENT_TENANT").is_some()
&& env::var("RMM_JOB_SIGNING_KEY_BOOTSTRAP").as_deref() == Ok("1")
&& env::var_os("RMM_RELEASE_LOCK").is_some()).then(|| dir.to_owned());
let mut moved = false;
if transfer_state.is_some() && let Some(binding) = binding.as_mut()
&& dir.join("job-signing-key.json").try_exists()? {
let identity: EnrollmentResponse = serde_json::from_slice(&fs::read(dir.join("identity.json"))?)?;
let signer = customer_transfer_signer(dir)?;
if let Some(tenant) = rmm_agent_core::customer_transfer::assigned_tenant(dir, &identity, &signer.public_key_hex)? {
binding.tenant_id = tenant;
moved = true;
}
}
let certificate_dir = dir.join("certificates");
let certificates = certificate_rotation::RotationStore::open(
&certificate_dir,
@@ -994,10 +1101,9 @@ async fn main() -> Result<()> {
let sync = binding
.as_ref()
.map(|b| reconcile::ClientSync::new(b.clone()));
let helpers = binding
.as_ref()
.map(helper_sync::Helpers::configured)
.transpose()?;
let helpers = if moved { None } else {
binding.as_ref().map(helper_sync::Helpers::configured).transpose()?
};
let mut rotation_policy = certificate_rotation::RotationPolicy::default();
if let Ok(after) = env::var("RMM_ROTATION_TEST_AFTER_SECONDS") {
ensure!(
@@ -1039,6 +1145,8 @@ async fn main() -> Result<()> {
rotation_policy,
catalog: agent_update_catalog::Checker::default(),
update_observer: agent_update_apply::Observer::default(),
transfer_state,
transfer_gate: None,
};
agent.recover_interrupted()?;
let mut backoff = 1_u64;
@@ -1048,6 +1156,9 @@ async fn main() -> Result<()> {
report_closed_gate(&error);
}
agent.close_gate();
if let Some(guard) = agent.transfer_gate.take() {
return Ok(guard);
}
eprintln!("channel unavailable; retrying with jitter (no jobs can execute)");
if started.elapsed() > Duration::from_secs(60) {
backoff = 1;
+88
View File
@@ -0,0 +1,88 @@
//! Local, read-only Proxmox VE API client for the `linux.proxmox` observation.
//!
//! Only `https://127.0.0.1:8006` with the cluster CA copied by the installer. The
//! PVEAuditor token arrives as the systemd credential `rmm-proxmox-token`; it is never
//! logged, persisted by the agent or placed in an observation.
use anyhow::{Context, Result, ensure};
use reqwest::{Client, StatusCode};
use rmm_agent_linux::inventory::{Observation, proxmox};
use serde_json::Value;
use std::{path::PathBuf, time::Duration};
const BASE: &str = "https://127.0.0.1:8006/api2/json";
const CREDENTIAL: &str = "rmm-proxmox-token";
const DEFAULT_CA: &str = "/etc/rmm/pve-ca.pem";
const MAX_RESPONSE: usize = 2 * 1024 * 1024;
const TOTAL_BUDGET: Duration = Duration::from_secs(45);
struct LocalApi {
client: Client,
authorization: String,
}
impl proxmox::Api for LocalApi {
async fn get(&self, path: &str) -> Result<Value> {
ensure!(path.starts_with('/') && !path.contains(".."), "PROXMOX_PATH");
let mut response = self.client.get(format!("{BASE}{path}"))
.header(reqwest::header::AUTHORIZATION, &self.authorization)
.send().await.map_err(|_| anyhow::anyhow!("PROXMOX_UNREACHABLE"))?;
match response.status() {
StatusCode::OK => (),
StatusCode::UNAUTHORIZED => anyhow::bail!("PROXMOX_TOKEN_REJECTED"),
StatusCode::FORBIDDEN => anyhow::bail!("PROXMOX_PERMISSION_DENIED"),
status => anyhow::bail!("PROXMOX_HTTP_{}", status.as_u16()),
}
ensure!(response.content_length().is_none_or(|n| n <= MAX_RESPONSE as u64), "PROXMOX_RESPONSE_LIMIT");
let mut body = Vec::new();
while let Some(chunk) = response.chunk().await? {
ensure!(chunk.len() <= MAX_RESPONSE.saturating_sub(body.len()), "PROXMOX_RESPONSE_LIMIT");
body.extend_from_slice(&chunk);
}
let mut value: Value = serde_json::from_slice(&body).map_err(|_| anyhow::anyhow!("PROXMOX_API_SHAPE"))?;
Ok(value["data"].take())
}
}
fn credential_path() -> Option<PathBuf> {
std::env::var_os("CREDENTIALS_DIRECTORY").map(|d| PathBuf::from(d).join(CREDENTIAL))
}
fn client(ca: &[u8]) -> Result<Client> {
Ok(Client::builder()
.https_only(true)
.tls_built_in_root_certs(false)
.add_root_certificate(reqwest::Certificate::from_pem(ca)?)
.no_proxy()
.redirect(reqwest::redirect::Policy::none())
.connect_timeout(Duration::from_secs(5))
.timeout(Duration::from_secs(15))
.build()?)
}
/// `None` on hosts without Proxmox VE: no scope is reported there at all.
pub async fn observation(proxmox_ve: bool) -> Option<Observation<proxmox::Proxmox>> {
if !proxmox_ve {
return None;
}
let Some(token) = credential_path().and_then(|p| std::fs::read_to_string(p).ok()) else {
return Some(proxmox::not_configured());
};
let Some(authorization) = proxmox::token_header(&token) else {
return Some(proxmox::failed("PROXMOX_TOKEN_INVALID"));
};
let ca_path = std::env::var("RMM_PROXMOX_CA_FILE").unwrap_or_else(|_| DEFAULT_CA.into());
let api = match std::fs::read(&ca_path).context("ca").and_then(|ca| client(&ca)) {
Ok(client) => LocalApi { client, authorization },
Err(_) => return Some(proxmox::failed("PROXMOX_CA_UNAVAILABLE")),
};
let now = chrono::Utc::now().timestamp();
Some(tokio::time::timeout(TOTAL_BUDGET, proxmox::collect(&api, now)).await
.unwrap_or_else(|_| proxmox::failed("PROXMOX_TIMEOUT")))
}
/// Wire form with the scope label the server's `linux.base` validation expects.
pub fn wire(observation: &Observation<proxmox::Proxmox>) -> Result<Value> {
let mut value = serde_json::to_value(observation)?;
value["scope"] = Value::String("linux.proxmox".into());
Ok(value)
}
+4
View File
@@ -152,6 +152,8 @@ fn new_process_persists_interruption_before_helper_io_without_unlock_or_repeat()
rotation_policy: certificate_rotation::RotationPolicy::default(),
catalog: agent_update_catalog::Checker::default(),
update_observer: agent_update_apply::Observer::default(),
transfer_state: None,
transfer_gate: None,
};
// A forged local consumption/start cannot become a purported grant result.
agent
@@ -541,6 +543,8 @@ async fn run(mode: wire_channel::Mode, trusted: bool, forged: bool, rounds: u64)
rotation_policy: certificate_rotation::RotationPolicy::default(),
catalog: agent_update_catalog::Checker::default(),
update_observer: agent_update_apply::Observer::default(),
transfer_state: None,
transfer_gate: None,
};
for _ in 0..rounds {
assert!(
@@ -42,7 +42,7 @@ fn fixture(platform: Platform, full: bool) -> serde_json::Value {
signature: URL_SAFE_NO_PAD.encode([1_u8; 64]),
};
let transition = match platform {
Platform::LinuxX64 if full => PeerPinTransition::LinuxGeneralFullV1 {
Platform::LinuxX64 | Platform::LinuxArm64 if full => PeerPinTransition::LinuxGeneralFullV1 {
general_pin_sha256: "10".repeat(32),
full: LinuxFullTransition {
schema_version: 1,
@@ -57,7 +57,7 @@ fn fixture(platform: Platform, full: bool) -> serde_json::Value {
effective_signing_key_id: "17".repeat(32),
},
},
Platform::LinuxX64 => PeerPinTransition::UnchangedGeneralPinV1 {
Platform::LinuxX64 | Platform::LinuxArm64 => PeerPinTransition::UnchangedGeneralPinV1 {
general_pin_sha256: "10".repeat(32),
},
Platform::WindowsX64 | Platform::WindowsGnuX64 => PeerPinTransition::WindowsCodePairV1 {
@@ -110,7 +110,7 @@ fn fixture(platform: Platform, full: bool) -> serde_json::Value {
update: Box::new(update),
},
execution_identity: match platform {
Platform::LinuxX64 => "root",
Platform::LinuxX64 | Platform::LinuxArm64 => "root",
Platform::WindowsX64 | Platform::WindowsGnuX64 => "system",
}
.into(),
+24 -1
View File
@@ -50,11 +50,34 @@ impl UpdateKeyringFile {
pub enum Platform {
#[serde(rename = "x86_64-unknown-linux-gnu")]
LinuxX64,
#[serde(rename = "aarch64-unknown-linux-gnu")]
LinuxArm64,
#[serde(rename = "x86_64-pc-windows-msvc")]
WindowsX64,
#[serde(rename = "x86_64-pc-windows-gnu")]
WindowsGnuX64,
}
pub const LINUX_ARM64_RELEASE_CAPABILITY: &str = "agent.release.linux.arm64/1";
impl Platform {
#[cfg(all(target_os = "linux", target_arch = "aarch64"))]
pub const NATIVE_LINUX: Self = Self::LinuxArm64;
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
pub const NATIVE_LINUX: Self = Self::LinuxX64;
pub const fn is_linux(self) -> bool {
matches!(self, Self::LinuxX64 | Self::LinuxArm64)
}
pub const fn target_triple(self) -> &'static str {
match self {
Self::LinuxX64 => "x86_64-unknown-linux-gnu",
Self::LinuxArm64 => "aarch64-unknown-linux-gnu",
Self::WindowsX64 => "x86_64-pc-windows-msvc",
Self::WindowsGnuX64 => "x86_64-pc-windows-gnu",
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum ArtifactRole {
@@ -708,7 +731,7 @@ impl ResolvedAgentUpdate {
}
pub fn execution_identity(&self) -> Option<&'static str> {
self.manifest().ok().map(|m| match m.platform {
Platform::LinuxX64 => "root",
Platform::LinuxX64 | Platform::LinuxArm64 => "root",
Platform::WindowsX64 | Platform::WindowsGnuX64 => "system",
})
}
+50
View File
@@ -0,0 +1,50 @@
use serde::{Deserialize, Serialize};
use uuid::Uuid;
pub const CAPABILITY: &str = "device.customer_transfer/1";
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct Offer {
pub transfer_id: Uuid,
pub source_tenant_id: Uuid,
pub source_agent_id: Uuid,
pub target_tenant_id: Uuid,
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Binding {
pub offer: Offer,
pub asset_id: Uuid,
pub agent_id: Uuid,
pub certificate_sha256: String,
}
impl Binding {
pub fn signing_bytes(&self) -> Result<Vec<u8>, serde_json::Error> {
let mut bytes = b"rust-discovery:customer-transfer:1\0".to_vec();
bytes.extend(serde_json::to_vec(self)?);
Ok(bytes)
}
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Receipt {
pub binding: Binding,
pub signature_hex: String,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct CertificateRequest {
pub csr_pem: String,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct CertificateResponse {
pub certificate_pem: String,
pub receipt: Receipt,
}
+65 -2
View File
@@ -1,8 +1,26 @@
//! Additive bounded passive metadata; the strict original Inventory stays intact.
mod opnsense;
pub mod proxmox;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use serde_json::Value;
pub const SOFTWARE_OBSERVATION_MAX_BYTES: usize = 128 * 1024;
pub const LINUX_SOFTWARE_OBSERVATION_MAX_BYTES: usize = 256 * 1024;
pub const PROXMOX_OBSERVATION_MAX_BYTES: usize = 128 * 1024;
pub fn firmware_identifier(value: &str) -> Option<String> {
let value = value.trim_matches(|c: char| c == '\0' || c.is_whitespace());
let lower = value.to_ascii_lowercase();
(!value.is_empty()
&& value.len() <= 255
&& !value.chars().any(char::is_control)
&& !matches!(lower.as_str(), "none" | "unknown" | "not specified" | "not applicable"
| "default string" | "to be filled by o.e.m." | "system serial number")
&& !value.bytes().all(|b| b == b'0'))
.then(|| value.to_owned())
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum DeviceType {
@@ -11,6 +29,22 @@ pub enum DeviceType {
Hypervisor,
}
#[derive(Clone, Copy)]
pub struct CpuTimes {
pub total: u64,
pub idle: u64,
}
impl CpuTimes {
pub fn usage_since(self, previous: Self) -> Option<u32> {
let total = self.total.checked_sub(previous.total)?;
let idle = self.idle.checked_sub(previous.idle)?;
let busy = total.checked_sub(idle)?;
(total != 0)
.then(|| ((u128::from(busy) * 10_000 + u128::from(total / 2)) / u128::from(total)) as u32)
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Observation {
@@ -33,6 +67,9 @@ pub fn validate_for_scope(
now: DateTime<Utc>,
scope: &str,
) -> Result<Value, &'static str> {
if scope == "opnsense.base" {
return opnsense::validate(bytes);
}
if scope == "agent.update" {
if bytes.len() > 65536 {
return Err("AGENT_UPDATE_INVENTORY_SIZE");
@@ -75,7 +112,7 @@ pub fn validate_for_scope(
.map_err(|_| "INVALID_AGENT_UPDATE_OBSERVATION")?
!= update
|| !typed.agent_update.validate()
|| (typed.agent_update.platform == crate::agent_update::Platform::LinuxX64) != (scope == "linux.base")
|| typed.agent_update.platform.is_linux() != (scope == "linux.base")
|| typed.agent_update.observed_at - now > chrono::Duration::seconds(45)
|| now - typed.agent_update.observed_at > chrono::Duration::hours(72)
{
@@ -136,10 +173,16 @@ pub fn validate_for_scope(
}
let mut scopes = std::collections::BTreeSet::new();
for item in observations {
let max_bytes = match (scope, item["scope"].as_str()) {
("linux.base", Some("linux.software")) => LINUX_SOFTWARE_OBSERVATION_MAX_BYTES,
("linux.base", Some("linux.proxmox")) => PROXMOX_OBSERVATION_MAX_BYTES,
("windows.base", Some("windows.software")) => SOFTWARE_OBSERVATION_MAX_BYTES,
_ => 32768,
};
if serde_json::to_vec(item)
.map_err(|_| "INVALID_OBSERVATION")?
.len()
> 32768
> max_bytes
{
return Err("OBSERVATION_SIZE");
}
@@ -174,6 +217,9 @@ pub fn validate_for_scope(
return Err("INVALID_AGENT_UPDATE_CATALOG_REPORT");
}
}
if o.scope == "linux.proxmox" && o.quality == "valid" {
proxmox::validate(&o.data)?;
}
if !o.scope.starts_with(prefix)
|| o.scope.len() > 64
|| !(o
@@ -234,6 +280,23 @@ mod tests {
assert!(validate(&serde_json::to_vec(&unknown).unwrap(), now).is_err());
}
#[test]
fn larger_linux_package_lists_fit_the_software_scope() {
let now = Utc::now();
let mut v = base();
let package = json!({"name":"some-package","version":"1.0","architecture":"amd64"});
v["collection_started_at"] = json!(now);
v["collection_completed_at"] = json!(now);
v["observations"] = json!([{"scope":"linux.software","source_id":"dpkg-query","observed_at":now,"quality":"valid","completeness":"complete","truncated":false,"error_code":null,"data":vec![package; 2200]}]);
let bytes = serde_json::to_vec(&v).unwrap();
assert!(bytes.len() > SOFTWARE_OBSERVATION_MAX_BYTES);
assert!(validate(&bytes, now).is_ok());
v["observations"][0]["scope"] = json!("linux.hardware");
assert_eq!(
validate(&serde_json::to_vec(&v).unwrap(), now).unwrap_err(),
"OBSERVATION_SIZE"
);
}
#[test]
fn windows_scope_does_not_weaken_linux_or_accept_platform_spoofing() {
let now = Utc::now();
let mut v = base();
@@ -0,0 +1,107 @@
//! Allowlisted wire document. No appliance configuration or VPN secrets on the wire.
use serde::Deserialize;
use serde_json::Value;
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Snapshot {
os: String,
hostname: String,
architecture: String,
root_total_bytes: u64,
root_available_bytes: u64,
cpu_model: String,
logical_processors: u32,
memory_total_bytes: u64,
firewall: Rows,
vpn_configured: Rows,
wireguard: Rows,
openvpn: Rows,
ipsec: Rows,
hardware: Option<Hardware>,
software: Option<Records<Software>>,
interfaces: Option<Records<Interface>>,
users: Option<Rows>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Hardware {
device_type: String,
booted_at: chrono::DateTime<chrono::Utc>,
manufacturer: Option<String>,
bios_serial_number: Option<String>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Records<T> { status: String, entries: Vec<T> }
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Software { name: String, version: String }
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Interface { name: String, mac: Option<String>, addresses: Vec<String> }
impl<T> Records<T> {
fn valid(&self, limit: usize, check: impl Fn(&T) -> bool) -> bool {
matches!(self.status.as_str(),"ok"|"partial"|"unavailable")
&& (self.status != "unavailable" || self.entries.is_empty())
&& self.entries.len() <= limit && self.entries.iter().all(check)
}
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Rows {
status: String,
entries: Vec<String>,
}
impl Rows {
fn valid(&self) -> bool {
matches!(self.status.as_str(), "ok" | "partial" | "unavailable")
&& (self.status != "unavailable" || self.entries.is_empty())
&& self.entries.len() <= 128
&& self.entries.iter().all(|s| clean(s, 512))
}
}
fn clean(s: &str, limit: usize) -> bool {
!s.trim().is_empty() && s.len() <= limit && !s.chars().any(char::is_control)
}
pub(super) fn validate(bytes: &[u8]) -> Result<Value, &'static str> {
let err = "INVALID_OPNSENSE_INVENTORY";
if bytes.len() > 384 * 1024 { return Err(err); }
let s: Snapshot = serde_json::from_slice(bytes).map_err(|_| err)?;
if !s.os.starts_with("OPNsense ") || !clean(&s.os, 128)
|| !clean(&s.hostname, 255) || s.architecture != "amd64"
|| s.root_total_bytes == 0 || s.root_available_bytes > s.root_total_bytes
|| !clean(&s.cpu_model, 256) || s.logical_processors == 0 || s.memory_total_bytes == 0
|| ![&s.firewall,&s.vpn_configured,&s.wireguard,&s.openvpn,&s.ipsec].iter().all(|r| r.valid())
|| s.users.is_some_and(|r| !r.valid())
|| s.hardware.is_some_and(|h| !matches!(h.device_type.as_str(),"physical"|"virtual")
|| h.booted_at.timestamp() < 0
|| [h.manufacturer,h.bios_serial_number].into_iter().flatten().any(|v| !clean(&v,256)))
|| s.software.is_some_and(|r| !r.valid(1024, |p| clean(&p.name,256) && clean(&p.version,256)))
|| s.interfaces.is_some_and(|r| !r.valid(128, |i| clean(&i.name,128)
&& i.mac.as_ref().is_none_or(|m| clean(m,64)) && i.addresses.len()<=256
&& i.addresses.iter().all(|s| clean(s,128))))
{ return Err(err); }
serde_json::from_slice(bytes).map_err(|_| err)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn no_config_blob_and_no_false_empty_on_error() {
let rows = json!({"status":"ok","entries":[]});
let mut doc = json!({"os":"OPNsense 26.7","hostname":"test","architecture":"amd64",
"root_total_bytes":20,"root_available_bytes":10,"cpu_model":"Test","logical_processors":2,
"memory_total_bytes":10,"firewall":rows,"vpn_configured":rows,"wireguard":rows,"ipsec":rows,"openvpn":rows});
assert!(validate(doc.to_string().as_bytes()).is_ok());
doc["wireguard"]["private_key"] = json!("not-permitted");
assert!(validate(doc.to_string().as_bytes()).is_err());
doc["wireguard"] = json!({"status":"unavailable","entries":["peer"]});
assert!(validate(doc.to_string().as_bytes()).is_err());
}
}
@@ -0,0 +1,296 @@
//! Strict `linux.proxmox` wire document and the deterministic backup evaluation.
//! The agent reports raw scheduler data; whether every scheduled backup ran is
//! decided here so the server, API and UI share one rule.
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use std::collections::{BTreeMap, BTreeSet};
/// Runs that start shortly before the computed due time still count (clock skew).
const START_TOLERANCE_SECONDS: i64 = 120;
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Proxmox {
pub node: String,
pub cluster: Option<String>,
pub version: Option<String>,
pub guests: Vec<Guest>,
pub backup_jobs: Vec<BackupJob>,
pub backup_runs: Vec<BackupRun>,
pub not_backed_up: Vec<u32>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Guest {
pub vmid: u32,
pub name: String,
pub kind: String,
pub status: String,
pub template: bool,
pub cpus: Option<u32>,
pub memory_bytes: Option<u64>,
pub disk_bytes: Option<u64>,
pub uptime_seconds: Option<u64>,
pub pool: Option<String>,
pub macs: Vec<String>,
pub smbios_uuid: Option<String>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct BackupJob {
pub id: String,
pub enabled: bool,
pub schedule: Option<String>,
pub storage: Option<String>,
pub mode: Option<String>,
pub selection: String,
pub covered_vmids: Vec<u32>,
pub next_run: Option<i64>,
pub last_expected_run: Option<i64>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct BackupRun {
pub upid: String,
pub started_at: i64,
pub ended_at: Option<i64>,
pub status: String,
pub message: Option<String>,
pub guests: Vec<GuestResult>,
}
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct GuestResult {
pub vmid: u32,
pub result: String,
}
fn clean(s: &str, limit: usize) -> bool {
!s.trim().is_empty() && s.len() <= limit && !s.chars().any(char::is_control)
}
fn opt(s: &Option<String>, limit: usize) -> bool {
s.as_deref().is_none_or(|s| clean(s, limit))
}
pub fn is_mac(s: &str) -> bool {
s.len() == 17 && s.split(':').count() == 6
&& s.split(':').all(|p| p.len() == 2 && p.bytes().all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)))
}
fn vmid(id: u32) -> bool {
(100..1_000_000_000).contains(&id)
}
/// Validates the `data` member of a valid `linux.proxmox` observation.
pub fn validate(data: &Value) -> Result<(), &'static str> {
let err = "INVALID_PROXMOX_OBSERVATION";
let p: Proxmox = serde_json::from_value(data.clone()).map_err(|_| err)?;
let guests_ok = p.guests.len() <= 512 && p.guests.iter().all(|g| {
vmid(g.vmid) && clean(&g.name, 128) && matches!(g.kind.as_str(), "qemu" | "lxc")
&& clean(&g.status, 32) && opt(&g.pool, 64) && g.macs.len() <= 32
&& g.macs.iter().all(|m| is_mac(m))
&& g.smbios_uuid.as_deref().is_none_or(|u| uuid::Uuid::parse_str(u).is_ok_and(|x| x.hyphenated().to_string() == u))
}) && p.guests.windows(2).all(|w| w[0].vmid < w[1].vmid);
let jobs_ok = p.backup_jobs.len() <= 64 && p.backup_jobs.iter().all(|j| {
clean(&j.id, 64) && opt(&j.schedule, 128) && opt(&j.storage, 64) && opt(&j.mode, 16)
&& matches!(j.selection.as_str(), "all" | "pool" | "vmid")
&& j.covered_vmids.len() <= 512 && j.covered_vmids.iter().all(|id| vmid(*id))
});
let runs_ok = p.backup_runs.len() <= 64 && p.backup_runs.iter().all(|r| {
clean(&r.upid, 160) && r.upid.starts_with("UPID:") && r.started_at > 0
&& r.ended_at.is_none_or(|e| e >= r.started_at)
&& matches!(r.status.as_str(), "ok" | "warning" | "error" | "running")
&& (r.status == "running") == r.ended_at.is_none()
&& opt(&r.message, 256) && r.guests.len() <= 512
&& r.guests.iter().all(|g| vmid(g.vmid) && matches!(g.result.as_str(), "ok" | "error"))
});
if !clean(&p.node, 64) || !opt(&p.cluster, 64) || !opt(&p.version, 32)
|| !guests_ok || !jobs_ok || !runs_ok
|| p.not_backed_up.len() > 512 || !p.not_backed_up.iter().all(|id| vmid(*id))
{
return Err(err);
}
Ok(())
}
/// Status per job and guest plus the overall answer "did every scheduled backup run?".
/// job: ok | failed | missed | running | pending (not yet due) | disabled | unknown
pub fn backup_status(data: &Value) -> Option<Value> {
let p: Proxmox = serde_json::from_value(data.clone()).ok()?;
let mut last_success: BTreeMap<u32, i64> = BTreeMap::new();
for run in &p.backup_runs {
for g in run.guests.iter().filter(|g| g.result == "ok") {
let at = run.ended_at.unwrap_or(run.started_at);
last_success.entry(g.vmid).and_modify(|t| *t = (*t).max(at)).or_insert(at);
}
}
let mut jobs = Vec::new();
let mut all_ran = true;
let mut problems = 0usize;
for job in &p.backup_jobs {
let mut guests = Vec::new();
let status = if !job.enabled {
"disabled"
} else if job.schedule.is_none() {
"unknown"
} else if let Some(due) = job.last_expected_run {
let since = due - START_TOLERANCE_SECONDS;
let window: Vec<&BackupRun> = p.backup_runs.iter().filter(|r| r.started_at >= since).collect();
let running = window.iter().any(|r| r.status == "running");
// vzdump can fail before its first guest (e.g. storage unavailable): the log
// then names no guest. Such a run counts as a failure, never as "missed".
let failed_before = |id: u32| window.iter().any(|r| r.status == "error" && r.guests.is_empty()
&& upid_vmid(&r.upid).is_none_or(|v| v == id));
let mut states = BTreeSet::new();
for id in &job.covered_vmids {
// Newest statement about this guest inside the due window wins.
let result = window.iter()
.filter_map(|r| r.guests.iter().find(|g| g.vmid == *id).map(|g| (r.started_at, g.result.as_str())))
.max_by_key(|(at, _)| *at).map(|(_, r)| r);
let state = match (result, running) {
(Some("ok"), _) => "ok",
(Some(_), _) => "failed",
(None, true) => "running",
(None, false) if failed_before(*id) => "failed",
(None, false) => "missed",
};
states.insert(state);
guests.push(json!({"vmid":id,"status":state,"last_success_at":last_success.get(id)}));
}
if states.contains("failed") { "failed" }
else if states.contains("missed") { "missed" }
else if states.contains("running") { "running" }
else { "ok" }
} else {
"pending"
};
if matches!(status, "failed" | "missed" | "unknown") {
all_ran = false;
problems += 1;
}
jobs.push(json!({"id":job.id,"status":status,"last_expected_run":job.last_expected_run,
"next_run":job.next_run,"guests":guests}));
}
let enabled = p.backup_jobs.iter().filter(|j| j.enabled).count();
Some(json!({
"all_scheduled_backups_ran": if enabled == 0 { Value::Null } else { Value::Bool(all_ran) },
"enabled_jobs": enabled,
"problem_jobs": problems,
"jobs": jobs,
"last_success_at": last_success,
}))
}
/// UPID:node:pid:pstart:starttime:type:id:user: with `id` set for single-guest tasks.
fn upid_vmid(upid: &str) -> Option<u32> {
upid.split(':').nth(6).filter(|id| !id.is_empty()).and_then(|id| id.parse().ok())
}
#[cfg(test)]
mod tests {
use super::*;
fn sample() -> Value {
json!({"node":"pve","cluster":null,"version":"9.2.2",
"guests":[
{"vmid":100,"name":"debian","kind":"qemu","status":"running","template":false,"cpus":1,
"memory_bytes":1,"disk_bytes":1,"uptime_seconds":5,"pool":null,
"macs":["bc:24:11:88:35:97"],"smbios_uuid":"787f4ae9-446f-4fac-b3d8-4074170bf823"},
{"vmid":101,"name":"alpine","kind":"lxc","status":"running","template":false,"cpus":1,
"memory_bytes":1,"disk_bytes":1,"uptime_seconds":5,"pool":null,"macs":[],"smbios_uuid":null}],
"backup_jobs":[
{"id":"a","enabled":true,"schedule":"*/2:00","storage":"local","mode":"snapshot",
"selection":"vmid","covered_vmids":[100],"next_run":2000,"last_expected_run":1000},
{"id":"b","enabled":true,"schedule":"daily","storage":"local","mode":"snapshot",
"selection":"all","covered_vmids":[100,101],"next_run":5000,"last_expected_run":900},
{"id":"c","enabled":false,"schedule":"daily","storage":null,"mode":null,
"selection":"all","covered_vmids":[101],"next_run":null,"last_expected_run":null},
{"id":"d","enabled":true,"schedule":"sat 03:00","storage":null,"mode":null,
"selection":"vmid","covered_vmids":[101],"next_run":9000,"last_expected_run":null}],
"backup_runs":[
{"upid":"UPID:pve:1:2:3:vzdump::root@pam:","started_at":1001,"ended_at":1050,"status":"error",
"message":"job errors","guests":[{"vmid":100,"result":"ok"},{"vmid":101,"result":"error"}]},
{"upid":"UPID:pve:1:2:4:vzdump:100:root@pam:","started_at":100,"ended_at":120,"status":"ok",
"message":null,"guests":[{"vmid":100,"result":"ok"}]}],
"not_backed_up":[]})
}
#[test]
fn accepts_agent_shape_and_rejects_unsafe_values() {
assert!(validate(&sample()).is_ok());
for (pointer, bad) in [
("/guests/0/macs/0", json!("BC:24:11:88:35:97")),
("/guests/0/kind", json!("docker")),
("/guests/0/vmid", json!(5)),
("/guests/0/smbios_uuid", json!("not-a-uuid")),
("/backup_runs/0/status", json!("running")),
("/backup_jobs/0/selection", json!("tag")),
("/node", json!("pve\nX")),
] {
let mut doc = sample();
*doc.pointer_mut(pointer).unwrap() = bad;
assert!(validate(&doc).is_err(), "{pointer}");
}
let mut doc = sample();
doc["extra"] = json!(1);
assert!(validate(&doc).is_err());
}
#[test]
fn evaluates_due_runs_per_job_and_guest() {
let status = backup_status(&sample()).unwrap();
let jobs: BTreeMap<String, Value> = status["jobs"].as_array().unwrap().iter()
.map(|j| (j["id"].as_str().unwrap().to_owned(), j.clone())).collect();
assert_eq!(jobs["a"]["status"], "ok");
assert_eq!(jobs["b"]["status"], "failed");
assert_eq!(jobs["b"]["guests"][1]["status"], "failed");
assert_eq!(jobs["c"]["status"], "disabled");
assert_eq!(jobs["d"]["status"], "pending");
assert_eq!(status["all_scheduled_backups_ran"], false);
assert_eq!(status["last_success_at"]["100"], 1050);
}
/// Observation reported by the Proxmox VE 9.2 test host on 2026-11-20.
#[test]
fn real_proxmox_92_report_validates_and_counts_the_due_run() {
let data: Value = serde_json::from_str(include_str!("proxmox_pve92_fixture.json")).unwrap();
validate(&data).unwrap();
let status = backup_status(&data).unwrap();
assert_eq!(status["jobs"][0]["id"], "backup-debian");
assert_eq!(status["jobs"][0]["status"], "ok");
assert_eq!(status["all_scheduled_backups_ran"], true);
assert_eq!(data["not_backed_up"], json!([101]));
}
/// Same host at 14:00: the scheduled run failed because its storage was disabled.
#[test]
fn a_run_failing_before_any_guest_is_failed_not_missed() {
let data: Value = serde_json::from_str(include_str!("proxmox_pve92_failed_fixture.json")).unwrap();
validate(&data).unwrap();
let status = backup_status(&data).unwrap();
assert_eq!(status["jobs"][0]["status"], "failed");
assert_eq!(status["jobs"][0]["guests"][0]["status"], "failed");
assert_eq!(status["all_scheduled_backups_ran"], false);
// The earlier successful backup stays visible as the last success.
assert!(status["last_success_at"]["100"].is_number());
// A failed single-guest task of another guest does not fail this job.
let mut other = data.clone();
other["backup_runs"][0]["upid"] = json!("UPID:pve:00006CA0:000B1402:6B004453:vzdump:101:root@pam:");
assert_eq!(backup_status(&other).unwrap()["jobs"][0]["status"], "missed");
}
#[test]
fn missing_and_running_runs_are_distinguished() {
let mut doc = sample();
doc["backup_runs"] = json!([]);
let status = backup_status(&doc).unwrap();
assert_eq!(status["jobs"][0]["status"], "missed");
doc["backup_runs"] = json!([{"upid":"UPID:pve:9:9:9:vzdump::root@pam:","started_at":1000,
"ended_at":null,"status":"running","message":null,"guests":[]}]);
assert_eq!(backup_status(&doc).unwrap()["jobs"][0]["status"], "running");
doc["backup_jobs"] = json!([]);
assert!(backup_status(&doc).unwrap()["all_scheduled_backups_ran"].is_null());
}
}
@@ -0,0 +1,80 @@
{
"backup_jobs": [
{
"covered_vmids": [
100
],
"enabled": true,
"id": "backup-debian",
"last_expected_run": 1795179600,
"mode": "snapshot",
"next_run": 1795186800,
"schedule": "*/2:00",
"selection": "vmid",
"storage": "local"
}
],
"backup_runs": [
{
"ended_at": 1795179610,
"guests": [],
"message": "could not activate storage 'local': storage 'local' is disabled",
"started_at": 1795179603,
"status": "error",
"upid": "UPID:pve:00006CA0:000B1402:6B004453:vzdump:100:root@pam:"
},
{
"ended_at": 1795172488,
"guests": [
{
"result": "ok",
"vmid": 100
}
],
"message": null,
"started_at": 1795172468,
"status": "ok",
"upid": "UPID:pve:00000C75:000030ED:6B002874:vzdump:100:root@pam:"
}
],
"cluster": null,
"guests": [
{
"cpus": 1,
"disk_bytes": 8589934592,
"kind": "qemu",
"macs": [
"bc:24:11:88:35:97"
],
"memory_bytes": 1073741824,
"name": "debian-guest",
"pool": null,
"smbios_uuid": "787f4ae9-446f-4fac-b3d8-4074170bf823",
"status": "running",
"template": false,
"uptime_seconds": 7166,
"vmid": 100
},
{
"cpus": 1,
"disk_bytes": 1020702720,
"kind": "lxc",
"macs": [
"bc:24:11:b7:4c:99"
],
"memory_bytes": 134217728,
"name": "alpine-ct",
"pool": null,
"smbios_uuid": null,
"status": "running",
"template": false,
"uptime_seconds": 7164,
"vmid": 101
}
],
"node": "pve",
"not_backed_up": [
101
],
"version": "9.2.2"
}
@@ -0,0 +1,72 @@
{
"backup_jobs": [
{
"covered_vmids": [
100
],
"enabled": true,
"id": "backup-debian",
"last_expected_run": 1795172400,
"mode": "snapshot",
"next_run": 1795179600,
"schedule": "*/2:00",
"selection": "vmid",
"storage": "local"
}
],
"backup_runs": [
{
"ended_at": 1795172488,
"guests": [
{
"result": "ok",
"vmid": 100
}
],
"message": null,
"started_at": 1795172468,
"status": "ok",
"upid": "UPID:pve:00000C75:000030ED:6B002874:vzdump:100:root@pam:"
}
],
"cluster": null,
"guests": [
{
"cpus": 1,
"disk_bytes": 8589934592,
"kind": "qemu",
"macs": [
"bc:24:11:88:35:97"
],
"memory_bytes": 1073741824,
"name": "debian-guest",
"pool": null,
"smbios_uuid": "787f4ae9-446f-4fac-b3d8-4074170bf823",
"status": "running",
"template": false,
"uptime_seconds": 2716,
"vmid": 100
},
{
"cpus": 1,
"disk_bytes": 1020702720,
"kind": "lxc",
"macs": [
"bc:24:11:b7:4c:99"
],
"memory_bytes": 134217728,
"name": "alpine-ct",
"pool": null,
"smbios_uuid": null,
"status": "running",
"template": false,
"uptime_seconds": 2714,
"vmid": 101
}
],
"node": "pve",
"not_backed_up": [
101
],
"version": "9.2.2"
}
+55 -1
View File
@@ -480,7 +480,25 @@ impl Action {
_ => None,
}
}
/// Shared plan/revalidation/start gate. Passive Windows never implies job support.
/// The first FreeBSD profile is one exact POSIX script. It does not make
/// an appliance eligible for the Linux library, arbitrary scripts or root.
pub fn opnsense_kernel_probe(&self) -> bool {
matches!(self, Self::RunScript { interpreter, script_utf8, arguments,
expected_stdout: Some(expected), windows_execution_profile: None, privileged_script_profile: None }
if interpreter == "posix_sh" && script_utf8 == "uname -s\n"
&& arguments.is_empty() && expected == "FreeBSD\n")
}
pub fn opnsense_root_script(&self) -> bool {
matches!(self, Self::RunScript {interpreter, windows_execution_profile:None,
privileged_script_profile:Some(PrivilegedScriptProfile::FullMaintenanceV1), ..}
if interpreter=="posix_sh") && self.validate()
}
pub fn opnsense_service_script(&self) -> bool {
matches!(self, Self::RunScript {interpreter,windows_execution_profile:None,privileged_script_profile:None,..}
if interpreter=="posix_sh") && self.validate()
}
/// Shared plan/revalidation/start gate. Passive inventory never implies job support.
pub fn available_for(&self, identity: &str, capabilities: &[String]) -> bool {
if !self.valid_execution_identity(identity)
|| (identity == "system"
@@ -490,6 +508,13 @@ impl Action {
return false;
}
let has = |cap: &str| capabilities.iter().any(|c| c == cap);
if has("opnsense.base/1") {
return has("run_script") && has("rmm.sync.v1") && !has("windows.base/1") && !has("linux.base/1")
&& ((identity=="agent_service" && ((has("opnsense.uname/1") && self.opnsense_kernel_probe())
|| (has("opnsense.posix/1") && self.opnsense_service_script())))
|| (identity=="root" && has("opnsense.root.posix/1") && has(FULL_MAINTENANCE_CAPABILITY)
&& has("helper.root.v1") && has("root:run_script") && self.opnsense_root_script()));
}
if !has(self.name())
|| (self.full_maintenance()
&& !((has(FULL_MAINTENANCE_CAPABILITY)
@@ -828,6 +853,35 @@ impl JobState {
mod tests {
use super::*;
#[test]
fn appliance_kernel_probe_rejects_unbound_or_changed_profiles() {
let caps: Vec<String> = ["run_script","opnsense.base/1","rmm.sync.v1","opnsense.uname/1"].map(String::from).into();
let mut a = Action::RunScript { interpreter:"posix_sh".into(),script_utf8:"uname -s\n".into(),
arguments:vec![],expected_stdout:Some("FreeBSD\n".into()),windows_execution_profile:None,privileged_script_profile:None };
assert!(a.available_for("agent_service",&caps));
for excluded in &caps {
let fewer = caps.iter().filter(|v| *v!=excluded).cloned().collect::<Vec<_>>();
assert!(!a.available_for("agent_service",&fewer));
}
assert!(!a.available_for("root",&caps));
let mut mixed = caps.clone(); mixed.push("linux.base/1".into());
assert!(!a.available_for("agent_service",&mixed));
if let Action::RunScript { script_utf8,.. } = &mut a { script_utf8.push(' '); }
assert!(!a.available_for("agent_service",&caps));
}
#[test]
fn opnsense_root_requires_full_profile_and_separate_capabilities() {
let caps:Vec<_>=["opnsense.base/1","run_script","rmm.sync.v1","opnsense.root.posix/1",
FULL_MAINTENANCE_CAPABILITY,"helper.root.v1","root:run_script"].map(str::to_owned).into();
let mut a=Action::RunScript{interpreter:"posix_sh".into(),script_utf8:"id -u\n".into(),
arguments:vec![],expected_stdout:None,windows_execution_profile:None,
privileged_script_profile:Some(PrivilegedScriptProfile::FullMaintenanceV1)};
assert!(a.available_for("root",&caps));
for omitted in &caps { assert!(!a.available_for("root",&caps.iter().filter(|x|*x!=omitted).cloned().collect::<Vec<_>>())); }
assert!(!a.available_for("agent_service",&caps));
if let Action::RunScript{interpreter,..}=&mut a { *interpreter="bash".into(); }
assert!(!a.available_for("root",&caps));
}
#[test]
fn windows_update_final_report_and_policy_are_exact() {
assert_eq!(
windows_update_reboot_report(
+1
View File
@@ -3,6 +3,7 @@ pub mod agent_update;
pub mod agent_update_provisioning;
pub mod apt_transaction;
pub mod checkpoint_repair;
pub mod customer_transfer;
pub mod inventory_document;
pub mod jobs;
pub mod reconciliation;
@@ -110,7 +110,7 @@ pub fn observe_record(
&& response.ledger_inode != 0
&& response.lock_inode != 0
&& response.observation.validate()
&& response.observation.platform == Platform::LinuxX64,
&& response.observation.platform == Platform::NATIVE_LINUX,
"UPDATE_OBSERVER_RESPONSE_BINDING"
);
let record = rmm_domain::agent_update_provisioning::InstalledReleaseRecord::from_bytes(
@@ -346,7 +346,7 @@ pub fn collect(db: &Connection, local: Local<'_>, request: &Request) -> Result<R
));
let observation = AgentUpdateObservation {
schema_version: 1,
platform: Platform::LinuxX64,
platform: Platform::NATIVE_LINUX,
observed_at,
quality: ObservationQuality::Valid,
predecessor: Some(trust.predecessor().clone()),
+12
View File
@@ -81,6 +81,17 @@ pub struct Heartbeat {
pub instance_id: String,
#[prost(uint64, tag = "2")]
pub uptime_seconds: u64,
#[prost(message, optional, tag = "3")]
pub system_usage: Option<SystemUsage>,
}
#[derive(Clone, PartialEq, Message)]
pub struct SystemUsage {
#[prost(uint32, optional, tag = "1")]
pub cpu_usage_basis_points: Option<u32>,
#[prost(uint64, optional, tag = "2")]
pub ram_available_bytes: Option<u64>,
#[prost(uint64, optional, tag = "3")]
pub ram_total_bytes: Option<u64>,
}
#[derive(Clone, PartialEq, Message)]
pub struct InventorySnapshot {
@@ -156,6 +167,7 @@ mod tests {
payload: Some(Payload::Heartbeat(Heartbeat {
instance_id: "x".into(),
uptime_seconds: 1,
system_usage: None,
})),
..Default::default()
};
+70 -3
View File
@@ -4,6 +4,7 @@ use chrono::{DateTime, Utc};
use prost::Message;
use rmm_domain::MAX_MESSAGE;
use rmm_protocol::*;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
use sqlx::Row;
use uuid::Uuid;
@@ -104,7 +105,7 @@ pub async fn ingest(
);
let mut tx = rmm_persistence::tenant_tx(&state.pool, tenant).await?;
control_lock(&mut tx, tenant, agent).await?;
let row=sqlx::query("SELECT revoked_at,connection_generation,transport,connection_boot_id,last_sequence,capabilities FROM agent_installations WHERE agent_id=$1 FOR UPDATE").bind(agent).fetch_one(&mut *tx).await?;
let row=sqlx::query("SELECT revoked_at,connection_generation,transport,connection_boot_id,last_sequence,capabilities,customer_transfer_pending FROM agent_installations WHERE agent_id=$1 FOR UPDATE").bind(agent).fetch_one(&mut *tx).await?;
let active = row.get::<Option<DateTime<Utc>>, _>("revoked_at").is_none()
&& row.get::<i64, _>("connection_generation") == i64::try_from(e.connection_generation)?
&& row.get::<String, _>("transport") == transport;
@@ -146,13 +147,20 @@ pub async fn ingest(
match &e.payload {
Some(Payload::Heartbeat(h)) => {
Uuid::parse_str(&h.instance_id)?;
sqlx::query("UPDATE agent_installations SET last_contact_at=now() WHERE agent_id=$1 AND revoked_at IS NULL").bind(agent).execute(&mut *tx).await?;
let usage = h.system_usage.as_ref().and_then(|sample| system_usage(sample, observed));
sqlx::query("UPDATE agent_installations SET last_contact_at=now(),
system_usage=CASE WHEN $2::jsonb IS NULL THEN system_usage
ELSE COALESCE(system_usage,'{}'::jsonb) || $2 END,
system_usage_observed_at=CASE WHEN $2::jsonb ? 'ram_available_bytes'
THEN $3 ELSE system_usage_observed_at END
WHERE agent_id=$1 AND revoked_at IS NULL")
.bind(agent).bind(usage).bind(observed).execute(&mut *tx).await?;
}
Some(Payload::InventorySnapshot(s)) => {
ensure!(
matches!(
s.scope.as_str(),
"linux.base" | "windows.base" | "agent.update"
"linux.base" | "windows.base" | "opnsense.base" | "agent.update"
) && s.schema_version == 1
&& s.chunk_count == 1
&& s.chunk_index == 0
@@ -182,12 +190,25 @@ pub async fn ingest(
.fetch_one(&mut *tx)
.await?;
ensure!(stored == hash, "message identity conflict");
if matches!(s.scope.as_str(), "linux.base" | "windows.base") {
crate::proxmox_links::reconcile(&mut tx, tenant).await;
}
}
Some(Payload::JobAccepted(_) | Payload::JobStateChanged(_) | Payload::JobResult(_)) => {
crate::job_gateway::event(&mut tx, tenant, agent, e).await?
}
_ => anyhow::bail!("unsupported capability"),
}
if row.get::<bool, _>("customer_transfer_pending")
&& (matches!(&e.payload, Some(Payload::Heartbeat(_)))
|| matches!(&e.payload, Some(Payload::InventorySnapshot(s)) if s.scope != "agent.update"))
{
sqlx::query("SELECT finish_device_customer_transfer($1,$2)")
.bind(tenant)
.bind(agent)
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
Ok(reply(
e,
@@ -197,3 +218,49 @@ pub async fn ingest(
e.connection_generation,
))
}
fn system_usage(sample: &SystemUsage, observed: DateTime<Utc>) -> Option<Value> {
let cpu = sample
.cpu_usage_basis_points
.filter(|&value| value <= 10_000);
let memory = sample
.ram_available_bytes
.zip(sample.ram_total_bytes)
.filter(|&(available, total)| total > 0 && available <= total);
let mut usage = serde_json::Map::new();
if let Some(cpu) = cpu {
usage.insert("cpu_usage_percent".into(), json!(f64::from(cpu) / 100.0));
usage.insert("cpu_usage_observed_at".into(), json!(observed));
}
if let Some((available, total)) = memory {
usage.insert("ram_available_bytes".into(), json!(available));
usage.insert("ram_total_bytes".into(), json!(total));
}
(!usage.is_empty()).then_some(Value::Object(usage))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn accepts_zero_usage_and_ignores_invalid_metrics_without_losing_the_heartbeat() {
let mut sample = SystemUsage {
cpu_usage_basis_points: Some(0),
ram_available_bytes: Some(0),
ram_total_bytes: Some(1),
};
let now = Utc::now();
let measured = system_usage(&sample, now).unwrap();
assert_eq!(measured["cpu_usage_percent"], json!(0.0));
assert_eq!(measured["cpu_usage_observed_at"], json!(now));
assert_eq!(measured["ram_available_bytes"], json!(0));
assert_eq!(measured["ram_total_bytes"], json!(1));
sample.cpu_usage_basis_points = Some(10_001);
let partial = system_usage(&sample, now).unwrap();
assert!(partial.get("cpu_usage_percent").is_none());
assert!(partial.get("cpu_usage_observed_at").is_none());
sample.ram_available_bytes = Some(2);
assert_eq!(system_usage(&sample, now), None);
}
}
+1 -1
View File
@@ -418,7 +418,7 @@ async fn issue(
&& snapshot.sha256 == hex::decode(hash(&snapshot.data)).map_err(|_| denied())?,
)?;
let scope = match action.manifest().map_err(|_| denied())?.platform {
update::Platform::LinuxX64 => "linux.base",
update::Platform::LinuxX64 | update::Platform::LinuxArm64 => "linux.base",
update::Platform::WindowsX64 | update::Platform::WindowsGnuX64 => "windows.base",
};
// Health requires a fresh real base inventory of the restarted platform;
+27 -28
View File
@@ -17,7 +17,7 @@ use axum::{
use chrono::{DateTime, Utc};
use futures_util::StreamExt;
use rmm_domain::{
agent_update::{ArtifactDescriptor, Platform, ReleaseEnvelope, ReleaseManifest},
agent_update::{ArtifactDescriptor, LINUX_ARM64_RELEASE_CAPABILITY, Platform, ReleaseEnvelope, ReleaseManifest},
jobs::{ArtifactReference, encode, hash},
};
use serde::Deserialize;
@@ -42,11 +42,7 @@ fn invalid() -> Error {
Error(StatusCode::BAD_REQUEST, "INVALID_AGENT_RELEASE")
}
pub fn platform_name(platform: Platform) -> &'static str {
match platform {
Platform::LinuxX64 => "x86_64-unknown-linux-gnu",
Platform::WindowsX64 => "x86_64-pc-windows-msvc",
Platform::WindowsGnuX64 => "x86_64-pc-windows-gnu",
}
platform.target_triple()
}
pub fn protected_file(path: &FsPath, maximum: u64) -> Result<Vec<u8>, Error> {
let key_path =
@@ -139,13 +135,6 @@ pub fn store() -> Result<UpdateArtifacts, Error> {
metadata.uid(),
)?)
}
fn verifier_platform(platform: Platform) -> rmm_updater::Platform {
match platform {
Platform::LinuxX64 => rmm_updater::Platform::LinuxX64,
Platform::WindowsX64 => rmm_updater::Platform::WindowsX64,
Platform::WindowsGnuX64 => rmm_updater::Platform::WindowsGnuX64,
}
}
/// Network input can choose only among statically named protected
/// configuration slots. It never supplies a path or introduces update trust.
fn keyring_path_with(
@@ -154,6 +143,7 @@ fn keyring_path_with(
) -> Result<std::path::PathBuf, Error> {
let name = match platform {
Platform::LinuxX64 => "RMM_AGENT_UPDATE_LINUX_KEYRING_FILE",
Platform::LinuxArm64 => "RMM_AGENT_UPDATE_LINUX_ARM64_KEYRING_FILE",
Platform::WindowsX64 => "RMM_AGENT_UPDATE_WINDOWS_KEYRING_FILE",
Platform::WindowsGnuX64 => {
// The shared device gateway and both management APIs must select the same
@@ -177,7 +167,7 @@ fn keyring_path_with(
}
// Exact legacy compatibility only when the platform-specific slot is
// absent. An invalid explicit slot must never silently choose other trust.
if platform == Platform::WindowsGnuX64 {
if matches!(platform, Platform::LinuxArm64 | Platform::WindowsGnuX64) {
return Err(fail("AGENT_UPDATE_NOT_CONFIGURED"));
}
lookup("RMM_AGENT_UPDATE_KEYRING_FILE")?
@@ -239,7 +229,7 @@ fn verify_envelope_with_keyring(
rmm_updater::TrustedUpdateKey::new(
raw.try_into()
.map_err(|_| fail("AGENT_UPDATE_TRUST_INVALID"))?,
verifier_platform(key.platform),
key.platform.into(),
key.first_sequence,
key.last_sequence,
key.revoked,
@@ -250,7 +240,7 @@ fn verify_envelope_with_keyring(
let keyring = rmm_updater::UpdateKeyring::new(config.generation, keys)
.map_err(|_| fail("AGENT_UPDATE_TRUST_INVALID"))?;
let context = rmm_updater::TrustedContext {
platform: verifier_platform(decoded.manifest.platform),
platform: decoded.manifest.platform.into(),
now_utc_seconds,
keyring_generation: config.generation,
protocol_major: 1,
@@ -275,6 +265,12 @@ fn verify_envelope_with_keyring(
}
Ok(decoded.manifest)
}
pub(crate) async fn ensure_customer_starters(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
) -> Result<(), Error> {
bootstrap::publish_starters(tx, tenant).await
}
pub fn routes(router: Router<crate::State>) -> Router<crate::State> {
router
.route(
@@ -612,7 +608,7 @@ async fn list(
pub struct Current {
platform: Platform,
}
async fn device(
pub(crate) async fn device(
s: &crate::State,
peer: crate::Peer,
headers: &HeaderMap,
@@ -658,18 +654,21 @@ async fn check_device_platform(
.fetch_one(&mut **tx)
.await?;
let caps: Vec<String> = serde_json::from_value(caps)?;
let expected = match platform {
Platform::LinuxX64 => "linux.base/1",
Platform::WindowsX64 if caps.iter().any(|c| c == "agent.release.windows.gnu/1") => {
return Err(missing());
}
Platform::WindowsX64 => "windows.base/1",
Platform::WindowsGnuX64 => "agent.release.windows.gnu/1",
};
if !caps.iter().any(|c| c == expected) {
return Err(missing());
if device_supports_release(platform, &caps) {
Ok(())
} else {
Err(missing())
}
}
fn device_supports_release(platform: Platform, caps: &[String]) -> bool {
let has = |expected: &str| caps.iter().any(|capability| capability == expected);
match platform {
Platform::LinuxX64 => has("linux.base/1") && !has(LINUX_ARM64_RELEASE_CAPABILITY),
Platform::LinuxArm64 => has("linux.base/1") && has(LINUX_ARM64_RELEASE_CAPABILITY),
Platform::WindowsX64 => has("windows.base/1") && !has("agent.release.windows.gnu/1"),
Platform::WindowsGnuX64 => has("agent.release.windows.gnu/1"),
}
Ok(())
}
/// A published release grants access to its two fixed binaries for authenticated
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/sh
# Usage: sudo sh linux.sh https://bootstrap.example [--enable-admin-scripts]
# Linux x86-64 / ARM64. Usage: sudo sh linux.sh https://bootstrap.example [--enable-admin-scripts]
set -eu
umask 077
file=$(mktemp)
+93 -7
View File
@@ -32,7 +32,6 @@ TIMER = 'rmm-discovery-update.timer'
HELPER = 'rmm-discovery-managed-script-helper.service'
HELPER_SOCKET = Path('/run/rmm-discovery-script-helper/control.sock')
OLD_HELPER_CONFIG = Path('/etc/rmm-discovery-managed-scripts')
PLATFORM = 'x86_64-unknown-linux-gnu'
MAX_BINARY = 256 * 1024 * 1024
UNIT_DIR = Path('/etc/systemd/system')
@@ -42,6 +41,13 @@ def require(condition, message):
raise ValueError(message)
def release_platform():
machine = platform.machine()
require(platform.system() == 'Linux' and machine in ('x86_64', 'aarch64'),
'Linux x86-64 or ARM64 required')
return machine + '-unknown-linux-gnu'
def origin(value):
parsed = urlsplit(value)
require(parsed.scheme == 'https' and parsed.hostname and parsed.username is None
@@ -151,7 +157,9 @@ def pair(folder, doc, source, updater):
manifest = doc['manifest']
release_id = canonical_id(manifest['release_id'])
prefix = '/agent/v1/bootstrap/releases' if source else '/agent/v1/releases'
suffix = '?platform=' + PLATFORM if source else ''
target = release_platform()
require(manifest['platform'] == target, 'release architecture differs from this device')
suffix = '?platform=' + target if source else ''
save(folder / 'envelope.json', doc['release_envelope'])
for artifact, role in zip(manifest['artifacts'], ('agent', 'helper'), strict=True):
require(artifact['role'] == role, 'incorrect artifact role')
@@ -186,7 +194,12 @@ def installed():
def healthy(version, baseline, duration=90, pin=None):
until = time.monotonic() + duration
started = time.time()
certificate_pointer = STATE / 'certificates/current.json'
while time.monotonic() < until:
# The agent writes this pointer after obtaining its first device certificate.
if not certificate_pointer.is_file():
time.sleep(2)
continue
try:
record = installed()
contact = record.get('last_contact_at')
@@ -457,6 +470,76 @@ WantedBy=timers.target
}
def provision_serial_credential():
# DMI serial files usually require root. Pass this one value to the agent
# through systemd's read-only per-service credentials directory.
source = Path('/sys/class/dmi/id/product_serial')
try:
with source.open('rb') as stream:
serial = stream.read(256).strip()
except OSError:
return
if not serial or len(serial) >= 255:
return
target = ROOT / 'bios-serial'
dropin = UNIT_DIR / (SERVICE + '.d') / '40-inventory-serial.conf'
content = f'[Service]\nLoadCredential=rmm-bios-serial:{target}\n'.encode()
require(not dropin.parent.is_symlink(), 'hardware credential directory conflict')
if dropin.exists() or dropin.is_symlink():
require(not dropin.is_symlink() and dropin.read_bytes() == content, 'hardware credential configuration conflict')
else:
dropin.parent.mkdir(mode=0o755, exist_ok=True)
save(target, serial, 0o600)
if not dropin.exists():
save(dropin, content)
def provision_proxmox_credential():
# Proxmox VE hosts: a dedicated read-only API token (PVEAuditor) lets the
# unprivileged agent list guests and backup jobs. The secret is only shown by
# Proxmox at creation, so an existing token without a local copy is rotated.
pveum = Path('/usr/sbin/pveum')
if not (Path('/usr/bin/pveversion').is_file() and pveum.is_file()):
return
user, name = 'rmm-discovery@pve', 'inventory'
target, ca = ROOT / 'proxmox-token', ROOT / 'pve-ca.pem'
dropin = UNIT_DIR / (SERVICE + '.d') / '41-proxmox-api.conf'
content = f'[Service]\nLoadCredential=rmm-proxmox-token:{target}\nEnvironment=RMM_PROXMOX_CA_FILE={ca}\n'.encode()
require(not dropin.parent.is_symlink(), 'proxmox credential directory conflict')
def pve(*args):
return subprocess.run([str(pveum), *args], check=True, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, timeout=60).stdout
try:
save(ca, Path('/etc/pve/pve-root-ca.pem').read_bytes(), 0o644)
current = target.read_text().strip() if target.is_file() else ''
if not current.startswith(f'{user}!{name}='):
users = json.loads(pve('user', 'list', '--output-format', 'json'))
if not any(u.get('userid') == user for u in users):
pve('user', 'add', user, '--comment', 'Rust Discovery inventory (read-only)')
tokens = json.loads(pve('user', 'token', 'list', user, '--output-format', 'json'))
if any(t.get('tokenid') == name for t in tokens):
pve('user', 'token', 'remove', user, name)
created = json.loads(pve('user', 'token', 'add', user, name, '--privsep', '1',
'--comment', 'Rust Discovery agent', '--output-format', 'json'))
secret = created['value']
require(re.fullmatch('[0-9a-f-]{36}', secret), 'unexpected proxmox token format')
save(target, f'{user}!{name}={secret}\n'.encode(), 0o600)
# A privilege-separated token holds the intersection of user and token rights.
pve('acl', 'modify', '/', '--users', user, '--roles', 'PVEAuditor')
pve('acl', 'modify', '/', '--tokens', f'{user}!{name}', '--roles', 'PVEAuditor')
except (OSError, ValueError, KeyError, subprocess.SubprocessError):
print('Proxmox VE read access could not be prepared; guest and backup inventory stays unavailable',
file=sys.stderr)
return
if dropin.exists() or dropin.is_symlink():
require(not dropin.is_symlink() and dropin.read_bytes() == content, 'proxmox credential configuration conflict')
else:
dropin.parent.mkdir(mode=0o755, exist_ok=True)
save(dropin, content)
def enable():
for name, data in units().items():
path = UNIT_DIR / name
@@ -464,6 +547,8 @@ def enable():
require(path.read_bytes() == data.encode(), 'service file conflict: ' + name)
else:
save(path, data.encode())
provision_serial_credential()
provision_proxmox_credential()
service('daemon-reload')
service('enable', '--now', SERVICE)
@@ -481,7 +566,7 @@ def save_token(base, token, gid):
def install(base, token_file, enable_admin_scripts=False):
require(platform.machine() == 'x86_64', 'Linux x86-64 required')
target = release_platform()
base = origin(base)
if ROOT.exists():
config = read(ROOT / 'config.json')
@@ -491,7 +576,7 @@ def install(base, token_file, enable_admin_scripts=False):
token = enrollment_token(token_file)
with tempfile.TemporaryDirectory() as work:
metadata = Path(work) / 'bootstrap.json'
remote((base, token), '/agent/v1/bootstrap/current?platform=' + PLATFORM, metadata)
remote((base, token), '/agent/v1/bootstrap/current?platform=' + target, metadata)
require(read(metadata)['tenant_id'] == config['tenant'], 'different enrollment tenant')
save_token(ROOT, token, pwd.getpwnam('rmm-discovery').pw_gid)
# An operator rerunning the HTTPS installer can update the lifecycle
@@ -524,7 +609,7 @@ def install(base, token_file, enable_admin_scripts=False):
with tempfile.TemporaryDirectory(prefix='.rmm-', dir=ROOT.parent) as work:
staging = Path(work) / 'installation'
staging.mkdir()
remote((base, token), '/agent/v1/bootstrap/current?platform=' + PLATFORM, staging / 'bootstrap.json')
remote((base, token), '/agent/v1/bootstrap/current?platform=' + target, staging / 'bootstrap.json')
doc = read(staging / 'bootstrap.json')
require(doc['schema_version'] == 1, 'unknown bootstrap profile')
save(staging / 'keyring.json', doc['keyring'])
@@ -532,7 +617,7 @@ def install(base, token_file, enable_admin_scripts=False):
save(staging / 'config.json', {'bootstrap': base, 'gateway': origin(doc['gateway_origin']),
'tenant': canonical_id(doc['tenant_id'])})
updater = staging / 'updater'
remote((base, token), '/agent/v1/bootstrap/updater?platform=' + PLATFORM + '&sha256=' + doc['updater']['sha256'], updater, doc['updater'])
remote((base, token), '/agent/v1/bootstrap/updater?platform=' + target + '&sha256=' + doc['updater']['sha256'], updater, doc['updater'])
updater.chmod(0o755)
releases = staging / 'releases'
releases.mkdir()
@@ -599,6 +684,7 @@ def revert(intent, report_failure=True):
def update():
target = release_platform()
with (ROOT / 'release.lock').open('rb') as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
pending = ROOT / 'pending.json'
@@ -608,7 +694,7 @@ def update():
with tempfile.TemporaryDirectory(prefix='.incoming-', dir=ROOT / 'releases') as work:
staging = Path(work) / 'generation'
staging.mkdir()
remote(None, '/agent/v1/releases/current?platform=' + PLATFORM, staging / 'current.json')
remote(None, '/agent/v1/releases/current?platform=' + target, staging / 'current.json')
response = read(staging / 'current.json')
if response['status'] == 'not_configured':
return
+3
View File
@@ -3,6 +3,8 @@
use super::*;
mod starter;
pub(super) use starter::publish_missing as publish_starters;
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Config {
@@ -31,6 +33,7 @@ fn official_windows(platform: Platform) -> bool {
async fn config(s: &crate::State, platform: Platform, tenant: Uuid, wanted: Option<&str>) -> Result<Config, Error> {
let slot = match platform {
Platform::LinuxX64 => "RMM_AGENT_BOOTSTRAP_LINUX_FILE",
Platform::LinuxArm64 => "RMM_AGENT_BOOTSTRAP_LINUX_ARM64_FILE",
Platform::WindowsX64 => "RMM_AGENT_BOOTSTRAP_WINDOWS_FILE",
Platform::WindowsGnuX64 => "RMM_AGENT_BOOTSTRAP_WINDOWS_GNU_FILE",
};
+5 -2
View File
@@ -10,11 +10,13 @@ struct Starter {
worker: Pin,
}
pub(super) async fn publish_missing(
pub(in crate::agent_updates) async fn publish_missing(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
) -> Result<(), Error> {
let Some(slot) = std::env::var_os("RMM_AGENT_BOOTSTRAP_LINUX_FILE") else {
let Some(slot) = std::env::var_os("RMM_AGENT_BOOTSTRAP_LINUX_FILE")
.or_else(|| std::env::var_os("RMM_AGENT_BOOTSTRAP_LINUX_ARM64_FILE"))
else {
return Ok(());
};
let Some(config_dir) = FsPath::new(&slot).parent() else {
@@ -29,6 +31,7 @@ pub(super) async fn publish_missing(
}
for platform in [
Platform::LinuxX64,
Platform::LinuxArm64,
Platform::WindowsGnuX64,
Platform::WindowsX64,
] {
+128 -1
View File
@@ -3,7 +3,7 @@ use axum::{
Json, Router,
extract::{Path, Query, State},
http::{HeaderMap, StatusCode},
routing::{delete, post},
routing::{delete, patch, post},
};
use serde::Deserialize;
use serde_json::{Value, json};
@@ -26,10 +26,18 @@ pub fn routes(r: Router<crate::State>) -> Router<crate::State> {
post(revoke_device),
)
.route("/api/v1/tenants/{tenant}/devices/{asset}", delete(remove_device))
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/hypervisor",
post(assign_hypervisor),
)
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/archive",
post(archive_device),
)
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/purpose",
patch(update_device_purpose),
)
.route(
"/api/v1/tenants/{tenant}/operation-plans/{plan}/revocation",
post(revoke_plan),
@@ -128,12 +136,131 @@ async fn archive_device(
tx.commit().await?;
Ok(Json(json!({"asset_id":asset, "archived":input.archived})))
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Purpose {
purpose: String,
}
fn normalize_purpose(raw: &str) -> Result<Option<String>, Error> {
let normalized = raw.replace("\r\n", "\n");
let text = normalized.trim();
if text.chars().count() > 500
|| normalized
.chars()
.any(|c| c.is_control() && !matches!(c, '\n' | '\t'))
{
return Err(Error(StatusCode::BAD_REQUEST, "VALIDATION_ERROR"));
}
Ok((!text.is_empty()).then(|| text.to_owned()))
}
async fn update_device_purpose(
State(s): State<crate::State>,
h: HeaderMap,
Path((tenant, asset)): Path<(Uuid, Uuid)>,
Json(input): Json<Purpose>,
) -> Result<Json<Value>, Error> {
const EVENT: &str = "device_purpose_updated";
let (p, mut tx) = context(&s, &h, tenant, "devices:revoke", EVENT).await?;
crate::permissions::require_principal(&mut tx, &p, "inventory:read").await?;
let purpose = normalize_purpose(&input.purpose)?;
let current: Option<String> = sqlx::query_scalar(
"SELECT purpose FROM assets WHERE tenant_id=$1 AND asset_id=$2
AND EXISTS (SELECT 1 FROM agent_installations a WHERE a.tenant_id=$1 AND a.asset_id=$2
AND NOT a.customer_transfer_pending)
FOR UPDATE",
)
.bind(tenant)
.bind(asset)
.fetch_optional(&mut *tx)
.await?
.ok_or(Error(StatusCode::NOT_FOUND, "NOT_FOUND"))?;
if current != purpose {
sqlx::query("UPDATE assets SET purpose=$3 WHERE tenant_id=$1 AND asset_id=$2")
.bind(tenant)
.bind(asset)
.bind(&purpose)
.execute(&mut *tx)
.await?;
audit(
&mut tx,
&p,
EVENT,
json!({"asset_id":asset,"cleared":purpose.is_none()}),
)
.await?;
}
tx.commit().await?;
Ok(Json(json!({"asset_id":asset,"purpose":purpose})))
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Pairing {
agent_id: Uuid,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct HypervisorAssignment {
// null clears the relation. A missing field is a malformed update.
#[serde(deserialize_with = "required_hypervisor_value")]
hypervisor_asset_id: Option<Uuid>,
/// true hands the decision back to automatic Proxmox detection (requires null id).
#[serde(default)]
automatic: bool,
}
fn required_hypervisor_value<'de, D: serde::Deserializer<'de>>(value: D) -> Result<Option<Uuid>, D::Error> {
Option::deserialize(value)
}
async fn assign_hypervisor(
State(s): State<crate::State>,
h: HeaderMap,
Path((tenant, asset)): Path<(Uuid, Uuid)>,
Query(pair): Query<Pairing>,
Json(input): Json<HypervisorAssignment>,
) -> Result<Json<Value>, Error> {
const EVENT: &str = "device_hypervisor_assigned";
let (p, mut tx) = context(&s, &h, tenant, "devices:revoke", EVENT).await?;
crate::permissions::require_principal(&mut tx, &p, "inventory:read").await?;
// Match the asset lock order for simultaneous edits of the same two devices.
let ids = [asset, input.hypervisor_asset_id.unwrap_or(asset)];
if input.automatic && input.hypervisor_asset_id.is_some() {
return Err(Error(StatusCode::UNPROCESSABLE_ENTITY, "AUTOMATIC_REQUIRES_NULL_HYPERVISOR"));
}
let rows: Vec<(Uuid, Option<Uuid>, Option<String>)> = sqlx::query_as(
"SELECT asset_id,hypervisor_asset_id,hypervisor_source FROM assets
WHERE tenant_id=$1 AND asset_id=ANY($2) ORDER BY asset_id FOR UPDATE",
)
.bind(tenant).bind(ids.as_slice()).fetch_all(&mut *tx).await?;
let (_, previous, previous_source) = rows.into_iter().find(|(id, _, _)| *id == asset)
.ok_or(Error(StatusCode::NOT_FOUND, "NOT_FOUND"))?;
crate::devices::validate_vm_host_assignment(
&mut tx, tenant, asset, pair.agent_id, input.hypervisor_asset_id,
).await?;
let source = (!input.automatic).then_some("manual");
if previous != input.hypervisor_asset_id || previous_source.as_deref() != source {
sqlx::query("UPDATE assets SET hypervisor_asset_id=$3,hypervisor_source=$4 WHERE tenant_id=$1 AND asset_id=$2")
.bind(tenant).bind(asset).bind(input.hypervisor_asset_id).bind(source).execute(&mut *tx).await?;
audit(&mut tx, &p, EVENT, json!({"asset_id":asset,"agent_id":pair.agent_id,
"previous_hypervisor_asset_id":previous,"hypervisor_asset_id":input.hypervisor_asset_id,
"previous_source":previous_source,"source":source.unwrap_or("automatic")})).await?;
}
if input.automatic {
crate::proxmox_links::reconcile(&mut tx, tenant).await;
}
let (current, current_source): (Option<Uuid>, Option<String>) = sqlx::query_as(
"SELECT hypervisor_asset_id,hypervisor_source FROM assets WHERE tenant_id=$1 AND asset_id=$2")
.bind(tenant).bind(asset).fetch_one(&mut *tx).await?;
tx.commit().await?;
Ok(Json(json!({"asset_id":asset,"hypervisor_asset_id":current,"hypervisor_source":current_source})))
}
async fn remove_device(
State(s): State<crate::State>,
h: HeaderMap,
+461
View File
@@ -0,0 +1,461 @@
use crate::{auth::Principal, jobs::Error};
use axum::{
Extension, Json, Router,
extract::{Path, Query, State},
http::{HeaderMap, StatusCode},
response::{IntoResponse, Response},
routing::{get, post},
};
use chrono::{DateTime, Utc};
use rmm_domain::{customer_transfer as wire, jobs::hash};
use serde::Deserialize;
use serde_json::{Value, json};
use sqlx::{Postgres, Row, Transaction};
use uuid::Uuid;
const NOT_FOUND: Error = Error(StatusCode::NOT_FOUND, "NOT_FOUND");
fn private_json(body: Value) -> Response {
([("cache-control", "private, no-store")], Json(body)).into_response()
}
pub(crate) fn gateway_routes(router: Router<crate::State>) -> Router<crate::State> {
router
.route("/agent/v1/customer-transfer", get(device_offer))
.route(
"/agent/v1/customer-transfer/{transfer}/certificate",
post(device_certificate),
)
}
pub(crate) fn routes(router: Router<crate::State>) -> Router<crate::State> {
router
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/customer-transfer",
get(status).post(begin),
)
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/customer-transfer/{transfer}/complete",
post(complete),
)
.route(
"/api/v1/tenants/{tenant}/devices/{asset}/customer-transfer/{transfer}/cancel",
post(cancel),
)
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Pairing {
agent_id: Uuid,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct Destination {
target_tenant_id: Uuid,
}
struct Transfer {
transfer_id: Uuid,
target_tenant_id: Uuid,
target_asset_id: Option<Uuid>,
target_agent_id: Option<Uuid>,
csr_sha256: Option<String>,
requested_by: String,
completed_at: Option<DateTime<Utc>>,
}
async fn source<'a>(
s: &'a crate::State,
headers: &HeaderMap,
tenant: Uuid,
write: bool,
) -> Result<(Principal, Transaction<'a, Postgres>), Error> {
let p = crate::jobs::principal(s, headers, write).await?;
crate::jobs::scope(&p, tenant, write)?;
if p.session_hash.is_none() {
return Err(Error(StatusCode::FORBIDDEN, "INTERACTIVE_AUTH_REQUIRED"));
}
let mut tx = rmm_persistence::tenant_tx(&s.pool, tenant).await?;
crate::permissions::require_principal(&mut tx, &p, "inventory:read").await?;
crate::permissions::require_principal(&mut tx, &p, "devices:revoke").await?;
Ok((p, tx))
}
async fn select_tenant(tx: &mut Transaction<'_, Postgres>, tenant: Uuid) -> Result<(), Error> {
sqlx::query("SELECT set_config('app.tenant_id',$1,true)")
.bind(tenant.to_string())
.execute(&mut **tx)
.await?;
Ok(())
}
async fn destination(
tx: &mut Transaction<'_, Postgres>,
p: &Principal,
target: Uuid,
) -> Result<(), Error> {
if target == p.tenant || !p.tenants.contains(&target) {
return Err(NOT_FOUND);
}
select_tenant(tx, target).await?;
crate::permissions::require_principal(tx, p, "inventory:read").await?;
crate::permissions::require_principal(tx, p, "devices:enroll").await
}
async fn transfer(
tx: &mut Transaction<'_, Postgres>,
agent: Uuid,
) -> Result<Option<Transfer>, Error> {
let row = sqlx::query(
"SELECT transfer_id,target_tenant_id,target_asset_id,target_agent_id,csr_sha256,requested_by,completed_at
FROM device_customer_transfers WHERE source_agent_id=$1 AND cancelled_at IS NULL",
)
.bind(agent)
.fetch_optional(&mut **tx)
.await?;
Ok(row.map(|row| Transfer {
transfer_id: row.get("transfer_id"),
target_tenant_id: row.get("target_tenant_id"),
target_asset_id: row.get("target_asset_id"),
target_agent_id: row.get("target_agent_id"),
csr_sha256: row.get("csr_sha256"),
requested_by: row.get("requested_by"),
completed_at: row.get("completed_at"),
}))
}
async fn target_hostname(
tx: &mut Transaction<'_, Postgres>,
t: &Transfer,
) -> Result<Option<String>, Error> {
let (Some(asset), Some(agent)) = (t.target_asset_id, t.target_agent_id) else {
return Ok(None);
};
Ok(sqlx::query_scalar(
"SELECT o.data->>'hostname' FROM agent_installations a
JOIN LATERAL (
SELECT data FROM observations WHERE tenant_id=a.tenant_id AND agent_id=a.agent_id AND source_id IN ('linux.base','windows.base')
ORDER BY observed_at DESC, received_at DESC, sequence DESC LIMIT 1
) o ON true
WHERE a.asset_id=$1 AND a.agent_id=$2 AND a.revoked_at IS NULL
AND a.last_authenticated_at IS NOT NULL AND length(o.data->>'hostname')>0",
)
.bind(asset).bind(agent)
.fetch_optional(&mut **tx).await?)
}
async fn status(
State(s): State<crate::State>,
headers: HeaderMap,
Path((tenant, asset)): Path<(Uuid, Uuid)>,
Query(pair): Query<Pairing>,
) -> Result<Response, Error> {
let (p, mut tx) = source(&s, &headers, tenant, false).await?;
verify_pairing(&mut tx, asset, pair.agent_id).await?;
let Some(t) = transfer(&mut tx, pair.agent_id).await? else {
return Ok(private_json(json!({"transfer":null})));
};
destination(&mut tx, &p, t.target_tenant_id).await?;
let hostname = if t.completed_at.is_some() {
target_hostname(&mut tx, &t).await?
} else {
None
};
let state = if t.completed_at.is_some() {
"completed"
} else if t.target_agent_id.is_some() {
"enrolled"
} else {
"pending"
};
tx.commit().await?;
Ok(private_json(json!({"transfer":{
"transfer_id":t.transfer_id, "state":state,
"target_tenant_id":t.target_tenant_id, "target_asset_id":t.target_asset_id,
"target_agent_id":t.target_agent_id, "target_hostname":hostname
}})))
}
async fn lock_source(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
asset: Uuid,
agent: Uuid,
) -> Result<bool, Error> {
crate::agent_service::control_lock(tx, tenant, agent).await?;
sqlx::query_scalar(
"SELECT i.revoked_at IS NULL AND i.last_authenticated_at IS NOT NULL AND a.archived_at IS NULL
FROM agent_installations i JOIN assets a USING (tenant_id,asset_id)
WHERE i.asset_id=$1 AND i.agent_id=$2 AND NOT i.customer_transfer_pending FOR UPDATE OF i",
)
.bind(asset)
.bind(agent)
.fetch_optional(&mut **tx)
.await?
.ok_or(NOT_FOUND)
}
async fn ensure_idle(tx: &mut Transaction<'_, Postgres>, agent: Uuid) -> Result<(), Error> {
let busy: bool = sqlx::query_scalar(
"SELECT EXISTS(SELECT FROM jobs WHERE agent_id=$1
AND state NOT IN ('succeeded','failed','cancelled','expired','closed_unknown'))
OR EXISTS(SELECT FROM approval_requests r JOIN operation_plans p USING(tenant_id,plan_id)
WHERE r.approval_state='pending_approval' AND p.expires_at>now()
AND convert_from(p.payload,'UTF8')::jsonb->'jobs'
@> jsonb_build_array(jsonb_build_object('payload',jsonb_build_object('agent_id',$1::uuid))))",
)
.bind(agent)
.fetch_one(&mut **tx)
.await?;
if busy {
return Err(Error(StatusCode::CONFLICT, "DEVICE_HAS_OPEN_WORK"));
}
Ok(())
}
async fn audit(
tx: &mut Transaction<'_, Postgres>,
p: &Principal,
event: &str,
details: Value,
) -> Result<(), Error> {
sqlx::query("INSERT INTO audit_events VALUES($1,$2,$3,$4,$5,now())")
.bind(p.tenant).bind(Uuid::new_v4()).bind(&p.name).bind(event)
.bind(json!({"decision":details,"session_hash":p.session_hash,"approval_origin":p.approval_origin(),"testsimulation":p.testsimulation}))
.execute(&mut **tx).await?;
Ok(())
}
async fn begin(
State(s): State<crate::State>,
headers: HeaderMap,
Path((tenant, asset)): Path<(Uuid, Uuid)>,
Query(pair): Query<Pairing>,
Json(input): Json<Destination>,
) -> Result<Response, Error> {
let (p, mut tx) = source(&s, &headers, tenant, true).await?;
destination(&mut tx, &p, input.target_tenant_id).await?;
crate::agent_updates::ensure_customer_starters(&mut tx, input.target_tenant_id).await?;
select_tenant(&mut tx, tenant).await?;
if !lock_source(&mut tx, tenant, asset, pair.agent_id).await? {
return Err(Error(StatusCode::CONFLICT, "TRANSFER_SOURCE_INACTIVE"));
}
ensure_idle(&mut tx, pair.agent_id).await?;
let supported: bool =
sqlx::query_scalar("SELECT capabilities ? $2 FROM agent_installations WHERE agent_id=$1")
.bind(pair.agent_id)
.bind(wire::CAPABILITY)
.fetch_one(&mut *tx)
.await?;
if !supported {
return Err(Error(
StatusCode::CONFLICT,
"TRANSFER_AGENT_UPDATE_REQUIRED",
));
}
if transfer(&mut tx, pair.agent_id).await?.is_some() {
return Err(Error(StatusCode::CONFLICT, "TRANSFER_ALREADY_PENDING"));
}
let id = Uuid::new_v4();
sqlx::query(
"INSERT INTO device_customer_transfers(transfer_id,source_tenant_id,source_asset_id,source_agent_id,target_tenant_id,requested_by)
VALUES($1,$2,$3,$4,$5,$6)",
).bind(id).bind(tenant).bind(asset).bind(pair.agent_id).bind(input.target_tenant_id).bind(&p.name)
.execute(&mut *tx).await?;
audit(&mut tx, &p, "device_customer_transfer_prepared", json!({
"transfer_id":id,"asset_id":asset,"agent_id":pair.agent_id,"target_tenant_id":input.target_tenant_id
})).await?;
tx.commit().await?;
Ok(private_json(json!({"transfer_id":id,"state":"pending"})))
}
async fn verify_pairing(
tx: &mut Transaction<'_, Postgres>,
asset: Uuid,
agent: Uuid,
) -> Result<(), Error> {
sqlx::query_scalar::<_, Uuid>(
"SELECT agent_id FROM agent_installations WHERE asset_id=$1 AND agent_id=$2",
)
.bind(asset)
.bind(agent)
.fetch_optional(&mut **tx)
.await?
.ok_or(NOT_FOUND)?;
Ok(())
}
async fn cancel(
State(s): State<crate::State>,
headers: HeaderMap,
Path((tenant, asset, id)): Path<(Uuid, Uuid, Uuid)>,
Query(pair): Query<Pairing>,
) -> Result<StatusCode, Error> {
let (p, mut tx) = source(&s, &headers, tenant, true).await?;
lock_source(&mut tx, tenant, asset, pair.agent_id).await?;
let changed = sqlx::query(
"UPDATE device_customer_transfers SET cancelled_at=now()
WHERE transfer_id=$1 AND source_agent_id=$2 AND target_agent_id IS NULL AND cancelled_at IS NULL",
).bind(id).bind(pair.agent_id).execute(&mut *tx).await?.rows_affected();
if changed == 0 {
return Err(Error(StatusCode::CONFLICT, "TRANSFER_ALREADY_STARTED"));
}
audit(
&mut tx,
&p,
"device_customer_transfer_cancelled",
json!({
"transfer_id":id,"asset_id":asset,"agent_id":pair.agent_id
}),
)
.await?;
tx.commit().await?;
Ok(StatusCode::NO_CONTENT)
}
async fn complete(
State(s): State<crate::State>,
headers: HeaderMap,
Path((tenant, asset, id)): Path<(Uuid, Uuid, Uuid)>,
Query(pair): Query<Pairing>,
) -> Result<Response, Error> {
let (p, mut tx) = source(&s, &headers, tenant, true).await?;
verify_pairing(&mut tx, asset, pair.agent_id).await?;
let t = transfer(&mut tx, pair.agent_id)
.await?
.filter(|t| t.transfer_id == id)
.ok_or(NOT_FOUND)?;
destination(&mut tx, &p, t.target_tenant_id).await?;
// Older WebUI bundles may still request completion after the atomic cutover.
if t.completed_at.is_none() {
return Err(Error(StatusCode::CONFLICT, "TRANSFER_TARGET_NOT_READY"));
}
tx.commit().await?;
Ok(private_json(json!({"state":"completed"})))
}
async fn device_offer(
State(s): State<crate::State>,
Extension(peer): Extension<crate::Peer>,
headers: HeaderMap,
) -> Result<Response, Error> {
let (tenant, agent) = crate::agent_updates::device(&s, peer, &headers).await?;
let mut tx = rmm_persistence::tenant_tx(&s.pool, tenant).await?;
let offer = transfer(&mut tx, agent)
.await?
.filter(|t| t.completed_at.is_none())
.map(|t| offer(tenant, agent, &t));
tx.commit().await?;
Ok(private_json(json!({"offer":offer})))
}
fn offer(tenant: Uuid, agent: Uuid, t: &Transfer) -> wire::Offer {
wire::Offer {
transfer_id: t.transfer_id,
source_tenant_id: tenant,
source_agent_id: agent,
target_tenant_id: t.target_tenant_id,
}
}
async fn device_certificate(
State(s): State<crate::State>,
Extension(peer): Extension<crate::Peer>,
headers: HeaderMap,
Path(id): Path<Uuid>,
Json(input): Json<wire::CertificateRequest>,
) -> Result<Response, Error> {
let (tenant, agent) = crate::agent_updates::device(&s, peer, &headers).await?;
let invalid = || Error(StatusCode::BAD_REQUEST, "INVALID_CSR");
if input.csr_pem.len() > 8192 {
return Err(invalid());
}
let csr = openssl::x509::X509Req::from_pem(input.csr_pem.as_bytes()).map_err(|_| invalid())?;
let public = csr.public_key().map_err(|_| invalid())?;
if !csr.verify(&public).map_err(|_| invalid())? {
return Err(invalid());
}
let digest = hash(&public.public_key_to_der().map_err(|_| invalid())?);
let mut tx = rmm_persistence::tenant_tx(&s.pool, tenant).await?;
crate::agent_service::control_lock(&mut tx, tenant, agent).await?;
let (asset, generation): (Uuid, i64) = sqlx::query_as(
"SELECT asset_id,connection_generation FROM agent_installations
WHERE agent_id=$1 AND revoked_at IS NULL FOR UPDATE",
)
.bind(agent)
.fetch_optional(&mut *tx)
.await?
.ok_or(NOT_FOUND)?;
let t = transfer(&mut tx, agent)
.await?
.filter(|t| t.transfer_id == id && t.completed_at.is_none())
.ok_or(NOT_FOUND)?;
ensure_idle(&mut tx, agent).await?;
if t.target_agent_id.is_none() {
crate::permissions::require(&mut tx, &t.requested_by, "inventory:read").await?;
crate::permissions::require(&mut tx, &t.requested_by, "devices:revoke").await?;
}
select_tenant(&mut tx, t.target_tenant_id).await?;
let (new_asset, new_agent, certificate_pem) = if let (Some(target_asset), Some(target_agent)) =
(t.target_asset_id, t.target_agent_id)
{
if t.csr_sha256.as_deref() != Some(&digest) {
return Err(Error(StatusCode::CONFLICT, "TRANSFER_KEY_MISMATCH"));
}
let pem: String = sqlx::query_scalar(
"SELECT certificate_pem FROM agent_installations WHERE asset_id=$1 AND agent_id=$2 AND revoked_at IS NULL",
).bind(target_asset).bind(target_agent).fetch_optional(&mut *tx).await?.ok_or(NOT_FOUND)?;
sqlx::query(
"UPDATE agent_installations SET connection_generation=GREATEST(connection_generation,$2)
WHERE agent_id=$1 AND last_authenticated_at IS NULL",
).bind(target_agent).bind(generation).execute(&mut *tx).await?;
(target_asset, target_agent, pem)
} else {
crate::permissions::require(&mut tx, &t.requested_by, "inventory:read").await?;
crate::permissions::require(&mut tx, &t.requested_by, "devices:enroll").await?;
let new_asset = Uuid::new_v4();
let new_agent = Uuid::new_v4();
let certificate =
crate::tls::issue(&s, &input.csr_pem, new_agent).map_err(|_| invalid())?;
let pem = String::from_utf8(certificate.to_pem().map_err(anyhow::Error::from)?)
.map_err(anyhow::Error::from)?;
sqlx::query("INSERT INTO assets(tenant_id,asset_id) VALUES($1,$2)")
.bind(t.target_tenant_id)
.bind(new_asset)
.execute(&mut *tx)
.await?;
sqlx::query(
"INSERT INTO agent_installations(tenant_id,asset_id,agent_id,certificate_sha256,certificate_pem,connection_generation,customer_transfer_pending)
VALUES($1,$2,$3,$4,$5,$6,true)",
).bind(t.target_tenant_id).bind(new_asset).bind(new_agent)
.bind(hash(&certificate.to_der().map_err(anyhow::Error::from)?)).bind(&pem).bind(generation)
.execute(&mut *tx).await?;
select_tenant(&mut tx, tenant).await?;
sqlx::query(
"UPDATE device_customer_transfers SET target_asset_id=$2,target_agent_id=$3,csr_sha256=$4
WHERE transfer_id=$1 AND source_asset_id=$5",
).bind(id).bind(new_asset).bind(new_agent).bind(digest).bind(asset).execute(&mut *tx).await?;
(new_asset, new_agent, pem)
};
let binding = wire::Binding {
offer: offer(tenant, agent, &t),
asset_id: new_asset,
agent_id: new_agent,
certificate_sha256: hash(certificate_pem.as_bytes()),
};
let signature_hex = hex::encode(s.job_signer.sign(&binding.signing_bytes()?)?);
tx.commit().await?;
Ok((
[("cache-control", "private, no-store")],
Json(wire::CertificateResponse {
certificate_pem,
receipt: wire::Receipt {
binding,
signature_hex,
},
}),
)
.into_response())
}
+5 -2
View File
@@ -2,7 +2,8 @@ WITH classified AS (
SELECT a.tenant_id,a.asset_id,a.agent_id,a.created_at,a.revoked_at,a.last_contact_at,
a.last_authenticated_at,a.last_authenticated_certificate_sha256,a.certificate_sha256,
a.enrollment_generation,a.assignment_version,a.capabilities,a.reported_agent_version,
asset.archived_at,t.name AS customer_name,
a.system_usage,a.system_usage_observed_at,
asset.archived_at,asset.hypervisor_asset_id,asset.hypervisor_source,asset.purpose,t.name AS customer_name,
CASE
WHEN a.revoked_at IS NOT NULL THEN 'revoked'
WHEN asset.archived_at IS NOT NULL THEN 'archived'
@@ -24,6 +25,8 @@ WITH classified AS (
JOIN assets asset USING (tenant_id,asset_id)
JOIN tenants t USING(tenant_id)
WHERE a.tenant_id=$1 AND a.created_at<=$2 AND a.agent_id>$3
AND NOT a.customer_transfer_pending
AND (a.customer_transfer_activated_at IS NULL OR a.customer_transfer_activated_at<=$2)
AND ($4::uuid IS NULL OR a.asset_id=$4)
AND ($5::uuid IS NULL OR a.agent_id=$5)
)
@@ -31,7 +34,7 @@ SELECT c.*,o.observed_at,o.received_at,o.data,o.source_id FROM classified c
LEFT JOIN LATERAL (
SELECT observed_at,received_at,data,source_id FROM observations
WHERE tenant_id=c.tenant_id AND agent_id=c.agent_id AND received_at<=$2
AND source_id IN ('linux.base','windows.base')
AND source_id IN ('linux.base','windows.base','opnsense.base')
ORDER BY observed_at DESC,received_at DESC,sequence DESC LIMIT 1
) o ON true
WHERE ($6::text IS NULL OR (c.lifecycle='current')=($6='current'))
+443 -18
View File
@@ -14,6 +14,7 @@ use rmm_domain::inventory_document::DeviceType;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use sqlx::{Postgres, Row, Transaction, postgres::PgRow};
use std::collections::HashMap;
use uuid::Uuid;
#[derive(Default, Deserialize, Serialize, schemars::JsonSchema, Clone, Copy)]
@@ -121,15 +122,13 @@ pub async fn list(
};
tenants.sort_unstable();
tenants.dedup();
let hostname = q.q.as_deref().unwrap_or("").trim();
if hostname.len() > 255 || hostname.chars().any(char::is_control) {
return Err(Error(StatusCode::BAD_REQUEST, "VALIDATION_ERROR"));
}
let hostname = hostname_filter(q.q.as_deref())?;
let binding = json!([tenant, tenants, q.view, q.status, hostname]);
let scope = format!(
"device-fleet:{}",
rmm_domain::jobs::hash(binding.to_string().as_bytes())
);
let first_page = q.cursor.is_none();
let mut page = Cursor::parse(
s,
&p,
@@ -145,10 +144,15 @@ pub async fn list(
let mut last = None;
let mut truncated = false;
let mut authorized = false;
let mut principals = Vec::new();
for customer in tenants {
if page.last_tenant.is_some_and(|after| customer < after) {
continue;
}
if customer == p.tenant {
principals.push(p.clone());
continue;
}
let mut headers = h.clone();
headers.insert(
"x-rmm-tenant",
@@ -157,7 +161,25 @@ pub async fn list(
);
let current = crate::jobs::principal(s, &headers, false).await?;
crate::jobs::scope(&current, customer, false)?;
let mut tx = rmm_persistence::tenant_tx(&s.pool, customer).await?;
principals.push(current);
}
// Read all customers from the same snapshot so a transfer cannot appear
// under both customers if its cutover commits while listing the fleet.
let mut tx = s.pool.begin().await?;
sqlx::query("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ READ ONLY")
.execute(&mut *tx)
.await?;
if first_page {
page.as_of = sqlx::query_scalar("SELECT clock_timestamp()")
.fetch_one(&mut *tx)
.await?;
}
for current in principals {
let customer = current.tenant;
sqlx::query("SELECT set_config('app.tenant_id',$1,true)")
.bind(customer.to_string())
.execute(&mut *tx)
.await?;
match crate::permissions::require_principal(&mut tx, &current, "inventory:read").await {
Err(Error(StatusCode::FORBIDDEN, _)) if tenant.is_none() => continue,
other => other?,
@@ -180,19 +202,20 @@ pub async fn list(
}
.rows(&mut tx)
.await?;
tx.commit().await?;
let hosts = assigned_hosts(&mut tx, customer, page.as_of, &rows).await?;
for row in rows {
if items.len() == capacity {
truncated = true;
break;
}
last = Some((customer, row.get::<Uuid, _>("agent_id")));
items.push(project(&row, page.as_of, tenant.is_some()));
items.push(project(&row, page.as_of, tenant.is_some(), &hosts));
}
if truncated {
break;
}
}
tx.commit().await?;
if !authorized {
return Err(Error(StatusCode::FORBIDDEN, "FORBIDDEN"));
}
@@ -236,8 +259,14 @@ pub async fn detail(
.await?
.pop()
.ok_or(Error(StatusCode::NOT_FOUND, "NOT_FOUND"))?;
let mut item = project(&row, now, true);
item["volumes"] = volumes(&item["inventory"]);
let hosts = assigned_hosts(&mut tx, tenant, now, std::slice::from_ref(&row)).await?;
let mut item = project(&row, now, true, &hosts);
if item["proxmox"]["data"].is_object() {
let rows: Vec<(Uuid, Uuid, Option<String>, Option<String>, Value)> =
sqlx::query_as(include_str!("proxmox_guest_devices.sql"))
.bind(tenant).bind(asset).fetch_all(&mut *tx).await?;
item["proxmox"]["guest_devices"] = proxmox_guest_devices(&item["proxmox"]["data"], &rows);
}
item["recent_jobs"] = if p.permits_scope("operations:read")
&& crate::permissions::allowed(&mut tx, &p.name, "operations:read").await?
{
@@ -259,6 +288,25 @@ pub async fn detail(
Ok(Json(item))
}
/// Linked devices per reported guest, matched by the MACs both sides report.
fn proxmox_guest_devices(
data: &Value,
rows: &[(Uuid, Uuid, Option<String>, Option<String>, Value)],
) -> Value {
let mut result = Vec::new();
for guest in data["guests"].as_array().into_iter().flatten() {
let macs: Vec<&str> = guest["macs"].as_array().into_iter().flatten()
.filter_map(Value::as_str).collect();
let linked = rows.iter().find(|(_, _, _, _, device)| device.as_array().into_iter().flatten()
.filter_map(Value::as_str).any(|m| macs.contains(&m)));
if let (Some(vmid), Some((asset_id, agent_id, source, hostname, _))) = (guest["vmid"].as_u64(), linked) {
result.push(json!({"vmid":vmid,"asset_id":asset_id,"agent_id":agent_id,
"hostname":hostname,"source":source}));
}
}
Value::Array(result)
}
fn observed<'a>(data: &'a Value, scope: &str) -> &'a Value {
data["observations"]
.as_array()
@@ -272,6 +320,14 @@ fn observed<'a>(data: &'a Value, scope: &str) -> &'a Value {
}
fn volumes(data: &Value) -> Value {
let mut items = Vec::new();
if let Some(drives) = observed(data, "windows.volumes")["drives"].as_array() {
let mut seen = std::collections::HashSet::new();
for drive in drives.iter().take(26) {
let Some(name) = drive["name"].as_str() else { continue };
if !matches!(name.as_bytes(), [b'A'..=b'Z', b':']) || !seen.insert(name) { continue; }
items.push(json!({"name":name,"free_bytes":drive["free_bytes"].as_u64(),"total_bytes":drive["total_bytes"].as_u64()}));
}
}
if let Some(rows) = observed(data, "linux.volumes").as_array() {
for entry in rows.iter().filter(|entry| entry["quality"] == "valid") {
let v = &entry["data"];
@@ -294,6 +350,136 @@ fn device_type(hardware: &Value) -> Option<DeviceType> {
DeviceType::deserialize(&hardware["device_type"]).ok()
}
fn inventory_device_type(data: &Value) -> Option<DeviceType> {
device_type(observed(data, "linux.hardware"))
.or_else(|| device_type(observed(data, "windows.hardware")))
.or_else(|| device_type(&data["hardware"]))
}
fn hostname_filter(query: Option<&str>) -> Result<&str, Error> {
let hostname = query.unwrap_or("").trim();
if hostname.len() > 255 || hostname.chars().any(char::is_control) {
return Err(Error(StatusCode::BAD_REQUEST, "VALIDATION_ERROR"));
}
Ok(hostname)
}
#[derive(Serialize)]
struct Host {
asset_id: Uuid,
agent_id: Uuid,
hostname: String,
}
async fn hypervisors(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
as_of: DateTime<Utc>,
ids: Option<&[Uuid]>,
hostname: &str,
limit: i64,
) -> Result<Vec<Host>, Error> {
let rows: Vec<(Uuid, Uuid, String)> = sqlx::query_as(include_str!("hypervisors.sql"))
.bind(tenant)
.bind(as_of)
.bind(ids)
.bind(hostname)
.bind(limit)
.fetch_all(&mut **tx)
.await?;
Ok(rows.into_iter().map(|(asset_id, agent_id, hostname)| Host {
asset_id, agent_id, hostname,
}).collect())
}
async fn assigned_hosts(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
as_of: DateTime<Utc>,
rows: &[PgRow],
) -> Result<HashMap<Uuid, Host>, Error> {
let mut ids: Vec<Uuid> = rows.iter().filter_map(|row| row.get("hypervisor_asset_id")).collect();
ids.sort_unstable();
ids.dedup();
if ids.is_empty() {
return Ok(HashMap::new());
}
Ok(hypervisors(tx, tenant, as_of, Some(&ids), "", ids.len() as i64)
.await?.into_iter().map(|host| (host.asset_id, host)).collect())
}
fn hypervisor_display(data: &Value, assigned: Option<&Host>) -> Value {
if inventory_device_type(data) != Some(DeviceType::Virtual) {
return Value::Null;
}
if let Some(host) = assigned {
return json!({"kind":"device","asset_id":host.asset_id,
"agent_id":host.agent_id,"hostname":host.hostname});
}
let linux = observed(data, "linux.hardware");
let windows = observed(data, "windows.hardware");
let manufacturer = linux["manufacturer"].as_str()
.filter(|value| !value.trim().is_empty())
.or_else(|| linux["readable_dmi"]["sys_vendor"].as_str())
.or_else(|| windows["manufacturer"].as_str());
if manufacturer.is_some_and(|name| matches!(name.trim().to_ascii_lowercase().as_str(),
"hetzner" | "hetzner cloud" | "hetzner online" | "hetzner online gmbh")) {
return json!({"kind":"placeholder","provider":"hetzner","hostname":"Hetzner Cloud"});
}
Value::Null
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub struct HypervisorFilter {
q: Option<String>,
}
pub async fn hypervisor_options(
State(s): State<crate::State>,
h: HeaderMap,
Path(tenant): Path<Uuid>,
Query(query): Query<HypervisorFilter>,
) -> Result<Json<Value>, Error> {
let p = crate::jobs::principal(&s, &h, false).await?;
crate::jobs::scope(&p, tenant, false)?;
let mut tx = rmm_persistence::tenant_tx(&s.pool, tenant).await?;
crate::permissions::require_principal(&mut tx, &p, "inventory:read").await?;
let mut hosts = hypervisors(&mut tx, tenant, Utc::now(), None,
hostname_filter(query.q.as_deref())?, 51).await?;
let truncated = hosts.len() > 50;
hosts.truncate(50);
tx.commit().await?;
Ok(Json(json!({"items":hosts,"truncated":truncated})))
}
pub(crate) async fn validate_vm_host_assignment(
tx: &mut Transaction<'_, Postgres>,
tenant: Uuid,
asset: Uuid,
agent: Uuid,
host: Option<Uuid>,
) -> Result<(), Error> {
let now = Utc::now();
let row = Selection {
tenant, asset: Some(asset), agent: Some(agent), view: Some(View::Current),
status: None, hostname: "", last: Uuid::nil(), as_of: now, limit: 1,
}.rows(tx).await?.pop().ok_or(Error(StatusCode::NOT_FOUND, "NOT_FOUND"))?;
let data = row.get::<Option<Value>, _>("data").unwrap_or(Value::Null);
if inventory_device_type(&data) != Some(DeviceType::Virtual) {
return Err(Error(StatusCode::CONFLICT, "HYPERVISOR_REQUIRES_VM"));
}
if let Some(host) = host {
if asset == host {
return Err(Error(StatusCode::CONFLICT, "INVALID_HYPERVISOR"));
}
if hypervisors(tx, tenant, now, Some(&[host]), "", 1).await?.is_empty() {
return Err(Error(StatusCode::NOT_FOUND, "HYPERVISOR_UNAVAILABLE"));
}
}
Ok(())
}
fn os_label(data: &Value) -> Option<String> {
let linux_hardware = observed(data, "linux.hardware");
if linux_hardware["proxmox_ve"].as_bool() == Some(true)
@@ -348,12 +534,25 @@ fn os_label(data: &Value) -> Option<String> {
Some("Linux".to_owned())
}
}
fn project(row: &PgRow, now: DateTime<Utc>, full: bool) -> Value {
fn project(row: &PgRow, now: DateTime<Utc>, full: bool, hosts: &HashMap<Uuid, Host>) -> Value {
let data = row.get::<Option<Value>, _>("data").unwrap_or(Value::Null);
let cpu = observed(&data, "linux.cpu");
let windows = observed(&data, "windows.hardware");
let device_type =
device_type(observed(&data, "linux.hardware")).or_else(|| device_type(windows));
let linux_memory = observed(&data, "linux.memory");
let usage = row
.get::<Option<Value>, _>("system_usage")
.unwrap_or(Value::Null);
let usage_at: Option<DateTime<Utc>> = row.get("system_usage_observed_at");
let inventory_at: Option<DateTime<Utc>> = row.get("observed_at");
let cpu_observed_at = usage["cpu_usage_observed_at"].as_str();
let cpu_usage = cpu_observed_at.and_then(|_| usage["cpu_usage_percent"].as_f64());
let ram_available = usage["ram_available_bytes"].as_u64();
let ram_total = usage["ram_total_bytes"]
.as_u64()
.or_else(|| linux_memory["total_bytes"].as_u64())
.or_else(|| windows["memory_total_bytes"].as_u64())
.or_else(|| data["memory_total_bytes"].as_u64());
let device_type = inventory_device_type(&data);
let last = row.get::<Option<DateTime<Utc>>, _>("last_contact_at");
let revoked = row.get::<Option<DateTime<Utc>>, _>("revoked_at");
let presence = match (revoked, last.map(|t| (now - t).num_seconds())) {
@@ -368,17 +567,62 @@ fn project(row: &PgRow, now: DateTime<Utc>, full: bool) -> Value {
"agent_id":row.get::<Uuid,_>("agent_id"),"asset_id":row.get::<Uuid,_>("asset_id"),
"status":row.get::<String,_>("status"),"lifecycle":row.get::<String,_>("lifecycle"),
"archived_at":row.get::<Option<DateTime<Utc>>,_>("archived_at"),"revoked_at":revoked,
"hostname":data["hostname"].as_str(),"device_type":device_type,"os":os_label(&data),
"hostname":data["hostname"].as_str(),"purpose":row.get::<Option<String>,_>("purpose"),
"device_type":device_type,"os":os_label(&data),
"disk_free_bytes":data["root_available_bytes"].as_u64(),"disk_total_bytes":data["root_total_bytes"].as_u64(),
"cpu_model":cpu["model"].as_str().or_else(||windows["cpu_model"].as_str()),
"cpu_logical_count":cpu["logical_processors"].as_u64().or_else(||windows["logical_processors_visible"].as_u64()),
"ram_total_bytes":observed(&data,"linux.memory")["total_bytes"].as_u64().or_else(||windows["memory_total_bytes"].as_u64()),
"cpu_model":cpu["model"].as_str().or_else(||windows["cpu_model"].as_str()).or_else(||data["cpu_model"].as_str()),
"cpu_logical_count":cpu["logical_processors"].as_u64().or_else(||windows["logical_processors_visible"].as_u64()).or_else(||data["logical_processors"].as_u64()),
"cpu_usage_percent":cpu_usage,
"cpu_usage_observed_at":cpu_observed_at,
"ram_total_bytes":ram_total,
"ram_available_bytes":ram_available.or_else(|| linux_memory["available_bytes"].as_u64().or_else(||windows["memory_available_bytes"].as_u64())
.filter(|available| ram_total.is_some_and(|total| *available <= total))),
"ram_observed_at":if ram_available.is_some() {usage_at} else {inventory_at},
"last_contact_at":last,"agent_version":row.get::<Option<String>,_>("reported_agent_version"),
"presence":presence,"observed_at":row.get::<Option<DateTime<Utc>>,_>("observed_at"),
"presence":presence,"observed_at":inventory_at,
"received_at":row.get::<Option<DateTime<Utc>>,_>("received_at"),
"source_id":row.get::<Option<String>,_>("source_id"),"quality":if data.is_null(){"unknown"}else{"valid"},
"completeness":"partial","health":"unknown"
});
if full || observed(&data, "windows.volumes").is_object() {
item["volumes"] = volumes(&data);
}
if let Some(scope) = data["observations"].as_array().and_then(|scopes|
scopes.iter().find(|s| s["scope"] == "linux.disks")) {
item["disks"] = if scope["quality"] == "valid" && scope["data"].is_array() {
scope["data"].clone()
} else { Value::Null };
item["disks_truncated"] = json!(scope["truncated"] == true);
}
if let Some(scope) = data["observations"].as_array().and_then(|scopes|
scopes.iter().find(|s| s["scope"] == "linux.proxmox")) {
let status = (scope["quality"] == "valid")
.then(|| rmm_domain::inventory_document::proxmox::backup_status(&scope["data"]))
.flatten();
item["proxmox_backup"] = status.as_ref().map_or(Value::Null, |s| json!({
"all_scheduled_backups_ran":s["all_scheduled_backups_ran"],
"problem_jobs":s["problem_jobs"],"enabled_jobs":s["enabled_jobs"]}));
if full {
item["proxmox"] = json!({
"quality":scope["quality"],"error_code":scope["error_code"],
"observed_at":scope["observed_at"],"truncated":scope["truncated"],
"data":if scope["quality"] == "valid" { scope["data"].clone() } else { Value::Null },
"backup_status":status,"guest_devices":[]});
}
}
if device_type == Some(DeviceType::Virtual) && row.get::<String, _>("lifecycle") == "current" {
let assigned_id: Option<Uuid> = row.get("hypervisor_asset_id");
let assigned = assigned_id.and_then(|id| hosts.get(&id));
item["hypervisor"] = hypervisor_display(&data, assigned);
let source = row.get::<Option<String>, _>("hypervisor_source");
if item["hypervisor"]["kind"] == "device" {
item["hypervisor"]["source"] = json!(source);
}
if full {
item["assigned_hypervisor_asset_id"] = json!(assigned_id);
item["hypervisor_source"] = json!(source);
}
}
if full {
item["inventory"] = data;
item["capabilities"] = row.get("capabilities");
@@ -397,6 +641,18 @@ fn project(row: &PgRow, now: DateTime<Utc>, full: bool) -> Value {
mod tests {
use super::*;
#[test]
fn windows_drives_include_other_letters_and_preserve_legacy_system_volume() {
let mut snapshot = json!({"root_available_bytes":4,"root_total_bytes":8,"observations":[
{"scope":"windows.volumes","quality":"valid","data":{}}]});
assert_eq!(volumes(&snapshot)[0]["name"], "Systemvolume");
let drives = json!([{"name":"C:","free_bytes":4,"total_bytes":8},
{"name":"D:","free_bytes":0,"total_bytes":64},
{"name":"E:","free_bytes":null,"total_bytes":null}]);
snapshot["observations"][0]["data"]["drives"] = drives.clone();
assert_eq!(volumes(&snapshot), drives);
}
fn windows_os(name: &str, build: &str) -> String {
let old = json!({"os":name,"observations":[{"scope":"windows.os","quality":"valid",
"data":{"product_name":name,"build":build,"display_version":"26H2"}}]});
@@ -439,6 +695,175 @@ mod tests {
assert_eq!(device_type(&json!({"device_type":"desktop"})), None);
}
#[test]
fn hetzner_placeholder_requires_a_valid_guest_and_yields_to_a_registered_host() {
let mut data = json!({"observations":[{"scope":"linux.hardware","quality":"valid",
"data":{"device_type":"virtual","readable_dmi":{"sys_vendor":" Hetzner "}}}]});
assert_eq!(hypervisor_display(&data, None)["kind"], "placeholder");
let host = Host { asset_id: Uuid::new_v4(), agent_id: Uuid::new_v4(), hostname: "pve01".into() };
assert_eq!(hypervisor_display(&data, Some(&host))["asset_id"], json!(host.asset_id));
data["observations"][0]["data"]["device_type"] = json!("physical");
assert!(hypervisor_display(&data, None).is_null());
data["observations"][0]["data"]["device_type"] = json!("virtual");
data["observations"][0]["quality"] = json!("error");
assert!(hypervisor_display(&data, None).is_null());
data["observations"][0] = json!({"scope":"windows.hardware","quality":"valid",
"data":{"device_type":"virtual","manufacturer":"HETZNER ONLINE GMBH"}});
assert_eq!(hypervisor_display(&data, None)["hostname"], "Hetzner Cloud");
data["observations"][0]["data"]["manufacturer"] = json!("Other company");
assert!(hypervisor_display(&data, None).is_null());
}
#[tokio::test]
#[ignore = "requires the disposable RLS database from tests/local_access_database.py"]
async fn proxmox_guest_macs_link_vms_automatically_but_never_override_people() {
let admin = sqlx::PgPool::connect(&std::env::var("RMM_TEST_REPAIR_ADMIN_URL").unwrap())
.await.unwrap();
let app = sqlx::PgPool::connect(&std::env::var("RMM_TEST_REPAIR_APP_URL").unwrap())
.await.unwrap();
let (tenant, other) = (Uuid::new_v4(), Uuid::new_v4());
let (host, vm, manual_vm, foreign_vm, twin_a, twin_b) =
(Uuid::new_v4(), Uuid::new_v4(), Uuid::new_v4(), Uuid::new_v4(), Uuid::new_v4(), Uuid::new_v4());
for customer in [tenant, other] {
sqlx::query("INSERT INTO tenants(tenant_id,name) VALUES($1,'Proxmox link test')")
.bind(customer).execute(&admin).await.unwrap();
}
let guests = |macs: &[&str]| json!({"node":"pve","guests":macs.iter().enumerate().map(|(i, m)|
json!({"vmid":100 + i,"macs":[m]})).collect::<Vec<_>>()});
let device = |kind: &str, mac: &str, proxmox: Option<Value>| {
let mut scopes = vec![
json!({"scope":"linux.hardware","quality":"valid","data":{"device_type":kind}}),
json!({"scope":"linux.interfaces","quality":"valid","data":[{"name":"eth0","mac":mac}]}),
];
if let Some(data) = proxmox {
scopes.push(json!({"scope":"linux.proxmox","quality":"valid","data":data}));
}
json!({"hostname":"h","os":"Debian","root_available_bytes":1,"root_total_bytes":2,"observations":scopes})
};
let insert = |customer: Uuid, id: Uuid, data: Value, sequence: i64| {
let admin = admin.clone();
async move {
sqlx::query("INSERT INTO observations(tenant_id,agent_id,message_id,boot_id,sequence,observed_at,source_id,quality,completeness,snapshot_id,data,payload_sha256) VALUES($1,$2,$3,$4,$5,now(),'linux.base','valid','partial',$6,$7,$8)")
.bind(customer).bind(id).bind(Uuid::new_v4()).bind(Uuid::new_v4()).bind(sequence)
.bind(Uuid::new_v4()).bind(data).bind("0".repeat(64)).execute(&admin).await.unwrap();
}
};
let all = ["bc:24:11:00:00:01", "bc:24:11:00:00:02", "bc:24:11:00:00:03", "bc:24:11:00:00:04"];
for (customer, id, data) in [
(tenant, host, device("hypervisor", "52:54:00:00:00:99", Some(guests(&all)))),
(tenant, vm, device("virtual", "BC:24:11:00:00:01", None)),
(tenant, manual_vm, device("virtual", "bc:24:11:00:00:02", None)),
(other, foreign_vm, device("virtual", "bc:24:11:00:00:03", None)),
(tenant, twin_a, device("virtual", "bc:24:11:00:00:04", None)),
(tenant, twin_b, device("virtual", "bc:24:11:00:00:04", None)),
] {
sqlx::query("INSERT INTO assets(tenant_id,asset_id) VALUES($1,$2)")
.bind(customer).bind(id).execute(&admin).await.unwrap();
sqlx::query("INSERT INTO agent_installations(tenant_id,asset_id,agent_id,certificate_sha256,last_contact_at,last_authenticated_at,last_authenticated_certificate_sha256) VALUES($1,$2,$2,$3,now(),now(),$3)")
.bind(customer).bind(id).bind(rmm_domain::jobs::hash(format!("{customer}:{id}").as_bytes()))
.execute(&admin).await.unwrap();
insert(customer, id, data, 1).await;
}
sqlx::query("UPDATE assets SET hypervisor_source='manual' WHERE asset_id=$1")
.bind(manual_vm).execute(&admin).await.unwrap();
let link = |id: Uuid| {
let admin = admin.clone();
async move {
sqlx::query_as::<_, (Option<Uuid>, Option<String>)>(
"SELECT hypervisor_asset_id,hypervisor_source FROM assets WHERE asset_id=$1")
.bind(id).fetch_one(&admin).await.unwrap()
}
};
let mut tx = rmm_persistence::tenant_tx(&app, tenant).await.unwrap();
crate::proxmox_links::reconcile(&mut tx, tenant).await;
tx.commit().await.unwrap();
assert_eq!(link(vm).await, (Some(host), Some("proxmox".into())));
assert_eq!(link(manual_vm).await, (None, Some("manual".into())));
assert_eq!(link(foreign_vm).await, (None, None));
assert_eq!(link(twin_a).await, (None, None), "duplicate MACs are ambiguous");
let audits: i64 = sqlx::query_scalar("SELECT count(*) FROM audit_events WHERE tenant_id=$1 AND event_type='device_hypervisor_detected'")
.bind(tenant).fetch_one(&admin).await.unwrap();
assert_eq!(audits, 1);
// Idempotent: a second pass changes nothing.
let mut tx = rmm_persistence::tenant_tx(&app, tenant).await.unwrap();
crate::proxmox_links::reconcile(&mut tx, tenant).await;
tx.commit().await.unwrap();
let audits: i64 = sqlx::query_scalar("SELECT count(*) FROM audit_events WHERE tenant_id=$1")
.bind(tenant).fetch_one(&admin).await.unwrap();
assert_eq!(audits, 1);
// The guest left the host: the automatic link is released.
insert(tenant, host, device("hypervisor", "52:54:00:00:00:99", Some(guests(&all[1..]))), 2).await;
let mut tx = rmm_persistence::tenant_tx(&app, tenant).await.unwrap();
crate::proxmox_links::reconcile(&mut tx, tenant).await;
tx.commit().await.unwrap();
assert_eq!(link(vm).await, (None, None));
}
#[tokio::test]
#[ignore = "requires the disposable RLS database from tests/local_access_database.py"]
async fn host_links_resolve_outside_the_page_and_stay_inside_the_customer() {
let admin = sqlx::PgPool::connect(&std::env::var("RMM_TEST_REPAIR_ADMIN_URL").unwrap())
.await.unwrap();
let app = sqlx::PgPool::connect(&std::env::var("RMM_TEST_REPAIR_APP_URL").unwrap())
.await.unwrap();
let tenant = Uuid::new_v4();
let other = Uuid::new_v4();
let mut ids = [Uuid::new_v4(), Uuid::new_v4()];
ids.sort_unstable();
let [vm, host] = ids;
let foreign_host = Uuid::new_v4();
for customer in [tenant, other] {
sqlx::query("INSERT INTO tenants(tenant_id,name) VALUES($1,'Hypervisor test')")
.bind(customer).execute(&admin).await.unwrap();
}
for (customer, id, name, kind) in [
(tenant, vm, "guest", "virtual"),
(tenant, host, "hypervisor-1", "hypervisor"),
(other, foreign_host, "hypervisor-2", "hypervisor"),
] {
sqlx::query("INSERT INTO assets(tenant_id,asset_id) VALUES($1,$2)")
.bind(customer).bind(id).execute(&admin).await.unwrap();
sqlx::query("INSERT INTO agent_installations(tenant_id,asset_id,agent_id,certificate_sha256,last_contact_at,last_authenticated_at,last_authenticated_certificate_sha256) VALUES($1,$2,$2,$3,now(),now(),$3)")
.bind(customer).bind(id).bind(rmm_domain::jobs::hash(format!("{customer}:{id}").as_bytes()))
.execute(&admin).await.unwrap();
let data = json!({"hostname":name,"os":"Ubuntu","root_available_bytes":10,"root_total_bytes":20,
"observations":[{"scope":"linux.hardware","quality":"valid",
"data":{"device_type":kind,"manufacturer":"Hetzner"}}]});
sqlx::query("INSERT INTO observations(tenant_id,agent_id,message_id,boot_id,sequence,observed_at,source_id,quality,completeness,snapshot_id,data,payload_sha256) VALUES($1,$2,$3,$4,1,now(),'linux.base','valid','partial',$5,$6,$7)")
.bind(customer).bind(id).bind(Uuid::new_v4()).bind(Uuid::new_v4())
.bind(Uuid::new_v4()).bind(data).bind("0".repeat(64)).execute(&admin).await.unwrap();
}
let mut tx = rmm_persistence::tenant_tx(&app, tenant).await.unwrap();
assert!(validate_vm_host_assignment(&mut tx, tenant, vm, vm, Some(foreign_host)).await.is_err());
assert!(validate_vm_host_assignment(&mut tx, tenant, vm, vm, Some(vm)).await.is_err());
validate_vm_host_assignment(&mut tx, tenant, vm, vm, Some(host)).await.unwrap();
sqlx::query("UPDATE assets SET hypervisor_asset_id=$2 WHERE asset_id=$1")
.bind(vm).bind(host).execute(&mut *tx).await.unwrap();
let q = Selection { tenant, asset: None, agent: None, view: Some(View::Current), status: None,
hostname: "", last: Uuid::nil(), as_of: Utc::now(), limit: 1 };
let page = q.rows(&mut tx).await.unwrap();
assert_eq!(page[0].get::<Uuid,_>("asset_id"), vm);
let hosts = assigned_hosts(&mut tx, tenant, q.as_of, &page).await.unwrap();
assert_eq!(project(&page[0], q.as_of, false, &hosts)["hypervisor"]["hostname"], "hypervisor-1");
let options = hypervisors(&mut tx, tenant, q.as_of, None, "HYPERVISOR", 51).await.unwrap();
assert_eq!(options.len(), 1);
assert_eq!(options[0].asset_id, host);
sqlx::query("UPDATE assets SET archived_at=now() WHERE asset_id=$1")
.bind(host).execute(&mut *tx).await.unwrap();
let hosts = assigned_hosts(&mut tx, tenant, q.as_of, &page).await.unwrap();
assert_eq!(project(&page[0], q.as_of, false, &hosts)["hypervisor"]["kind"], "placeholder");
let result = sqlx::query("UPDATE assets SET hypervisor_asset_id=$2 WHERE asset_id=$1")
.bind(vm).bind(foreign_host).execute(&mut *tx).await.unwrap_err();
assert_eq!(result.as_database_error().unwrap().code().as_deref(), Some("23503"));
tx.rollback().await.unwrap();
}
#[tokio::test]
#[ignore = "requires the disposable RLS database from tests/local_access_database.py"]
async fn visibility_is_filtered_before_the_cursor_limit() {
@@ -481,7 +906,7 @@ mod tests {
limit: 1,
};
let row = q.rows(&mut tx).await.unwrap().pop().unwrap();
let summary = project(&row, q.as_of, false);
let summary = project(&row, q.as_of, false, &HashMap::new());
assert_eq!(summary["status"], "offline");
assert_eq!(summary["ram_total_bytes"], 1024);
assert_eq!(summary["cpu_model"], "Test CPU");
+31
View File
@@ -0,0 +1,31 @@
SELECT asset.asset_id, host.agent_id, inventory.data->>'hostname' AS hostname
FROM assets asset
JOIN LATERAL (
SELECT a.agent_id, a.revoked_at
FROM agent_installations a
WHERE a.tenant_id=asset.tenant_id AND a.asset_id=asset.asset_id AND a.created_at<=$2
AND NOT a.customer_transfer_pending
AND (a.customer_transfer_activated_at IS NULL OR a.customer_transfer_activated_at<=$2)
AND a.last_authenticated_at IS NOT NULL AND a.last_contact_at IS NOT NULL
ORDER BY a.enrollment_generation DESC, a.created_at DESC, a.agent_id
LIMIT 1
) host ON host.revoked_at IS NULL
JOIN LATERAL (
SELECT data FROM observations
WHERE tenant_id=asset.tenant_id AND agent_id=host.agent_id AND received_at<=$2
AND source_id IN ('linux.base','windows.base')
ORDER BY observed_at DESC, received_at DESC, sequence DESC
LIMIT 1
) inventory ON true
WHERE asset.tenant_id=$1 AND asset.archived_at IS NULL
AND ($3::uuid[] IS NULL OR asset.asset_id=ANY($3))
AND nullif(btrim(inventory.data->>'hostname'),'') IS NOT NULL
AND ($4='' OR strpos(lower(inventory.data->>'hostname'), lower($4))>0)
AND EXISTS (
SELECT FROM jsonb_array_elements(CASE WHEN jsonb_typeof(inventory.data->'observations')='array'
THEN inventory.data->'observations' ELSE '[]'::jsonb END) o
WHERE o->>'scope' IN ('linux.hardware','windows.hardware')
AND o->>'quality'='valid' AND o#>>'{data,device_type}'='hypervisor'
)
ORDER BY lower(inventory.data->>'hostname'), asset.asset_id
LIMIT $5
+12 -5
View File
@@ -292,7 +292,8 @@ fn validate_variants(variants: &[Variant]) -> Result<()> {
privileged_script_profile: None,
};
if v.encoding != "utf-8"
|| action.script_platform() != Some(v.os_family.as_str())
|| !(action.script_platform() == Some(v.os_family.as_str())
|| (v.os_family == "opnsense" && v.interpreter == "posix_sh"))
|| !action.validate()
{
return Err(fail("UNSUPPORTED_CAPABILITY"));
@@ -346,9 +347,11 @@ fn target_action(input: &PlanInput, caps: &[String]) -> Result<Action> {
let platform = match (
caps.iter().any(|c| c == "linux.base/1"),
caps.iter().any(|c| c == "windows.base/1"),
caps.iter().any(|c| c == "opnsense.base/1"),
) {
(true, false) => "linux",
(false, true) => "windows",
(true, false, false) => "linux",
(false, true, false) => "windows",
(false, false, true) => "opnsense",
_ => return Err(fail("UNSUPPORTED_CAPABILITY")),
};
let v = input
@@ -427,7 +430,7 @@ pub(crate) async fn plan_resolved(
let artifacts = crate::catalog::resolve_artifacts(&mut tx, tenant, &input.artifact_ids).await?;
let mut jobs = Vec::new();
for asset in ids {
let rows=sqlx::query("SELECT agent_id,enrollment_generation,assignment_version,capabilities FROM agent_installations WHERE asset_id=$1 AND revoked_at IS NULL ORDER BY agent_id FOR SHARE").bind(asset).fetch_all(&mut *tx).await?;
let rows=sqlx::query("SELECT agent_id,enrollment_generation,assignment_version,capabilities FROM agent_installations WHERE asset_id=$1 AND revoked_at IS NULL AND NOT customer_transfer_pending AND NOT EXISTS (SELECT FROM device_customer_transfers t WHERE t.source_agent_id=agent_installations.agent_id AND t.completed_at IS NULL AND t.cancelled_at IS NULL) ORDER BY agent_id FOR SHARE").bind(asset).fetch_all(&mut *tx).await?;
if rows.len() != 1 {
return Err(fail(if rows.is_empty() {
"NOT_FOUND"
@@ -438,6 +441,10 @@ pub(crate) async fn plan_resolved(
let row = &rows[0];
let caps = capabilities(row.get("capabilities"))?;
let action = target_action(&input, &caps)?;
if input.execution_identity=="agent_service" && caps.iter().any(|s| s=="opnsense.base/1") && !caps.iter().any(|s| s=="opnsense.posix/1") &&
(input.max_runtime_seconds != 10 || input.max_output_bytes != 1024) {
return Err(fail("UNSUPPORTED_CAPABILITY"));
}
if !action.valid_job_limits(input.max_runtime_seconds, input.max_output_bytes) {
return Err(fail("VALIDATION_ERROR"));
}
@@ -683,7 +690,7 @@ async fn recheck_for(
if job.payload.start_before <= Utc::now() {
return Err(fail("PLAN_EXPIRED"));
}
let r=sqlx::query("SELECT enrollment_generation,assignment_version,capabilities FROM agent_installations WHERE agent_id=$1 AND asset_id=$2 AND revoked_at IS NULL FOR SHARE").bind(job.payload.agent_id).bind(job.payload.asset_id).fetch_optional(&mut **tx).await?.ok_or_else(||fail("AUTHORIZATION_REVOKED"))?;
let r=sqlx::query("SELECT enrollment_generation,assignment_version,capabilities FROM agent_installations WHERE agent_id=$1 AND asset_id=$2 AND revoked_at IS NULL AND NOT customer_transfer_pending AND NOT EXISTS (SELECT FROM device_customer_transfers t WHERE t.source_agent_id=agent_installations.agent_id AND t.completed_at IS NULL AND t.cancelled_at IS NULL) FOR SHARE").bind(job.payload.agent_id).bind(job.payload.asset_id).fetch_optional(&mut **tx).await?.ok_or_else(||fail("AUTHORIZATION_REVOKED"))?;
if r.get::<i64, _>("assignment_version") as u64 != job.payload.assignment_version
|| r.get::<i64, _>("enrollment_generation") as u64 != job.payload.enrollment_generation
{
+29
View File
@@ -492,6 +492,8 @@ mod tests {
"rollout":{"max_parallel":1,"halt_after_failures":1},"reason":"test customer authorization"});
for request in [
client.get(format!("{base}/{b}/devices/{asset}")),
client.patch(format!("{base}/{b}/devices/{asset}/purpose"))
.json(&json!({"purpose":"foreign write"})),
client.delete(format!("{base}/{b}/devices/{asset}?agent_id={id}")),
client
.post(format!("{base}/{b}/operation-plans"))
@@ -530,6 +532,14 @@ mod tests {
.send().await.unwrap().json::<Value>().await.unwrap();
let deletion = || client.delete(&pair).header("origin", origin).header("cookie", cookie);
let csrf = session["csrf_token"].as_str().unwrap();
let detail = format!("{base}/{a}/devices/{asset}");
let purpose = format!("{detail}/purpose");
let update = |text: &str| client.patch(&purpose).header("origin", origin)
.header("cookie", cookie).json(&json!({"purpose":text}));
assert_eq!(client.patch(&purpose).bearer_auth(&token).json(&json!({"purpose":"token write"}))
.send().await.unwrap().status(), StatusCode::FORBIDDEN);
assert_eq!(update("reader denied").header("x-csrf-token", csrf)
.send().await.unwrap().status(), StatusCode::FORBIDDEN);
assert_eq!(deletion().header("x-csrf-token", csrf).send().await.unwrap().status(), StatusCode::FORBIDDEN);
sqlx::query("UPDATE memberships SET permissions=array_append(permissions,'devices:revoke') WHERE tenant_id=$1 AND principal=$2")
.bind(a).bind(&issuer).execute(&admin).await.unwrap();
@@ -537,6 +547,25 @@ mod tests {
.bind(a).bind(asset).execute(&admin).await.unwrap();
sqlx::query("INSERT INTO agent_installations(tenant_id,agent_id,asset_id,certificate_sha256) VALUES($1,$2,$3,$4)")
.bind(a).bind(id).bind(asset).bind(hash(id.as_bytes())).execute(&admin).await.unwrap();
assert_eq!(update("missing CSRF").send().await.unwrap().status(), StatusCode::FORBIDDEN);
assert_eq!(update(&"ü".repeat(501)).header("x-csrf-token", csrf)
.send().await.unwrap().status(), StatusCode::BAD_REQUEST);
let text = "Dateiserver für Nürnberg\nNur Entwicklung";
let saved: Value = update(" Dateiserver für Nürnberg\r\nNur Entwicklung ")
.header("x-csrf-token", csrf).send().await.unwrap().error_for_status().unwrap().json().await.unwrap();
assert_eq!(saved["purpose"], text);
let device: Value = client.get(&detail).bearer_auth(&token)
.send().await.unwrap().error_for_status().unwrap().json().await.unwrap();
assert_eq!(device["purpose"], text);
let other: Option<String> = sqlx::query_scalar("SELECT purpose FROM assets WHERE tenant_id=$1 AND asset_id=$2")
.bind(b).bind(asset).fetch_one(&admin).await.unwrap();
assert!(other.is_none());
let cleared: Value = update(" \n ").header("x-csrf-token", csrf)
.send().await.unwrap().error_for_status().unwrap().json().await.unwrap();
assert_eq!(cleared["purpose"], Value::Null);
let events: Vec<Value> = sqlx::query_scalar("SELECT data->'decision' FROM audit_events WHERE tenant_id=$1 AND event_type='device_purpose_updated' ORDER BY created_at")
.bind(a).fetch_all(&admin).await.unwrap();
assert_eq!(events, [json!({"asset_id":asset,"cleared":false}), json!({"asset_id":asset,"cleared":true})]);
assert_eq!(deletion().send().await.unwrap().status(), StatusCode::FORBIDDEN);
assert_eq!(deletion().header("x-csrf-token",csrf).send().await.unwrap().status(), StatusCode::CONFLICT);
let revoked = client.post(format!("{base}/{a}/devices/{asset}/revocation?agent_id={id}"))
+5 -2
View File
@@ -13,6 +13,7 @@ mod auth;
mod local_auth;
mod mcp_tokens;
mod customers;
mod customer_transfer;
mod catalog;
mod checkpoint_repair;
mod controls;
@@ -26,6 +27,7 @@ mod monitoring;
mod oidc;
mod pagination;
mod permissions;
mod proxmox_links;
mod root_policy;
mod managed_scripts;
mod scheduler;
@@ -174,7 +176,7 @@ async fn main() -> Result<()> {
}
let mut routes = Router::new().route("/healthz", get(|| async { "ok" }));
if role.gateway() {
routes = agent_updates::gateway_routes(routes);
routes = customer_transfer::gateway_routes(agent_updates::gateway_routes(routes));
routes = agent_update_health::gateway_routes(routes);
routes = agent_time::gateway_routes(routes);
routes = routes
@@ -216,12 +218,13 @@ async fn main() -> Result<()> {
.route("/api/v1/tenants/{tenant}", axum::routing::patch(customers::rename))
.route("/api/v1/devices", get(devices::all))
.route("/api/v1/tenants/{tenant}/devices", get(devices::page))
.route("/api/v1/tenants/{tenant}/hypervisors", get(devices::hypervisor_options))
.route("/api/v1/tenants/{tenant}/devices/{asset}", get(devices::detail));
routes = mcp_tokens::routes(routes);
routes = routes.route("/auth/local/login", post(auth::local_login));
routes = jobs::routes(routes);
routes = script_library::routes(routes);
routes = controls::routes(routes);
routes = customer_transfer::routes(controls::routes(routes));
routes = catalog::routes(routes);
routes = agent_updates::routes(routes);
routes = routes
+10 -3
View File
@@ -3,7 +3,7 @@
use crate::{jobs::Error, root_policy::RootScope, windows_policy::SystemScope};
use axum::http::StatusCode;
use rmm_domain::{
agent_update::{ArtifactRole, Platform, ReleaseManifest, digest},
agent_update::{ArtifactRole, LINUX_ARM64_RELEASE_CAPABILITY, Platform, ReleaseManifest, digest},
jobs::{Action, FULL_MAINTENANCE_CAPABILITY, JobPayload},
};
use serde::Deserialize;
@@ -16,6 +16,8 @@ use uuid::Uuid;
struct Policy {
schema_version: u8,
linux: Vec<RootScope>,
#[serde(default)]
opnsense: Vec<RootScope>,
windows: Vec<SystemScope>,
#[serde(default)]
release_anchors: Vec<ReleaseAnchor>,
@@ -60,7 +62,7 @@ impl ReleaseAnchor {
};
let field = |name: &str, value: &str| format!(r#""{name}":"{value}""#);
match manifest.platform {
Platform::LinuxX64 => replace(
Platform::LinuxX64 | Platform::LinuxArm64 => replace(
field("helper_binary_sha256", &self.helper_sha256),
field("helper_binary_sha256", next(ArtifactRole::Helper)?),
1,
@@ -123,6 +125,8 @@ impl Policy {
return false;
}
let permits_subject = |subject: &str| match job.execution_identity.as_str() {
"root" if caps.iter().any(|c| c=="opnsense.base/1") => self.opnsense.iter()
.any(|scope| scope.permits(subject,job,caps,None)),
"root" => self
.linux
.iter()
@@ -164,6 +168,9 @@ fn check_pair(
caps: &[String],
) -> Result<(), Error> {
let platform = match identity {
"root" if caps.iter().any(|cap| cap == LINUX_ARM64_RELEASE_CAPABILITY) => {
Platform::LinuxArm64
}
"root" => Platform::LinuxX64,
"system" => Platform::WindowsGnuX64,
_ => return Err(binding_error()),
@@ -203,7 +210,7 @@ pub(crate) async fn permits(
.map_err(|_| invalid())?;
let policy: Policy = serde_json::from_slice(&bytes).map_err(|_| invalid())?;
if policy.schema_version != 1
|| policy.linux.len() + policy.windows.len() > 64
|| policy.linux.len() + policy.windows.len() + policy.opnsense.len() > 64
|| policy.release_anchors.len() > 64
|| policy
.release_anchors
@@ -0,0 +1,28 @@
-- Current devices of customer $1 linked to host $2, with the MACs they report themselves.
SELECT asset.asset_id, inst.agent_id, asset.hypervisor_source,
inventory.data->>'hostname' AS hostname,
COALESCE((
SELECT jsonb_agg(DISTINCT lower(i->>'mac'))
FROM jsonb_array_elements(CASE WHEN jsonb_typeof(inventory.data->'observations')='array'
THEN inventory.data->'observations' ELSE '[]'::jsonb END) o,
jsonb_array_elements(CASE
WHEN jsonb_typeof(o->'data')='array' THEN o->'data'
WHEN jsonb_typeof(o#>'{data,entries}')='array' THEN o#>'{data,entries}'
ELSE '[]'::jsonb END) i
WHERE o->>'scope' IN ('linux.interfaces','windows.interfaces') AND o->>'quality'='valid'
AND i->>'mac' IS NOT NULL
), '[]'::jsonb) AS macs
FROM assets asset
JOIN LATERAL (
SELECT a.agent_id, a.revoked_at FROM agent_installations a
WHERE a.tenant_id=asset.tenant_id AND a.asset_id=asset.asset_id AND NOT a.customer_transfer_pending
ORDER BY a.enrollment_generation DESC, a.created_at DESC, a.agent_id LIMIT 1
) inst ON inst.revoked_at IS NULL
LEFT JOIN LATERAL (
SELECT data FROM observations
WHERE tenant_id=asset.tenant_id AND agent_id=inst.agent_id AND source_id IN ('linux.base','windows.base')
ORDER BY observed_at DESC, received_at DESC, sequence DESC LIMIT 1
) inventory ON true
WHERE asset.tenant_id=$1 AND asset.hypervisor_asset_id=$2 AND asset.archived_at IS NULL
ORDER BY asset.asset_id
LIMIT 1024
+24
View File
@@ -0,0 +1,24 @@
//! Automatic VM -> Proxmox host mapping from the hosts' `linux.proxmox` guest lists.
//! Runs inside the ingesting customer transaction behind a savepoint: a failed
//! reconciliation never blocks acknowledging the inventory snapshot itself.
use sqlx::{Acquire, Postgres, Transaction};
use uuid::Uuid;
pub async fn reconcile(tx: &mut Transaction<'_, Postgres>, tenant: Uuid) {
let result = async {
let mut savepoint = tx.begin().await?;
let events: Vec<Uuid> = sqlx::query_scalar(include_str!("proxmox_links.sql"))
.bind(tenant)
.fetch_all(&mut *savepoint)
.await?;
savepoint.commit().await?;
Ok::<_, sqlx::Error>(events.len())
}
.await;
match result {
Ok(0) => (),
Ok(changed) => eprintln!("proxmox guest links updated: {changed}"),
// Database detail stays out of the log line; the snapshot is still acknowledged.
Err(_) => eprintln!("proxmox guest link reconciliation skipped"),
}
}
+103
View File
@@ -0,0 +1,103 @@
-- Automatic VM -> Proxmox host links for one customer ($1). A link is set only for a
-- unique MAC match between a host's current guest list and a current virtual device of
-- the same customer. Manual decisions (hypervisor_source='manual') are never touched.
WITH current_installation AS (
SELECT asset.asset_id, asset.hypervisor_asset_id, asset.hypervisor_source, inst.agent_id
FROM assets asset
JOIN LATERAL (
SELECT a.agent_id, a.revoked_at
FROM agent_installations a
WHERE a.tenant_id=asset.tenant_id AND a.asset_id=asset.asset_id
AND NOT a.customer_transfer_pending
AND a.last_authenticated_at IS NOT NULL
ORDER BY a.enrollment_generation DESC, a.created_at DESC, a.agent_id
LIMIT 1
) inst ON inst.revoked_at IS NULL
WHERE asset.tenant_id=$1 AND asset.archived_at IS NULL
),
latest AS (
SELECT c.*, inventory.data
FROM current_installation c
JOIN LATERAL (
SELECT data FROM observations
WHERE tenant_id=$1 AND agent_id=c.agent_id AND source_id IN ('linux.base','windows.base')
ORDER BY observed_at DESC, received_at DESC, sequence DESC
LIMIT 1
) inventory ON true
),
scopes AS (
SELECT l.asset_id, o
FROM latest l,
jsonb_array_elements(CASE WHEN jsonb_typeof(l.data->'observations')='array'
THEN l.data->'observations' ELSE '[]'::jsonb END) o
WHERE o->>'quality'='valid'
),
proxmox_hosts AS (
SELECT asset_id, o->'data' AS data FROM scopes WHERE o->>'scope'='linux.proxmox'
),
guest_macs AS (
SELECT DISTINCT h.asset_id AS host_asset_id, lower(mac) AS mac
FROM proxmox_hosts h,
jsonb_array_elements(h.data->'guests') g,
jsonb_array_elements_text(g->'macs') mac
),
virtual_devices AS (
SELECT asset_id FROM scopes
WHERE o->>'scope' IN ('linux.hardware','windows.hardware') AND o#>>'{data,device_type}'='virtual'
),
device_macs AS (
SELECT DISTINCT s.asset_id, lower(i->>'mac') AS mac
FROM scopes s
JOIN virtual_devices v ON v.asset_id=s.asset_id,
jsonb_array_elements(CASE
WHEN jsonb_typeof(s.o->'data')='array' THEN s.o->'data'
WHEN jsonb_typeof(s.o#>'{data,entries}')='array' THEN s.o#>'{data,entries}'
ELSE '[]'::jsonb END) i
WHERE s.o->>'scope' IN ('linux.interfaces','windows.interfaces')
AND i->>'mac' ~* '^[0-9a-f]{2}(:[0-9a-f]{2}){5}$'
AND lower(i->>'mac') <> '00:00:00:00:00:00'
),
-- A MAC that more than one device or more than one host claims proves nothing.
unique_macs AS (
SELECT d.asset_id, g.host_asset_id
FROM device_macs d JOIN guest_macs g ON g.mac=d.mac
WHERE d.asset_id <> g.host_asset_id
AND (SELECT count(*) FROM device_macs d2 WHERE d2.mac=d.mac)=1
AND (SELECT count(*) FROM guest_macs g2 WHERE g2.mac=d.mac)=1
),
matches AS (
SELECT asset_id, min(host_asset_id::text)::uuid AS host_asset_id
FROM unique_macs GROUP BY asset_id HAVING count(DISTINCT host_asset_id)=1
),
desired AS (
-- New or changed automatic links.
SELECT l.asset_id, l.hypervisor_asset_id AS previous, m.host_asset_id AS next
FROM latest l JOIN matches m ON m.asset_id=l.asset_id
WHERE l.hypervisor_source IS DISTINCT FROM 'manual'
AND (l.hypervisor_asset_id IS DISTINCT FROM m.host_asset_id OR l.hypervisor_source IS NULL)
UNION ALL
-- The assigned host reports a valid guest list that no longer contains this device.
SELECT l.asset_id, l.hypervisor_asset_id, NULL
FROM latest l
WHERE l.hypervisor_source='proxmox'
AND NOT EXISTS (SELECT FROM matches m WHERE m.asset_id=l.asset_id)
AND EXISTS (SELECT FROM proxmox_hosts h WHERE h.asset_id=l.hypervisor_asset_id)
AND EXISTS (SELECT FROM device_macs d WHERE d.asset_id=l.asset_id)
),
changed AS (
UPDATE assets a
SET hypervisor_asset_id=d.next,
hypervisor_source=CASE WHEN d.next IS NULL THEN NULL ELSE 'proxmox' END
FROM desired d
WHERE a.tenant_id=$1 AND a.asset_id=d.asset_id
RETURNING a.asset_id, d.previous, d.next
)
INSERT INTO audit_events(tenant_id,event_id,principal,event_type,data,created_at)
SELECT $1, gen_random_uuid(), 'system:proxmox-guest-detection',
CASE WHEN next IS NULL THEN 'device_hypervisor_released' ELSE 'device_hypervisor_detected' END,
jsonb_build_object('asset_id',asset_id,'previous_hypervisor_asset_id',previous,
'hypervisor_asset_id',next,'match','mac'),
now()
FROM changed
WHERE previous IS DISTINCT FROM next
RETURNING event_id
+3
View File
@@ -380,6 +380,9 @@ pub async fn authorize(
if crate::managed_scripts::permits(tx, principal, job, &caps).await? {
return Ok(());
}
if caps.iter().any(|c| c=="opnsense.base/1") {
return Err(Error(StatusCode::FORBIDDEN,"SCRIPT_HELPER_SCOPE_DENIED"));
}
let policy = load()?;
let global_log_units = if matches!(job.action, Action::QueryLogs { .. }) {
parse_log_units(std::env::var("RMM_ROOT_LOG_UNITS").ok().as_deref())?
+3 -11
View File
@@ -218,11 +218,7 @@ fn current_keyring(file: &UpdateKeyringFile) -> Result<UpdateKeyring> {
.map_err(|_| anyhow::anyhow!("UPDATE_HISTORY_KEY_SIZE"))?;
keys.push(TrustedUpdateKey::new(
public,
match key.platform {
Platform::LinuxX64 => ReleasePlatform::LinuxX64,
Platform::WindowsX64 => ReleasePlatform::WindowsX64,
Platform::WindowsGnuX64 => ReleasePlatform::WindowsGnuX64,
},
key.platform.into(),
key.first_sequence,
key.last_sequence,
key.revoked,
@@ -307,11 +303,7 @@ fn verify_record_release(
.release_envelope
.decode()
.map_err(anyhow::Error::msg)?;
let platform = match record.platform {
Platform::LinuxX64 => ReleasePlatform::LinuxX64,
Platform::WindowsX64 => ReleasePlatform::WindowsX64,
Platform::WindowsGnuX64 => ReleasePlatform::WindowsGnuX64,
};
let platform = ReleasePlatform::from(record.platform);
let envelope: &ReleaseEnvelope = &record.release_envelope;
verify_release(
SignedRelease {
@@ -403,7 +395,7 @@ fn verify_update(
ensure!(
job.execution_identity
== match record.platform {
Platform::LinuxX64 => "root",
Platform::LinuxX64 | Platform::LinuxArm64 => "root",
Platform::WindowsX64 | Platform::WindowsGnuX64 => "system",
}
&& update.release_envelope == record.release_envelope
+1 -1
View File
@@ -45,7 +45,7 @@ mod linux {
let args: Vec<_> = std::env::args_os().skip(1).collect();
if args.first().is_some_and(|arg| arg == "--verify-bootstrap") {
let release =
rmm_updater::verify_bootstrap_files(rmm_updater::Platform::LinuxX64, &args[1..])?;
rmm_updater::verify_bootstrap_files(rmm_domain::agent_update::Platform::NATIVE_LINUX.into(), &args[1..])?;
println!("{}", serde_json::to_string(release.manifest())?);
return Ok(());
}
+2 -7
View File
@@ -4,7 +4,7 @@ use crate::{
VerifiedRelease,
};
use anyhow::{Context, Result, ensure};
use rmm_domain::agent_update::{Platform as WirePlatform, ReleaseEnvelope, UpdateKeyringFile};
use rmm_domain::agent_update::{ReleaseEnvelope, UpdateKeyringFile};
use std::{
ffi::OsString,
fs::File,
@@ -31,18 +31,13 @@ pub fn verify_bootstrap_files(platform: Platform, args: &[OsString]) -> Result<V
.keys
.into_iter()
.map(|key| {
let platform = match key.platform {
WirePlatform::LinuxX64 => Platform::LinuxX64,
WirePlatform::WindowsX64 => Platform::WindowsX64,
WirePlatform::WindowsGnuX64 => Platform::WindowsGnuX64,
};
let raw = hex::decode(key.public_key_hex)?
.try_into()
.ok()
.context("Ed25519 public key length")?;
TrustedUpdateKey::new(
raw,
platform,
key.platform.into(),
key.first_sequence,
key.last_sequence,
key.revoked,
+14
View File
@@ -16,12 +16,26 @@ pub const RELEASE_SIGNATURE_DOMAIN: &[u8] = b"rmm.agent-release.v1\0";
pub enum Platform {
#[serde(rename = "x86_64-unknown-linux-gnu")]
LinuxX64,
#[serde(rename = "aarch64-unknown-linux-gnu")]
LinuxArm64,
#[serde(rename = "x86_64-pc-windows-msvc")]
WindowsX64,
#[serde(rename = "x86_64-pc-windows-gnu")]
WindowsGnuX64,
}
impl From<rmm_domain::agent_update::Platform> for Platform {
fn from(platform: rmm_domain::agent_update::Platform) -> Self {
use rmm_domain::agent_update::Platform as WirePlatform;
match platform {
WirePlatform::LinuxX64 => Self::LinuxX64,
WirePlatform::LinuxArm64 => Self::LinuxArm64,
WirePlatform::WindowsX64 => Self::WindowsX64,
WirePlatform::WindowsGnuX64 => Self::WindowsGnuX64,
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ArtifactRole {
+28 -2
View File
@@ -66,11 +66,18 @@ fn manifest() -> ReleaseManifest {
}
fn verify_bytes(payload: &[u8], context: &TrustedContext) -> anyhow::Result<VerifiedRelease> {
verify_bytes_with_key(payload, context, key(false, 1, 20))
}
fn verify_bytes_with_key(
payload: &[u8],
context: &TrustedContext,
pinned: TrustedUpdateKey,
) -> anyhow::Result<VerifiedRelease> {
let signer = SigningKey::from_bytes(&[41; 32]);
let mut message = RELEASE_SIGNATURE_DOMAIN.to_vec();
message.extend_from_slice(payload);
let signature = signer.sign(&message).to_bytes();
let pinned = key(false, 1, 20);
let key_id = pinned.key_id().to_owned();
verify_release(
SignedRelease {
@@ -166,7 +173,7 @@ fn signed_noncanonical_unknown_duplicate_and_oversize_payloads_are_rejected() {
payload.replacen("{", "{\"schema_version\":1,", 1),
payload.replacen("\"bytes\":11", "\"bytes\":11,\"path\":\"/etc/passwd\"", 1),
payload.replacen("\"release_sequence\":4", "\"release_sequence\":4.0", 1),
payload.replacen("x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", 1),
payload.replacen("x86_64-unknown-linux-gnu", "riscv64gc-unknown-linux-gnu", 1),
] {
assert!(verify_bytes(changed.as_bytes(), &context()).is_err());
}
@@ -174,6 +181,25 @@ fn signed_noncanonical_unknown_duplicate_and_oversize_payloads_are_rejected() {
assert!(verify_bytes(b"", &context()).is_err());
}
#[test]
fn arm64_release_requires_arm64_context_and_key() {
let mut target = manifest();
target.platform = Platform::LinuxArm64;
let mut context = context();
assert!(verify(&target, &context).is_err());
context.platform = Platform::LinuxArm64;
assert!(verify(&target, &context).is_err());
let arm64_key = TrustedUpdateKey::new(
SigningKey::from_bytes(&[41; 32]).verifying_key().to_bytes(),
Platform::LinuxArm64,
1,
20,
false,
).unwrap();
let payload = serde_json::to_vec(&target).unwrap();
assert!(verify_bytes_with_key(&payload, &context, arm64_key).is_ok());
}
#[test]
fn incompatible_clock_platform_state_protocol_and_trust_contexts_are_rejected() {
let mut ctx = context();
+24
View File
@@ -0,0 +1,24 @@
# OPNsense-Testclient
Test-VM `discovery-opnsense` auf `qemu:///system`, 2 vCPU, 1536 MiB RAM, 12 GiB virtuelle Platte. Basis: offizielles Nano-Image OPNsense 26.7 amd64; SHA-256 des komprimierten Images `28d5e2f37e40d87468a924e3006ef10e2ddc6de485b85333d9e3958c84d0cb9d`, gegen die Herstellerankündigung verglichen und die Signatur des entpackten Images geprüft. Start bei Bedarf: `virsh -c qemu:///system start discovery-opnsense`. Konsole: `virsh -c qemu:///system console discovery-opnsense`. Das LAN `192.168.247.0/24` ist eine isolierte libvirt-Bridge, das WAN `192.168.246.0/24` verwendet NAT. Management vom Notebook: `https://192.168.247.1`. Der Hostpin stammt aus der seriellen Konsole; Zugangswerte liegen ausschließlich im privaten Zugangsspeicher des ursprünglichen Checkouts.
## Quelle und Entwicklungsinstallation
Der OPNsense-eigene Python-Interpreter führt zwei Zipapps aus. Der lokale Collector läuft mit den für PF, VPN-Sockets und Konfiguration nötigen Rechten. Er veröffentlicht ausschließlich den ausgewählten Bestand atomar in `/var/db/discovery-observations/inventory.json`. Das Dienstkonto `discovery` kann die Datei lesen, aber weder Collector noch Cache verändern. Nur dieses Konto hält die Geräteidentität und nutzt den ausgehenden HTTPS-Kanal. Für die Erfassung wird keine eingehende Netzwerkverbindung benötigt.
1. In einer Build-venv `grpcio-tools==1.76.0` und `protobuf==6.33.5` installieren. `python deploy/opnsense/build.py ARTIFAKTORDNER` generiert die Protobuf-Bindung aus `proto/agent.proto` und packt die portable Python-Laufzeit mit ein. `python deploy/opnsense/test_agent.py ARTIFAKTORDNER/agent.pyz` prüft die Wiederholung einer nicht bestätigten Inventarmeldung mit unveränderter Nachrichtenidentität.
2. Über den gepinnten Entwicklungszugang `agent.pyz`, `collect.pyz`, `discovery_agent` und den **bestätigten öffentlichen CA-Anker des Gerätegateways** als `ca.pem` in einen privaten Stagingordner im Gast übertragen. `sh install-dev.sh STAGINGORDNER` auf dem Gast ausführen. Diese Strecke ist eine direkte Entwicklungsbereitstellung, kein signierter Releasekanal.
3. Bei einer frischen Installation in der Discovery-WebUI den Testkunden wählen und einen Einmalcode erzeugen. Den Code über geschützte Standardeingabe an `su -m discovery -c '/usr/local/bin/python3 -I /usr/local/libexec/discovery-agent/agent.pyz enroll'` übergeben. Danach `service discovery_agent start`. Bei vorhandener Identität nicht neu enrollen. Der Dienst verwaltet beide Prozesse; `rc.conf.local` aktiviert sie beim Start.
4. In **Geräte** den Hostnamen öffnen. Das Inventar wird ungefähr einmal pro Minute erhoben, der Heartbeat etwa alle 20 Sekunden gesendet. Das Betriebssystem liefert CPU-Zeitdifferenzen und eine RAM-Schätzung aus freien und inaktiven Seiten. Fehlgeschlagene Quellen erscheinen als „Nicht ermittelbar“, ein erfolgreicher leerer Befund gesondert. Pro Liste werden höchstens 128 Zeilen gemeldet; Auszüge sind gekennzeichnet.
**Systeminventar:** Kern-Bootzeit, VM-Erkennung, verfügbare SMBIOS-Daten, Netzwerkkarten mit MAC und IPs, angemeldete Benutzer und lokale FreeBSD-Pakete mit Version. Bei Software liegt die Obergrenze bei 1024 Paketen.
**VPN:** Beschreibungen und Aktivierung der OPNsense-Modelle für WireGuard-Instanzen/-Peers, OpenVPN-Instanzen und IPsec-Verbindungen. Separat: WireGuard-Handshakes (kein dauerhafter Online-Status), OpenVPN-Managementstatus und IKE-Sicherheitsassoziationen. Ohne erreichbaren IPsec-Dienst ist der Laufzeitstatus unbekannt. Keine privaten Schlüssel, PSKs, Zertifikate oder vollständige Gerätekonfiguration im Inventar.
**Firewall:** aktiver PF-Hauptregelsatz in Kernelreihenfolge, einschließlich automatisch erzeugter Regeln. Kein Editor und kein rekursives Auslesen der Anchor-Regelsätze.
**Skripte:** Unter **Aufträge → Neuer Auftrag** die OPNsense-Zielgeräte wählen. Unter **Aus der Bibliothek** eine feste POSIX-Shell-Revision wählen, alternativ ein **Einmaliges Skript** eingeben. **Ausführen als** unterstützt Dienstkonto und bei entsprechender Berechtigung `root`. **Zusammenfassung ansehen** bindet das Skript samt Zielen und Grenzen; **Zur Genehmigung einreichen** legt den Antrag an. Die persönliche Freigabe erfolgt unter **Genehmigungen**, die Ausgabe erscheint im Auftrag. OPNsense kann POSIX-Shellskripte ausführen; Linux-spezifische Programme und Optionen werden nicht übersetzt. Zwei [Startrevisionen aus dem Repository](../../migrations/0047_opnsense_library.sql) zeigen Version/Ausführungskonto bzw. aktive Firewallregeln. Im Gerätedetail stehen Inventar und Verweise auf vorhandene Aufträge.
**Probe bereitstellen:** Im Gast steht die gepinnte Toolchain unter `/opt/discovery-rust`. `cargo build --locked -p rmm-opnsense-jobs` nativ bauen und `rmm-opnsense-jobs` root-eigen (0555) neben den Zipapps installieren. Auf der bereits aufgenommenen Testinstallation `python3 -I pin-probe.py TENANT ASSET AGENT ENROLLMENT_GENERATION ASSIGNMENT_VERSION` mit den bestätigten Kennungen aufrufen. Das Skript bezieht den öffentlichen Jobschlüssel über das bestehende Gerätezertifikat und überschreibt keine andere Bindung. Den Agentendienst anschließend neu starten. Der Worker nutzt bestehende Nachrichtenfolgen, den signierten Abgleich und getrennte dauerhafte Start-/Ergebnisjournale. Er läuft ohne Netzwerklistener als `discovery`. Der [native Integrationstest](../../crates/agent-opnsense-jobs/src/native/tests.rs) prüft mit eigener Testidentität und eigenem Schlüssel die Ablehnung einer defekten Signatur, erwartete Ausgabe, einen Verbrauch und keine zweite Ausführung bei Wiederzustellung. Das ist ein separater Beleg vom persönlich genehmigten Geräteauftrag.
**Root-Helper (ab `0.1.0-opnsense.6`):** Native Binärdatei zusätzlich in den privaten Stagingordner kopieren. Auf dem Gast `python3 -I enable-root.py /absoluter/Stagingpfad/rmm-opnsense-jobs` und `sh install-dev.sh /absoluter/Stagingpfad` ausführen. Die lokale Policy bindet Worker-Hash, Gerätebindung, Dienst-UID und Gruppe. Der zusätzliche `rc.d`-Prozess läuft als root, hört ausschließlich auf einem geschützten Unix-Socket und prüft den Peer über `getpeereid`. Den vom Helper bestätigten Binär- und Policyhash mit `tools/dev_opnsense_root_scope.py --help` für die genaue Kopplung und den berechtigten lokalen Principal registrieren. Es gibt keine automatische Freigabe oder automatische Pinrotation. Root akzeptiert `posix_sh` mit `full_maintenance_v1` (bis 300 Sekunden und 65536 Ausgabebyte), verifiziert selbst Sync, Grant, Skriptbytes, Boot und Uhr, und führt ein eigenes Verbrauchsjournal in `/var/db/discovery-root-jobs`. Ohne Startbeleg gibt es kein positives Ergebnis. Bei `unknown` zuerst die bestehenden Journale abgleichen. Der [isolierte native Helper-Test](test_root_helper.py) benutzt nur eine Wegwerfidentität und separate Journale.
+317
View File
@@ -0,0 +1,317 @@
"""Unprivileged OPNsense telemetry and signed job transport."""
import hashlib
import contextlib
import fcntl
import http.client
import json
import os
from pathlib import Path
import re
import sqlite3
import ssl
import subprocess
import sys
import time
import uuid
import agent_pb2 as pb
STATE = Path('/var/db/discovery-agent')
CA = '/usr/local/etc/discovery-agent/ca.pem'
HOST = 'discovery.ease-technologies.de'
MEDIA = 'application/x-protobuf'
VERSION = '0.1.0-opnsense.6'
WORKER = Path('/usr/local/libexec/discovery-agent/rmm-opnsense-jobs')
BINDING = Path('/usr/local/etc/discovery-agent/job-binding.json')
def post(ctx, path, data, media):
conn = http.client.HTTPSConnection(HOST, context=ctx, timeout=15)
try:
conn.request('POST', '/agent/v1/'+path, data, {'Content-Type':media, 'Accept':media})
res = conn.getresponse()
if res.status != 200 or res.getheader('Content-Type', '').split(';')[0] != media:
raise ValueError(f'{path}: HTTP {res.status}')
body = res.read(1024*1024 + 1)
if len(body) > 1024*1024:
raise ValueError('response size')
return body
finally:
conn.close()
def context():
return ssl.create_default_context(cafile=CA)
def enroll():
# Existing keys are never overwritten, including after an ambiguous POST.
key = STATE/'device.key'
if key.exists() or (STATE/'identity.json').exists():
raise RuntimeError('installation already initialized')
token = sys.stdin.readline(128).strip()
if not re.fullmatch('[0-9a-f]{64}', token):
raise ValueError('invalid enrollment code')
csr = subprocess.check_output(['/usr/bin/openssl','req','-new','-newkey','ec','-pkeyopt',
'ec_paramgen_curve:P-256','-nodes','-subj','/CN=discovery-agent','-keyout',str(key)],
stderr=subprocess.DEVNULL)
result = json.loads(post(context(), 'enroll', json.dumps(dict(token=token, csr_pem=csr.decode())),
'application/json'))
for name in ('agent_id','asset_id'):
uuid.UUID(result[name])
(STATE/'device.pem').write_text(result.pop('certificate_pem'))
context().load_cert_chain(STATE/'device.pem', key)
(STATE/'identity.json').write_text(json.dumps(result))
def durable_write(path, data):
tmp = path.with_suffix('.new')
with tmp.open('wb') as f:
f.write(data)
f.flush()
os.fsync(f.fileno())
os.replace(tmp, path)
fd = os.open(STATE, os.O_DIRECTORY)
try:
os.fsync(fd)
finally:
os.close(fd)
def renew(ctx, identity):
# Reuse a durable request after a lost reply; gateway renewal is idempotent
# for the same CSR. Keep the currently working certificate until key match.
cert, key, csr = (STATE/n for n in ('device.pem','device.key','renewal.csr'))
if not csr.exists():
check = subprocess.run(['/usr/bin/openssl','x509','-in',str(cert),'-noout','-checkend','604800'],
stdout=subprocess.DEVNULL, check=False)
if check.returncode == 0:
return False
durable_write(csr,subprocess.check_output(['/usr/bin/openssl','req','-new','-key',str(key),
'-subj','/CN=discovery-agent']))
result = json.loads(post(ctx,'certificates/renew',json.dumps({'csr_pem':csr.read_text()}),'application/json'))
if result['agent_id'] != identity['agent_id']:
raise ValueError('renewal identity mismatch')
candidate = STATE/'candidate.pem'
durable_write(candidate,result['certificate_pem'].encode())
context().load_cert_chain(candidate,key)
durable_write(cert,candidate.read_bytes())
candidate.unlink()
csr.unlink()
return True
def usage(previous):
def counters(*keys):
return [int(x) for x in subprocess.check_output(['/sbin/sysctl', '-n', *keys],timeout=3).split()]
ticks = counters('kern.cp_time')
free, inactive, page, total = counters('vm.stats.vm.v_free_count','vm.stats.vm.v_inactive_count',
'hw.pagesize','hw.physmem')
sample = pb.SystemUsage(ram_available_bytes=min((free+inactive)*page,total),ram_total_bytes=total)
if previous and len(previous) == len(ticks) and all(x >= y for x,y in zip(ticks,previous)):
delta = sum(ticks)-sum(previous)
if delta > 0:
sample.cpu_usage_basis_points = 10000*(delta-ticks[-1]+previous[-1])//delta
return sample,ticks
class Jobs:
"""Private pipe to the non-root worker. The mTLS agent owns the transport."""
def __init__(self, process):
self.process = process
def read(self):
line = self.process.stdout.readline(2_100_000)
if not line.endswith(b'\n'):
raise ValueError('job worker disconnected')
return line[:-1]
def call(self, line):
self.process.stdin.write(line+b'\n')
self.process.stdin.flush()
return self.read()
def received(self, envelope):
if self.call(envelope.SerializeToString().hex().encode()) != b'ok':
raise ValueError('job response not committed')
def pump(self, ctx, agent, boot, generation):
for _ in range(12):
frame = self.call(b'next')
if not frame:
return
e = pb.Envelope.FromString(bytes.fromhex(frame.decode('ascii')))
kind = e.WhichOneof('payload')
if (e.protocol_major!=1 or e.agent_id!=agent or e.enrollment_generation!=1 or
e.connection_generation!=generation or kind not in (
'job_accepted','job_state_changed','job_result','sync_request','execution_permit_request')):
raise ValueError('invalid outbound job event')
batch = pb.MessageBatch(protocol_major=1,messages=[e])
reply = pb.MessageBatch.FromString(post(ctx,'messages',batch.SerializeToString(),MEDIA))
expected = {'sync_request':'sync_response','execution_permit_request':'execution_permit'}.get(kind,'ack')
check_reply(reply,e,expected)
for r in reply.messages:
self.received(r)
def check_reply(batch,e,expected):
if batch.protocol_major!=1 or not 1<=len(batch.messages)<=16:
raise ValueError('unexpected reply')
first = batch.messages[0]
generation = first.server_hello.connection_generation if expected=='server_hello' else e.connection_generation
for index,r in enumerate(batch.messages):
if (r.protocol_major,r.protocol_minor,r.agent_id,r.enrollment_generation,r.connection_generation,
r.boot_id,r.sequence)!=(1,0,e.agent_id,e.enrollment_generation,generation,e.boot_id,e.sequence):
raise ValueError('reply identity')
kind = r.WhichOneof('payload')
if (not index and kind!=expected) or (index and kind not in ('job_offer','cancel_request')):
raise ValueError('unexpected job reply')
if expected=='ack' and first.ack.message_id!=e.message_id:
raise ValueError('unconfirmed message')
return first
def session(db, identity, boot, instance):
with contextlib.ExitStack() as stack:
return run_channel(db, identity, boot, instance, stack)
def run_channel(db, identity, boot, instance, stack):
ctx = context()
ctx.load_cert_chain(STATE/'device.pem', STATE/'device.key')
generation = 0
jobs = None
enabled = BINDING.is_file() and WORKER.is_file()
def send(kind, payload, captured=None):
# A lost inventory ACK replays the exact durable envelope, including its
# original time, boot and sequence; only the connection fence changes.
with db:
pending = db.execute('SELECT envelope FROM pending WHERE id=1').fetchone() if kind=='inventory_snapshot' else None
if pending:
e = pb.Envelope.FromString(pending[0])
if e.agent_id != identity['agent_id']:
raise ValueError('pending snapshot identity')
e.connection_generation = generation
else:
seq = db.execute('UPDATE sequence SET n=n+1 RETURNING n').fetchone()[0]
e = pb.Envelope(protocol_major=1, message_id=str(uuid.uuid4()), agent_id=identity['agent_id'],
enrollment_generation=1, connection_generation=generation, boot_id=boot,
sequence=seq, observed_at=captured or time.time_ns()//1_000_000)
getattr(e, kind).CopyFrom(payload)
if kind == 'inventory_snapshot':
db.execute('INSERT INTO pending VALUES(1,?,?)',(e.SerializeToString(),e.observed_at))
batch = pb.MessageBatch(protocol_major=1, messages=[e])
reply = pb.MessageBatch.FromString(post(ctx, 'messages', batch.SerializeToString(), MEDIA))
r = check_reply(reply,e,'server_hello' if kind=='client_hello' else 'ack')
if len(reply.messages)>1 and jobs is None:
raise ValueError('work outside established job session')
for extra in reply.messages[1:]:
jobs.received(extra)
if kind == 'client_hello':
if (r.WhichOneof('payload') != 'server_hello' or r.connection_generation == 0 or
r.connection_generation != r.server_hello.connection_generation or
r.server_hello.max_message_bytes != 1024*1024):
raise ValueError('invalid server hello')
return r.connection_generation
if (r.connection_generation != generation or r.WhichOneof('payload') != 'ack'
or r.ack.message_id != e.message_id):
raise ValueError('unconfirmed message')
if kind == 'inventory_snapshot':
with db:
db.execute('DELETE FROM pending WHERE id=1')
if renew(ctx, identity):
ctx = context()
ctx.load_cert_chain(STATE/'device.pem', STATE/'device.key')
caps = ['heartbeat/1','opnsense.base/1']
root_caps=[]
if enabled:
caps.extend(['rmm.sync.v1','run_script','opnsense.uname/1','opnsense.posix/1'])
if Path('/usr/local/etc/discovery-agent/root-policy.json').is_file():
probe=subprocess.run([WORKER,'--root-capabilities'],capture_output=True,timeout=15)
if probe.returncode==0:
root_caps=json.loads(probe.stdout)
caps.extend(root_caps)
generation = send('client_hello',pb.ClientHello(agent_version=VERSION, capabilities=caps))
if enabled:
env=os.environ.copy();env.pop('DISCOVERY_ROOT_SESSION',None)
if root_caps:env['DISCOVERY_ROOT_SESSION']='1'
p = subprocess.Popen([WORKER,identity['agent_id'],identity['asset_id'],boot,str(generation)],env=env,
stdin=subprocess.PIPE,stdout=subprocess.PIPE,close_fds=True)
def close():
p.stdin.close()
try:
p.wait(timeout=15)
except subprocess.TimeoutExpired:
p.kill(); p.wait()
finally:
p.stdout.close()
stack.callback(close)
jobs = Jobs(p)
if jobs.read()!=b'ready':
raise ValueError('job worker unavailable')
last = None
previous = None
check_cert = time.monotonic()+12*3600
while True:
if time.monotonic() >= check_cert:
if renew(ctx, identity):
return
check_cert = time.monotonic()+12*3600
sample,previous = usage(previous)
send('heartbeat',pb.Heartbeat(instance_id=instance,uptime_seconds=int(time.clock_gettime(time.CLOCK_UPTIME)),
system_usage=sample))
with db:
# The ingestion contract accepts observations for 72 hours.
db.execute('DELETE FROM pending WHERE observed < ?', (time.time_ns()//1_000_000 - 72*3600*1000,))
if db.execute('SELECT 1 FROM pending').fetchone():
send('inventory_snapshot',None)
file = Path('/var/db/discovery-observations/inventory.json')
if file.exists():
with file.open('rb') as f:
stamp = os.fstat(f.fileno()).st_mtime_ns
if stamp != last and 0 <= time.time()-stamp/1e9 <= 120:
raw = f.read(384*1024+1)
if len(raw) > 384*1024:
raise ValueError('snapshot size')
send('inventory_snapshot',pb.InventorySnapshot(snapshot_id=str(uuid.uuid4()),
scope='opnsense.base',schema_version=1,chunk_count=1,data=raw,
sha256=hashlib.sha256(raw).digest(),completeness='partial'),stamp//1_000_000)
last = stamp
if jobs and (root_caps or not Path('/usr/local/etc/discovery-agent/root-policy.json').is_file()):
jobs.pump(ctx,identity['agent_id'],boot,generation)
elif jobs:
# Preserve the established helper journal chain across a helper
# restart. Keep telemetry flowing, then renegotiate capabilities.
if subprocess.run([WORKER,'--root-capabilities'],capture_output=True,timeout=15).returncode==0:
return
time.sleep(20)
def main():
os.umask(0o077)
if sys.argv[1:] == ['enroll']:
return enroll()
# Never allow a second network actor to fence or race an in-flight worker.
lock = (STATE/'channel.lock').open('ab')
fcntl.flock(lock,fcntl.LOCK_EX | fcntl.LOCK_NB)
identity = json.loads((STATE/'identity.json').read_bytes())
db = sqlite3.connect(STATE/'channel.sqlite')
with db:
db.execute('CREATE TABLE IF NOT EXISTS sequence(id INTEGER PRIMARY KEY CHECK(id=1),n INTEGER NOT NULL)')
db.execute('CREATE TABLE IF NOT EXISTS pending(id INTEGER PRIMARY KEY CHECK(id=1),envelope BLOB NOT NULL,observed INTEGER NOT NULL)')
db.execute('INSERT OR IGNORE INTO sequence VALUES(1,0)')
started = subprocess.check_output(['/sbin/sysctl','-n','kern.boottime']).decode().split('}')[0]
boot = str(uuid.uuid5(uuid.UUID(identity['agent_id']),started))
instance = str(uuid.uuid4())
while True:
try:
session(db, identity, boot, instance)
except Exception as exc:
print('OPNsense channel:', type(exc).__name__, str(exc)[:160], flush=True)
time.sleep(30)
if __name__ == '__main__':
main()
+33
View File
@@ -0,0 +1,33 @@
"""Build two FreeBSD Python zipapps; use grpcio-tools 1.76.0 + protobuf 6.33.5."""
import importlib.metadata as metadata
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import zipapp
here = Path(__file__).resolve().parent
out = Path(sys.argv[1])
out.mkdir(exist_ok=True, parents=True)
if metadata.version('protobuf') != '6.33.5' or metadata.version('grpcio-tools') != '1.76.0':
sys.exit('Unexpected protocol builder dependencies')
with tempfile.TemporaryDirectory() as tmp:
work = Path(tmp)
proto = here.parents[1]/'proto'
subprocess.run([sys.executable,'-m','grpc_tools.protoc','-I'+str(proto),'--python_out='+tmp,
str(proto/'agent.proto')],check=True)
dist = metadata.distribution('protobuf')
for f in dist.files:
if str(f).startswith('google/protobuf/') and f.suffix == '.py':
dst = work/f
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(dist.locate_file(f), dst)
(work/'google/__init__.py').touch()
license_file = next(f for f in dist.files if f.name=='LICENSE' and '.dist-info/' in str(f))
shutil.copyfile(dist.locate_file(license_file),work/'LICENSE-protobuf')
shutil.copyfile(here/'agent.py',work/'__main__.py')
zipapp.create_archive(work,out/'agent.pyz',compressed=True)
with tempfile.TemporaryDirectory() as tmp:
shutil.copyfile(here/'collect.py',Path(tmp)/'__main__.py')
zipapp.create_archive(tmp,out/'collect.pyz',compressed=True)
+161
View File
@@ -0,0 +1,161 @@
"""Bounded local inventory projection. Runs independently of the network agent."""
import datetime
import json
import os
from pathlib import Path
import re
import resource
import socket
import subprocess
import tempfile
import time
import xml.etree.ElementTree as ET
CACHE = Path('/var/db/discovery-observations')
def run(*args):
# Appliance commands may produce much more than one snapshot. Cap the
# producer before reading its output, including on a misconfigured host.
def limits():
resource.setrlimit(resource.RLIMIT_FSIZE, (2**21, 2**21))
with tempfile.TemporaryFile() as out:
subprocess.run(args, stdout=out, stderr=subprocess.DEVNULL, stdin=subprocess.DEVNULL,
check=True, timeout=10, preexec_fn=limits)
out.seek(0)
return out.read(2**21).decode('utf8', errors='replace').strip()
def sysctl(name):
return run('/sbin/sysctl', '-n', name)
def rows(provider):
entries = []
partial = False
try:
for text in provider():
if len(entries) == 128:
partial = True
break
text = ' '.join(text.split())
if len(text.encode()) > 512:
partial = True
text = text.encode()[:509].decode(errors='ignore') + '…'
if text:
entries.append(text)
except (OSError, ValueError, KeyError, TypeError, ET.ParseError, subprocess.SubprocessError):
return dict(status='partial' if entries else 'unavailable', entries=entries)
return dict(status='partial' if partial else 'ok', entries=entries)
def configured(config_path='/conf/config.xml'):
# Project only these model paths and display fields. Never serialize XML.
with open(config_path, 'rb') as f:
data = f.read(8 * 1024 * 1024 + 1)
if len(data) > 8 * 1024 * 1024:
raise ValueError('config size')
root = ET.fromstring(data)
for title, path, field in [
('WireGuard-Instanz', 'wireguard/server/servers/server', 'name'),
('WireGuard-Peer', 'wireguard/client/clients/client', 'name'),
('IPsec', 'Swanctl/Connections/Connection', 'description'),
('OpenVPN', 'OpenVPN/Instances/Instance', 'description'),
]:
for item in root.findall('OPNsense/' + path):
enabled = {'0': 'deaktiviert', '1': 'aktiviert'}.get(item.findtext('enabled'), 'Status unbekannt')
yield f"{title} · {item.findtext(field) or item.get('uuid') or '–'} · {enabled}"
def wireguard():
for line in run('/usr/bin/wg', 'show', 'all', 'latest-handshakes').splitlines():
interface, public_key, epoch = line.split()
stamp = datetime.datetime.fromtimestamp(int(epoch), datetime.timezone.utc).isoformat() if int(epoch) else 'nie'
yield f'{interface} · {public_key} · {stamp}'
def openvpn():
data = json.loads(run('/usr/local/opnsense/scripts/openvpn/ovpn_status.py', '--options', 'client,server'))
for role in ('server', 'client'):
for instance, status in data.get(role, {}).items():
yield f"{role} {instance} · {status.get('status', 'unbekannt')}"
for peer in status.get('client_list', []):
yield f"{instance} · {peer.get('common_name', '–')} · {peer.get('real_address', '–')}"
def ipsec():
# Only IKE summary lines, not CHILD_SA details or authentication settings.
for line in run('/usr/local/sbin/swanctl', '--list-sas').splitlines():
if re.match(r'^\S.*: #\d+, ', line):
yield line
def interfaces():
item = None
for line in run('/sbin/ifconfig','-f','inet:cidr,inet6:cidr','-a').splitlines():
if line and not line[0].isspace():
if item:
yield item
item = dict(name=line.split(':',1)[0],mac=None,addresses=[])
elif item:
fields = line.split()
if len(fields)>1 and fields[0]=='ether':
item['mac'] = fields[1]
elif len(fields)>1 and fields[0] in ('inet','inet6'):
item['addresses'].append(fields[1])
if item:
yield item
def packages():
for line in run('/usr/sbin/pkg','query','%n\t%v').splitlines():
name,version = line.split('\t',1)
yield dict(name=name,version=version)
def inventory_rows(fn, limit):
entries = []
try:
for item in fn():
if len(entries) >= limit:
return dict(status='partial',entries=entries)
entries.append(item)
return dict(status='ok',entries=entries)
except (OSError,ValueError,subprocess.SubprocessError):
return dict(status='partial' if entries else 'unavailable',entries=entries)
def snapshot():
disk = os.statvfs('/')
boot = int(re.search(r'sec = (\d+)',sysctl('kern.boottime'))[1])
hardware = {'device_type':'physical' if sysctl('kern.vm_guest')=='none' else 'virtual',
'booted_at':datetime.datetime.fromtimestamp(boot,datetime.timezone.utc).isoformat()}
for key,name in [('smbios.system.maker','manufacturer'),('smbios.system.serial','bios_serial_number')]:
try:
value = run('/bin/kenv',key)
if value and value.lower() not in ('none','unknown','not specified','default string'):
hardware[name]=value[:128]
except (OSError,subprocess.SubprocessError):
pass
return dict(os=run('/usr/local/sbin/opnsense-version'), hostname=socket.gethostname(),
architecture=os.uname().machine, root_total_bytes=disk.f_blocks*disk.f_frsize,
root_available_bytes=max(0,disk.f_bavail)*disk.f_frsize, cpu_model=sysctl('hw.model'),
logical_processors=int(sysctl('hw.ncpu')), memory_total_bytes=int(sysctl('hw.physmem')),
firewall=rows(lambda: run('/sbin/pfctl', '-sr').splitlines()), vpn_configured=rows(configured),
wireguard=rows(wireguard), openvpn=rows(openvpn), ipsec=rows(ipsec),
hardware=hardware, interfaces=inventory_rows(interfaces,128), software=inventory_rows(packages,1024),
users=rows(lambda:sorted(set(run('/usr/bin/users').split()))))
if __name__ == '__main__':
while True:
try:
with tempfile.NamedTemporaryFile(dir=CACHE, delete=False) as f:
f.write(json.dumps(snapshot(), ensure_ascii=False).encode())
os.fchmod(f.fileno(), 0o640)
os.replace(f.name, CACHE/'inventory.json')
except Exception as exc:
print('OPNsense collection:', type(exc).__name__, flush=True)
if 'f' in locals():
Path(f.name).unlink(missing_ok=True)
time.sleep(60)
+38
View File
@@ -0,0 +1,38 @@
#!/bin/sh
# PROVIDE: discovery_agent
# REQUIRE: NETWORKING
# KEYWORD: shutdown
. /etc/rc.subr
name=discovery_agent
rcvar=discovery_agent_enable
load_rc_config "$name"
: "${discovery_agent_enable:=NO}"
start_cmd=discovery_start
stop_cmd=discovery_stop
discovery_start() {
if [ -f /usr/local/etc/discovery-agent/root-policy.json ]; then
install -d -o root -g wheel -m 0700 /var/db/discovery-root-jobs || return 1
install -d -o root -g discovery -m 0750 /var/run/discovery-root-jobs || return 1
pid=/var/run/discovery-root-jobs.pid
if [ -z "$(check_pidfile "$pid" /usr/sbin/daemon)" ]; then
/usr/sbin/daemon -f -r -R 5 -P "$pid" -S -T discovery-root-jobs \
/usr/local/libexec/discovery-agent/rmm-opnsense-jobs --root-helper || return 1
fi
fi
for part in collect agent; do
pid=/var/run/discovery-$part.pid
[ -z "$(check_pidfile "$pid" /usr/sbin/daemon)" ] || continue
user=root
[ "$part" != agent ] || user=discovery
/usr/sbin/daemon -f -r -R 10 -P "$pid" -u "$user" -S -T "discovery-$part" \
/usr/local/bin/python3 -I /usr/local/libexec/discovery-agent/"$part".pyz || return 1
done
}
discovery_stop() {
for part in agent collect root-jobs; do
pid=$(check_pidfile "/var/run/discovery-$part.pid" /usr/sbin/daemon)
[ -z "$pid" ] || kill -TERM "$pid"
done
}
run_rc_command "$1"
+38
View File
@@ -0,0 +1,38 @@
"""In the guest, bind a staged native helper to the existing signed job identity.
Print only the hashes used by the central operator. Never replace another policy.
"""
import hashlib
import json
import os
from pathlib import Path
import pwd
import stat
import sys
def main():
if os.getuid()!=0 or os.geteuid()!=0 or len(sys.argv)!=2:
raise SystemExit('Run as root: enable-root.py /absolute/staging/rmm-opnsense-jobs')
app=Path(sys.argv[1]).resolve(strict=True)
conf=Path('/usr/local/etc/discovery-agent')
binding=conf/'job-binding.json'
for path in (app,*app.parents,binding,*binding.parents):
info=path.lstat()
if info.st_uid!=0 or info.st_mode & 0o022 or stat.S_ISLNK(info.st_mode):
raise ValueError('untrusted provisioning path')
user=pwd.getpwnam('discovery')
if not user.pw_uid or not user.pw_gid: raise ValueError('invalid agent identity')
image=hashlib.file_digest(app.open('rb'),'sha256').hexdigest()
policy=dict(schema_version=1,profile='full_maintenance_v1',agent_uid=user.pw_uid,agent_gid=user.pw_gid,
binding_sha256=hashlib.sha256(binding.read_bytes()).hexdigest(),helper_binary_sha256=image)
raw=(json.dumps(policy,separators=(',',':'))+'\n').encode()
path=conf/'root-policy.json'
if path.exists():
if path.is_symlink() or path.read_bytes()!=raw: raise ValueError('existing policy differs')
else:
with path.open('xb') as f:
os.fchmod(f.fileno(),0o444); f.write(raw);f.flush();os.fsync(f.fileno())
print(json.dumps(dict(helper_binary_sha256=image,helper_policy_sha256=hashlib.sha256(raw).hexdigest())))
if __name__=='__main__':main()
+51
View File
@@ -0,0 +1,51 @@
#!/bin/sh
# Direct artifact deployment in the test guest. Enrollment is a separate step.
set -eu
[ "$(id -u)" = 0 ] && [ "$(uname -s)" = FreeBSD ] && [ -x /usr/local/sbin/opnsense-version ]
[ "$#" = 1 ] || { echo 'Usage: install-dev.sh /path/to/staged-artifacts' >&2; exit 2; }
src=$1
for f in agent.pyz collect.pyz discovery_agent ca.pem; do [ -f "$src/$f" ]; done
/usr/bin/openssl x509 -in "$src/ca.pem" -noout -checkend 0 >/dev/null
pw groupshow discovery >/dev/null 2>&1 || pw groupadd discovery
pw usershow discovery >/dev/null 2>&1 || pw useradd discovery -g discovery -d /var/db/discovery-agent -s /usr/sbin/nologin
[ "$(id -gn discovery)" = discovery ] && [ "$(id -u discovery)" != 0 ]
app=/usr/local/libexec/discovery-agent
conf=/usr/local/etc/discovery-agent
rc=/usr/local/etc/rc.d/discovery_agent
install -d -o root -g wheel -m 0755 "$app" "$conf"
install -d -o discovery -g discovery -m 0700 /var/db/discovery-agent
install -d -o root -g discovery -m 2750 /var/db/discovery-observations
if [ -e "$conf/ca.pem" ]; then
cmp -s "$conf/ca.pem" "$src/ca.pem" || { echo 'Existing trust anchor differs' >&2; exit 1; }
fi
for part in collect agent; do
install -o root -g wheel -m 0444 "$src/$part.pyz" "$app/$part.next"
done
if [ -x "$rc" ]; then
"$rc" onestop
for _ in 1 2 3 4 5; do
[ -f /var/run/discovery-agent.pid ] || [ -f /var/run/discovery-collect.pid ] || [ -f /var/run/discovery-root-jobs.pid ] || break
sleep 1
done
if [ -f /var/run/discovery-agent.pid ] || [ -f /var/run/discovery-collect.pid ] || [ -f /var/run/discovery-root-jobs.pid ]; then
echo 'Existing service still running' >&2; exit 1
fi
fi
if [ -f "$src/rmm-opnsense-jobs" ]; then
install -o root -g wheel -m 0555 "$src/rmm-opnsense-jobs" "$app/jobs.next"
if [ -f "$conf/root-policy.json" ] && ! /usr/local/bin/python3 -I -c \
'import hashlib,json,sys;sys.exit(hashlib.file_digest(open(sys.argv[1],"rb"),"sha256").hexdigest()!=json.load(open(sys.argv[2]))["helper_binary_sha256"])' \
"$app/jobs.next" "$conf/root-policy.json"; then
echo 'Native helper differs from the local policy pin' >&2; "$rc" start; exit 1
fi
mv -f "$app/jobs.next" "$app/rmm-opnsense-jobs"
fi
for part in collect agent; do mv -f "$app/$part.next" "$app/$part.pyz"; done
install -o root -g wheel -m 0444 "$src/ca.pem" "$conf/ca.pem"
install -o root -g wheel -m 0555 "$src/discovery_agent" "$rc"
sysrc -f /etc/rc.conf.local discovery_agent_enable=YES >/dev/null
if [ -s /var/db/discovery-agent/identity.json ]; then
"$rc" start
else
echo 'Ready for enrollment under the discovery service account.'
fi
+53
View File
@@ -0,0 +1,53 @@
"""Provision the first service-account probe on an existing development installation.
Run as root in FreeBSD with the operator-confirmed tenant/asset/agent UUIDs and
enrollment/assignment generations. Fetch only the public signing pin as discovery.
"""
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import sys
import uuid
def main():
if os.geteuid() or len(sys.argv)!=6:
raise SystemExit('pin-probe.py TENANT ASSET AGENT ENROLLMENT_GENERATION ASSIGNMENT_VERSION')
tenant,asset,agent = (str(uuid.UUID(s)) for s in sys.argv[1:4])
enroll,assignment = map(int,sys.argv[4:])
if min(enroll,assignment)<1:
raise ValueError('invalid generation')
# Device-owned inputs are opened only after dropping privileges.
source = """
import hashlib,http.client,json,ssl
from pathlib import Path
p=Path('/var/db/discovery-agent')
i=json.loads((p/'identity.json').read_bytes())
ctx=ssl.create_default_context(cafile='/usr/local/etc/discovery-agent/ca.pem')
ctx.load_cert_chain(p/'device.pem',p/'device.key')
c=http.client.HTTPSConnection('discovery.ease-technologies.de',context=ctx,timeout=10)
c.request('GET','/api/v1/job-signing-key')
r=c.getresponse()
if r.status!=200:raise ValueError('device trust endpoint rejected')
print(json.dumps([i['agent_id'],i['asset_id'],json.loads(r.read(4097))]))
"""
current,bound_asset,k=json.loads(subprocess.check_output(
['su','-m','discovery','-c','/usr/local/bin/python3 -I -'],input=source.encode(),timeout=15))
key = k['public_key_hex']
if (current,bound_asset)!=(agent,asset) or not re.fullmatch('[a-f0-9]{64}',key) or (
k['algorithm'],k['schema_version'],k['key_id'])!=('Ed25519',1,hashlib.sha256(bytes.fromhex(key)).hexdigest()):
raise ValueError('conflicting device or signing key')
payload=dict(agent_id=agent,asset_id=asset,tenant_id=tenant,enrollment_generation=enroll,
assignment_version=assignment,public_key_hex=key)
p=Path('/usr/local/etc/discovery-agent/job-binding.json')
try:
with p.open('x') as f:
os.fchmod(f.fileno(),0o444); json.dump(payload,f); f.flush(); os.fsync(f.fileno())
except FileExistsError:
if p.is_symlink() or json.loads(p.read_bytes())!=payload: raise ValueError('existing binding differs')
if __name__=='__main__':
main()
+80
View File
@@ -0,0 +1,80 @@
"""Run from the protocol-builder venv: python test_agent.py <built agent.pyz>."""
import importlib.util
import os
from pathlib import Path
import sqlite3
import sys
import tempfile
import unittest
from unittest.mock import patch
import uuid
sys.path.insert(0,sys.argv.pop(1))
spec = importlib.util.spec_from_file_location('opn_channel', Path(__file__).with_name('agent.py'))
a = importlib.util.module_from_spec(spec)
spec.loader.exec_module(a)
class Reconnect(unittest.TestCase):
def test_piggybacked_work_requires_same_authenticated_channel_and_exact_receipt(self):
e=a.pb.Envelope(protocol_major=1,enrollment_generation=1,connection_generation=3,
agent_id=str(uuid.uuid4()),boot_id=str(uuid.uuid4()),sequence=42,message_id=str(uuid.uuid4()))
ack=a.pb.Envelope();ack.CopyFrom(e);ack.ack.message_id=e.message_id
offer=a.pb.Envelope();offer.CopyFrom(e);offer.job_offer.job_id=str(uuid.uuid4())
batch=a.pb.MessageBatch(protocol_major=1,messages=[ack,offer])
self.assertEqual(a.check_reply(batch,e,'ack'),ack)
batch.messages[1].connection_generation=2
with self.assertRaisesRegex(ValueError,'identity'): a.check_reply(batch,e,'ack')
del batch.messages[1:]
batch.messages[0].ack.message_id=str(uuid.uuid4())
with self.assertRaisesRegex(ValueError,'unconfirmed'): a.check_reply(batch,e,'ack')
def test_lost_ack_keeps_original_identity_after_restart(self):
with tempfile.TemporaryDirectory() as temp:
db = sqlite3.connect(':memory:')
db.executescript('CREATE TABLE sequence(id INTEGER PRIMARY KEY,n INTEGER);INSERT INTO sequence VALUES(1,0);'
'CREATE TABLE pending(id INTEGER PRIMARY KEY,envelope BLOB,observed INTEGER);')
cache = Path(temp)/'inventory.json'
cache.write_text('{}')
identity = {'agent_id':str(uuid.uuid4())}
instance = str(uuid.uuid4())
captures = []
generation = 0
def gateway(_ctx,path,data,media):
nonlocal generation
req = a.pb.MessageBatch.FromString(data).messages[0]
reply = a.pb.Envelope()
reply.CopyFrom(req)
kind = req.WhichOneof('payload')
if kind == 'client_hello':
generation += 1
reply.connection_generation = generation
reply.server_hello.CopyFrom(a.pb.ServerHello(connection_generation=generation,max_message_bytes=1024*1024))
else:
if kind == 'inventory_snapshot':
captures.append(req)
if len(captures)==1: raise OSError('lost ACK')
reply.ack.CopyFrom(a.pb.Ack(message_id=req.message_id))
return a.pb.MessageBatch(protocol_major=1,messages=[reply]).SerializeToString()
with patch.object(a,'context'), patch.object(a,'renew',return_value=False), \
patch.object(a,'post',gateway), patch.object(a,'Path',return_value=cache), \
patch.object(a,'usage',return_value=(a.pb.SystemUsage(),[])), \
patch.object(a.time,'clock_gettime',return_value=100), \
patch.object(a.time,'CLOCK_UPTIME',1,create=True), patch.object(a.time,'sleep',side_effect=StopIteration):
with self.assertRaises(OSError):
a.session(db,identity,str(uuid.uuid4()),instance)
self.assertIsNotNone(db.execute('SELECT 1 FROM pending').fetchone())
os.unlink(cache)
with self.assertRaises(StopIteration):
a.session(db,identity,str(uuid.uuid4()),instance)
self.assertIsNone(db.execute('SELECT 1 FROM pending').fetchone())
self.assertEqual(len(captures),2)
self.assertNotEqual(captures[0].connection_generation,captures[1].connection_generation)
captures[0].connection_generation=captures[1].connection_generation
self.assertEqual(captures[0],captures[1])
if __name__=='__main__':
unittest.main()
+28
View File
@@ -0,0 +1,28 @@
import tempfile
import unittest
from pathlib import Path
import collect
class ApplianceProjection(unittest.TestCase):
def test_configured_disabled_is_not_a_connection_and_secrets_stay_local(self):
with tempfile.TemporaryDirectory() as root:
p = Path(root)/'config.xml'
p.write_text('<opnsense><OPNsense><wireguard><server><servers><server uuid="uuid">'
'<name>Test</name><enabled>0</enabled><privkey>do-not-export</privkey>'
'</server></servers></server></wireguard><OpenVPN><Instances><Instance>'
'<enabled>1</enabled><description>Lab</description><tlskey>do-not-export</tlskey>'
'</Instance></Instances></OpenVPN></OPNsense></opnsense>')
self.assertEqual(collect.rows(lambda:collect.configured(p)), {
'status':'ok','entries':['WireGuard-Instanz · Test · deaktiviert','OpenVPN · Lab · aktiviert']})
p.write_text('<broken')
self.assertEqual(collect.rows(lambda:collect.configured(p)),{'status':'unavailable','entries':[]})
def test_truncation_is_explicit_and_utf8_stays_within_the_wire_limit(self):
result = collect.rows(lambda:iter(['ü'*1000]*200))
self.assertEqual((result['status'],len(result['entries'])),('partial',128))
self.assertTrue(all(len(s.encode())<=512 for s in result['entries']))
if __name__=='__main__':
unittest.main()
+132
View File
@@ -0,0 +1,132 @@
"""Native development integration: private chroot, disposable signer and identities.
Runs the installed FreeBSD helper binary and a child with the discovery uid in an
empty root; the installed device's socket, config, keys and journals are not used.
"""
import datetime
import hashlib
import json
import os
from pathlib import Path
import pwd
import re
import shutil
import socket
import sqlite3
import struct
import subprocess
import sys
import tempfile
import time
import uuid
APP='/usr/local/libexec/discovery-agent/rmm-opnsense-jobs'
CONF='/usr/local/etc/discovery-agent'
SOCK='/var/run/discovery-root-jobs/control.sock'
STATE='/var/db/discovery-root-jobs'
pack=lambda v:json.dumps(v,separators=(',',':')).encode()
sha=lambda b:hashlib.sha256(b).hexdigest()
new=lambda:str(uuid.uuid4())
now=lambda:datetime.datetime.now(datetime.timezone.utc)
def check(jail, folder, result):
import agent_pb2 as pb
u=pwd.getpwnam('discovery')
def inside(path):return jail/path.lstrip('/')
def copy(path):
dst=inside(path);dst.parent.mkdir(exist_ok=True,parents=True);shutil.copyfile(path,dst);dst.chmod(0o555)
copy('/libexec/ld-elf.so.1')
for path in [APP,'/bin/sh','/usr/bin/id','/sbin/sysctl']:
copy(path)
for lib in re.findall(r'=> (/\S+)',subprocess.check_output(['ldd',path],text=True)):copy(lib)
for path in [CONF,STATE,str(Path(SOCK).parent),'/tmp','/dev']:inside(path).mkdir(exist_ok=True,parents=True)
inside(STATE).chmod(0o700);inside('/tmp').chmod(0o1777);inside('/dev/null').touch()
key=folder/'fixture.key'
subprocess.run(['openssl','genpkey','-algorithm','Ed25519','-out',key],check=True)
public=subprocess.check_output(['openssl','pkey','-in',key,'-pubout','-outform','DER'])[-32:]
def signed(data):
f=folder/'message';f.write_bytes(data)
return list(subprocess.check_output(['openssl','pkeyutl','-sign','-rawin','-inkey',key,'-in',f]))
b=dict(tenant_id=new(),asset_id=new(),agent_id=new(),assignment_version=1,enrollment_generation=1)
binding=pack(dict(**b,public_key_hex=public.hex()));inside(CONF+'/job-binding.json').write_bytes(binding)
image=sha(inside(APP).read_bytes())
policy=pack(dict(schema_version=1,profile='full_maintenance_v1',agent_uid=u.pw_uid,agent_gid=u.pw_gid,
helper_binary_sha256=image,binding_sha256=sha(binding)))
inside(CONF+'/root-policy.json').write_bytes(policy)
helper=subprocess.Popen([APP,'--root-helper'],preexec_fn=lambda:(os.chroot(jail),os.chdir('/')),
stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL)
def invoke(op,deny=False,uid=u.pw_uid):
# Socket comes from the isolated chroot. The peer uid seen by getpeereid
# must be a real kernel credential, not a field in the test frame.
a,z=socket.socketpair();pid=os.fork()
if pid==0:
a.close()
try:
os.chroot(jail);os.chdir('/');os.setgroups([]);os.setgid(u.pw_gid);os.setuid(uid)
with socket.socket(socket.AF_UNIX) as s:
s.settimeout(12);s.connect(SOCK)
nonce=new();raw=pack(dict(nonce=nonce,policy=sha(policy),image=image,op=op))
s.sendall(struct.pack('>I',len(raw))+raw)
with s.makefile('rb') as f:
hdr=f.read(4)
if not hdr:os._exit(0 if deny else 1)
n=struct.unpack('>I',hdr)[0];assert n<=8*1024*1024
r=json.loads(f.read(n));assert not deny and r['nonce']==nonce
z.sendall(pack(r['data']))
os._exit(0)
except (ConnectionResetError,BrokenPipeError):
os._exit(0 if deny else 1)
except BaseException:os._exit(1)
z.close()
with a,a.makefile('rb') as f:data=f.read()
assert os.waitpid(pid,0)[1]==0, 'unexpected response from helper'
return json.loads(data) if data else None
try:
end=time.monotonic()+8
while not inside(SOCK).is_socket():
assert helper.poll() is None and time.monotonic()<end
time.sleep(.05)
invoke('Probe',True,uid=65534)
assert invoke('Probe')=='Pins'
snap=invoke({'Snapshot':1})['Snapshot']; q=pb.SyncRequest(request_id=new(),journal_id=new(),checkpoint=1,
helper_journal_id=snap['journal_id'],helper_checkpoint=snap['checkpoint'],assignment_version=1,journal_complete=True)
raw=q.SerializeToString();t=now()
dec=pack(dict(**b,domain='rmm.reconciliation-decision',schema_version=1,decision_id=new(),request_id=q.request_id,
request_hash=sha(raw),connection_generation=1,boot_id=snap['boot_id'],journal_id=q.journal_id,checkpoint=1,
helper_journal_id=q.helper_journal_id,helper_checkpoint=q.helper_checkpoint,issued_at=t.isoformat(),
valid_until=(t+datetime.timedelta(seconds=59)).isoformat(),ready=True,helper_ready=True,reason='CONSISTENT',
attempts=[],acknowledged_message_ids=[],key_id=sha(public)))
invoke({'Apply':[list(raw),dict(decision_bytes=list(dec),signature=signed(dec))]})
t=now();job=dict(**b,schema_version=1,job_id=new(),attempt_id=new(),execution_identity='root',
not_before=(t-datetime.timedelta(minutes=2)).isoformat(),start_before=(t+datetime.timedelta(minutes=2)).isoformat(),
max_runtime_seconds=10,max_output_bytes=1024,action=dict(action='run_script',interpreter='posix_sh',
script_utf8='id -u\n',arguments=[],expected_stdout='0\n',privileged_script_profile='full_maintenance_v1'))
body=pack(job);aid=new();ph=sha(b'isolated plan')
permit=pack(dict(**b,domain='rmm.execution-permit',schema_version=1,grant_id=new(),authorization_id=aid,
job_id=job['job_id'],attempt_id=job['attempt_id'],plan_hash=ph,job_payload_hash=sha(body),action='run_script',
execution_identity='root',not_before=(t-datetime.timedelta(seconds=6)).isoformat(),
start_before=(t+datetime.timedelta(seconds=50)).isoformat(),max_runtime_seconds=10,key_id=sha(public)))
req=dict(job_bytes=list(body),permit_bytes=list(permit),signature=signed(permit),plan_hash=ph,authorization_id=aid)
invoke({'Execute':[1,dict(req,signature=[0]*64)]},True)
invoke({'Execute':[2,req]},True)
invoke({'Execute':[1,dict(req,job_bytes=list(body.replace(b'id -u',b'id -g')))]},True)
db=sqlite3.connect('file:'+str(inside(STATE+'/consumption.sqlite'))+'?mode=ro',uri=True)
assert db.execute('SELECT count(*) FROM consumed').fetchone()==(0,)
out=invoke({'Execute':[1,req]})['Outcome']
assert out['state']=='succeeded' and out['exit_code']==0 and bytes(out['stdout'])==b'0\n'
assert out['evidence']['execution_uid']==0 and out['evidence']['script_sha256']==sha(b'id -u\n')
assert invoke({'Execute':[1,req]})['Outcome']==out
assert db.execute('SELECT count(*) FROM consumed JOIN execution_evidence USING(grant_id) WHERE result IS NOT NULL AND start IS NOT NULL').fetchone()==(1,)
db.close()
result.update(native_chroot_root_uid=0,replay_consumptions=1,foreign_peer_denied=True,
bad_signature_denied=True,altered_script_denied=True,stale_channel_denied=True)
finally:
helper.terminate();helper.wait(timeout=8)
if __name__=='__main__':
if not sys.platform.startswith('freebsd') or os.getuid()!=0:sys.exit('requires FreeBSD test guest as root')
sys.path.insert(0,'/usr/local/libexec/discovery-agent/agent.pyz')
with tempfile.TemporaryDirectory(prefix='discovery-helper-fixture-') as name:
folder=Path(name);jail=folder/'root';jail.mkdir();result={};check(jail,folder,result);print(json.dumps(result))
+45
View File
@@ -0,0 +1,45 @@
# Proxmox-VE-Host
Ein Proxmox-VE-Host ist ein Debian-System. Er verwendet deshalb den regulären Linux-Agenten, seinen Helper und seinen Updater. Zusätzlich liest der Agent über die lokale Proxmox-API die gehosteten Gäste, die geplanten Backupjobs und die Ergebnisse der letzten vzdump-Läufe. Er meldet sie als Inventarbereich `linux.proxmox`. Die WebUI zeigt diese Daten in den Gerätedetails des Hosts. Der Agent ordnet die gemeldeten VMs automatisch dem Host zu. [Sammelmodul](../../crates/agent-linux/src/inventory/proxmox.rs) · [API-Zugriff](../../crates/agent/src/proxmox_api.rs)
## Aufnahme
Für die Aufnahme den gemeinsamen [HTTPS-Installer](../../docs/development.md#linux-gerät-per-https-aufnehmen) als `root` auf dem Proxmox-Host ausführen. Der Installer erkennt Proxmox VE an `pveversion` und `pveum` und richtet einen nur lesenden Zugang ein:
- Proxmox-Benutzer `rmm-discovery@pve` und das privilegiengetrennte API-Token `rmm-discovery@pve!inventory`. Beide erhalten auf `/` ausschließlich die Rolle `PVEAuditor`. Bei einem privilegiengetrennten Token gilt die Schnittmenge aus Benutzer- und Tokenrechten. Deshalb braucht auch der Benutzer die Rolle.
- Den Tokenwert schreibt er nach `/opt/rust-discovery-client/proxmox-token` (Eigentümer root, Modus 0600). Die Cluster-CA landet als `/opt/rust-discovery-client/pve-ca.pem`.
- Das systemd-Drop-in `rmm-discovery-agent.service.d/41-proxmox-api.conf` übergibt den Token mit `LoadCredential=rmm-proxmox-token` an den Dienst und setzt `RMM_PROXMOX_CA_FILE`.
Der Agent läuft weiter unprivilegiert als `rmm-discovery`. Er verbindet sich nur mit `https://127.0.0.1:8006` und prüft das Zertifikat ausschließlich gegen die kopierte Cluster-CA. Token und Tokenwert erscheinen weder im Inventar noch im Log. Bei einer erneuten Installation bleiben ein vorhandener Token und die Geräteidentität erhalten. Fehlt die lokale Kopie, erzeugt der Installer das Token neu, weil Proxmox den Wert nur bei der Erstellung ausgibt.
## Gemeldete Daten
- **Gäste des lokalen Nodes:** VMID, Name, VM oder Container, Status, Vorlage, vCPU, RAM, Disk, Pool, MAC-Adressen der `netN`-Schnittstellen und für QEMU die SMBIOS-UUID. Höchstens 512 Gäste. Gäste auf anderen Cluster-Nodes meldet der Agent des jeweiligen Nodes.
- **Backupjobs** aus `/cluster/backup`: Zeitplan, Storage, Modus, Auswahl (alle, Pool oder VMIDs, abzüglich Ausnahmen) und die danach gesicherten lokalen Gäste. Den letzten fälligen Zeitpunkt berechnet der Proxmox-Scheduler selbst über `/cluster/jobs/schedule-analyze`.
- **vzdump-Läufe** der letzten acht Tage (höchstens 64) mit Start, Ende und Gesamtstatus. Das Ergebnis je Gast stammt aus den Zeilen `Finished Backup of VM` bzw. `Backup of VM … failed` des Tasklogs.
- **Gäste ohne Backupjob** aus `/cluster/backup-info/not-backed-up`.
Der Server bewertet jeden aktiven Job: **Gelaufen**, wenn jeder gesicherte Gast seit dem letzten fälligen Zeitpunkt (zwei Minuten Toleranz) erfolgreich gesichert wurde. **Fehlgeschlagen** bei einem gemeldeten Fehler, **Verpasst** ohne Lauf, **Läuft** bei aktivem vzdump. Ein Job, der in den letzten acht Tagen noch nicht fällig war, gilt als **Noch nicht fällig**. „Alle geplanten Backups sind gelaufen“ steht nur, wenn kein aktiver Job fehlgeschlagen oder verpasst ist. [Bewertung](../../crates/domain/src/inventory_document/proxmox.rs)
## Zuordnung der VMs
Nach jeder Linux- oder Windows-Inventarmeldung vergleicht der Server die MAC-Adressen der aktuellen Proxmox-Gastlisten mit den Netzwerkkarten der aktuellen virtuellen Geräte desselben Kunden. Bei einem eindeutigen Treffer trägt er den Proxmox-Host als Hypervisor ein (`hypervisor_source = proxmox`) und auditiert `device_hypervisor_detected`. Eine MAC, die mehrere Geräte oder mehrere Hosts melden, führt zu keiner Zuordnung. Verschwindet der Gast aus der gültigen Gastliste seines Hosts, wird die automatische Zuordnung gelöst. Eine manuelle Auswahl unter **Gerät verwalten → Hypervisor zuordnen** hat Vorrang. Die Option **Automatisch (Proxmox-Gastliste)** gibt die Entscheidung an die Erkennung zurück. [Abgleich](../../crates/server/src/proxmox_links.sql)
## Diagnose
| Anzeige im Gerätedetail | Ursache |
| --- | --- |
| „kein Lesezugang eingerichtet“ (`PROXMOX_NOT_CONFIGURED`) | Credential fehlt. HTTPS-Installer erneut als root ausführen und den Agentdienst neu starten. |
| `PROXMOX_TOKEN_REJECTED` | Token gelöscht oder abgelaufen. `/opt/rust-discovery-client/proxmox-token` entfernen und den Installer erneut ausführen. |
| `PROXMOX_PERMISSION_DENIED` | Der Benutzer oder das Token hat keine Rolle `PVEAuditor` auf `/`. |
| `PROXMOX_CA_UNAVAILABLE`, `PROXMOX_UNREACHABLE` | CA-Kopie fehlt bzw. `pveproxy` ist auf Port 8006 nicht erreichbar. |
Zum Widerrufen `pveum user token remove rmm-discovery@pve inventory` ausführen. Die übrige Inventur läuft dann weiter, der Proxmox-Bereich meldet einen Fehler.
## Testhost auf dem Entwicklungsnotebook
VM `proxmox` auf `qemu:///system`: 4 vCPU (`host-passthrough`, verschachtelte Virtualisierung), 4 GiB RAM, 64 GiB qcow2 im Pool `default`, Netz `default` mit fester DHCP-Reservierung `192.168.122.50` (MAC `52:54:00:50:7e:01`). Basis ist das offizielle `proxmox-ve_9.2-1.iso` (SHA-256 `4e88fe416df9b527624a175f24c9aa07c714d3332afb1ee3dbf3879573ef2c6c`). Die Installation erfolgte unbeaufsichtigt mit dem `proxmox-auto-install-assistant` aus dem Proxmox-Repository. Die Paketquelle ist `pve-no-subscription`. FQDN `pve.discovery.lan`, WebUI `https://192.168.122.50:8006`.
Testobjekte: VM 100 `debian-guest` (Debian-13-Cloud-Image, 1 GiB, cloud-init) und Container 101 `alpine-ct` (Alpine 3.24, ohne Backupjob). Der Backupjob `backup-debian` sichert VM 100 alle zwei Stunden (`*/2:00`) nach `local`.
SSH-Zugang, Root-Passwort und Hostpin liegen im privaten Zugangsverzeichnis (`proxmox-vm.conf`, `proxmox-dev-key`, `proxmox-dev-root`, `proxmox-dev.known_hosts`). `tools/dev_stage.py proxmox …` verwendet diese Dateien. Start bei Bedarf: `virsh -c qemu:///system start proxmox`.
+2
View File
@@ -21,9 +21,11 @@ services:
RMM_BOOTSTRAP_ORIGIN: https://discovery.ease-technologies.de:8443
RMM_AGENT_UPDATE_ARTIFACT_DIR: /release-objects
RMM_AGENT_UPDATE_LINUX_KEYRING_FILE: /release-config/linux-keyring.json
RMM_AGENT_UPDATE_LINUX_ARM64_KEYRING_FILE: /release-config/linux-arm64-keyring.json
RMM_AGENT_UPDATE_WINDOWS_KEYRING_FILE: /release-config/windows-keyring.json
RMM_AGENT_UPDATE_WINDOWS_GNU_OFFICIAL_KEYRING_FILE: /release-config/windows-gnu-official-keyring.json
RMM_AGENT_BOOTSTRAP_LINUX_FILE: /release-config/linux-bootstrap.json
RMM_AGENT_BOOTSTRAP_LINUX_ARM64_FILE: /release-config/linux-arm64-bootstrap.json
RMM_AGENT_BOOTSTRAP_WINDOWS_FILE: /release-config/windows-bootstrap.json
RMM_AGENT_BOOTSTRAP_WINDOWS_GNU_FILE: /release-config/windows-gnu-official-bootstrap.json
ports: ["127.0.0.1:18445:18443"]
+2
View File
@@ -1,8 +1,10 @@
x-release-environment: &release-environment
RMM_AGENT_UPDATE_ARTIFACT_DIR: /release-objects
RMM_AGENT_UPDATE_LINUX_KEYRING_FILE: /release-config/linux-keyring.json
RMM_AGENT_UPDATE_LINUX_ARM64_KEYRING_FILE: /release-config/linux-arm64-keyring.json
RMM_AGENT_UPDATE_WINDOWS_KEYRING_FILE: /release-config/windows-keyring.json
RMM_AGENT_BOOTSTRAP_LINUX_FILE: /release-config/linux-bootstrap.json
RMM_AGENT_BOOTSTRAP_LINUX_ARM64_FILE: /release-config/linux-arm64-bootstrap.json
RMM_AGENT_BOOTSTRAP_WINDOWS_FILE: /release-config/windows-bootstrap.json
RMM_AGENT_UPDATE_WINDOWS_GNU_OFFICIAL_KEYRING_FILE: /release-config/windows-gnu-official-keyring.json
RMM_AGENT_BOOTSTRAP_WINDOWS_GNU_FILE: /release-config/windows-gnu-official-bootstrap.json
+25 -2
View File
@@ -1,15 +1,22 @@
# Entwickeln, bauen und ausliefern
## Änderungen in Gitea sichern
Nach jeder abgeschlossenen Änderung die geprüften Dateien committen und den Commit auf den zugehörigen Gitea-Branch pushen. Das gilt für Code, Konfiguration und Dokumentation. Bei einem neuen Arbeitsbranch `git push -u origin HEAD` verwenden. Danach prüfen, dass der Commit auf dem Zielbranch in Gitea liegt.
## Quellen bauen
Der Buildrunner verpackt die Quellen eines eingecheckten Git-Stands und baut mit den gepinnten Docker-Buildern. Die folgenden Befehle laufen vom Wurzelverzeichnis des Repositorys. `REVISION` durch den zu liefernden Commit ersetzen. Vor dem ersten Frontendbuild `npm --prefix web ci` ausführen. Für den Rust-Build werden Python 3, Git, Docker, tar, rsync und die zum Ziel passenden lokalen Builder-Images benötigt. [Buildrunner](../tools/build.py)
| Ziel | Befehl | Ergebnis |
| --- | --- | --- |
| Linux-Client x86-64 | `python3 tools/build.py linux-x64 --ref REVISION` | `rmm-agent`, `rmm-privileged-helper`, `rmm-agent-updater` |
| Linux-Client ARM64 | `python3 tools/build.py linux-arm64 --ssh --ref REVISION` | `rmm-agent`, `rmm-privileged-helper`, `rmm-agent-updater` für AArch64 |
| Offizieller Windows-Client x64 | `python3 tools/build.py windows-gnu-x64 --ref REVISION` | `rmm-agent-windows.exe`, `rmm-helper-windows.exe`, `rmm-updater-windows.exe` für GNU |
| Server auf dem dedizierten ARM64-Host | `python3 tools/build.py server-arm64 --ssh --ref REVISION` | Server und Linux-Zeitbroker im Remote-Buildbereich |
| WebUI | `npm --prefix web run build` | TypeScript-Prüfung und Bundle in `web/dist/` |
`--plan` zeigt den Rust-Build einschließlich Quellenrevision und Ziel an. Der Runner verwendet je Ziel einen gemeinsamen Cargo-Cache, lädt bei Bedarf die gesperrten Abhängigkeiten und kompiliert offline. Er meldet den konkreten Ausgabeordner. Lokal liegt er unter `devstate/build/<Ziel>/artifacts/<Commit>/<Manifesthash>/`, beim SSH-Build unter `~/rmm-dev-build/server-arm64/artifacts/` auf dem Server. Die Buildquittung `manifest.json` bindet Ziel, Git-Revision, Builder und Binärhashes.
`--plan` zeigt den Rust-Build einschließlich Quellenrevision und Ziel an. Der Runner verwendet je Ziel einen gemeinsamen Cargo-Cache, lädt bei Bedarf die gesperrten Abhängigkeiten und kompiliert offline. Er meldet den konkreten Ausgabeordner. Lokal liegt er unter `devstate/build/<Ziel>/artifacts/<Commit>/<Manifesthash>/`, beim SSH-Build unter `~/rmm-dev-build/<Ziel>/artifacts/` auf dem Server. Die Buildquittung `manifest.json` bindet Ziel, Git-Revision, Builder und Binärhashes.
## Entwicklungsserver und WebUI aktualisieren
@@ -45,7 +52,9 @@ python3 tools/dev_agent_release.py --official-windows \
--minimum-updater-version 2 --prepare-only
```
Für Linux gilt derselbe Ablauf mit dem Linux-Buildordner und ohne `--official-windows`. Die Mindestversion entsprechend der vom Paket benötigten Updaterfähigkeit setzen. Der offizielle Windows-Kanal nutzt einen eigenen Signierschlüssel. Private Signierschlüssel verbleiben auf dem Signierhost.
Für Linux gilt derselbe Ablauf mit dem Linux-Buildordner und ohne `--official-windows`. Die Mindestversion entsprechend der vom Paket benötigten Updaterfähigkeit setzen. Der offizielle Windows-Kanal nutzt einen eigenen Signierschlüssel. Linux x86-64 und ARM64 haben ebenfalls getrennte Signierschlüssel, Starter und Updater-Pins. Private Signierschlüssel verbleiben auf dem Signierhost.
Für ein reines Linux-Inventarupdate kann `tools/linux_inventory_release.py` den neuen Agenten mit dem bisherigen Helper und Updater zusammenstellen. `--new` und `--previous` bezeichnen die hashadressierten Buildordner derselben Zielarchitektur; `--published-helper-sha256` stammt aus dem bisherigen signierten Release. Die neue Paketquittung hält pro Datei den tatsächlichen Quellbuild fest. Der resultierende Ordner durchläuft anschließend dieselbe Signaturprüfung und Veröffentlichung.
Für die Erstaufnahme in neuen Kundenkatalogen kann der Betreiber einen veröffentlichten Release als Starter auswählen. Die Quelle wird anhand ihrer Kunden-, Plattform- und Release-ID geprüft. Bereits befüllte Kataloge werden je Kunde gepflegt. [Starter veröffentlichen](../tools/dev_customer_starters.py)
@@ -54,8 +63,22 @@ python3 tools/dev_customer_starters.py --tenant "$TENANT_ID" \
--platform x86_64-pc-windows-gnu --official-windows --release "$RELEASE_ID"
```
Für ARM64 beim Starter `--platform aarch64-unknown-linux-gnu` verwenden und `--official-windows` weglassen. Die beiden Release-Overlays für den Server und die öffentliche API binden den ARM64-Schlüsselring und den Bootstrap-Pin ein.
Ein neuer stabiler Updater benötigt eine eigene Rotation des Bootstrap-Hashes. `--rotate-worker-only --rotate-bootstrap-from SHA256` bindet die Umstellung im Zielkunden an den bisherigen Hash. Für den gezielten lokalen Austausch des stabilen Windows-Updaters im Entwicklungsbetrieb prüft der PowerShell-Operator beide Hashes, die Geräteidentität und die installierten Releasepaare. [Releasewerkzeuge](../tools/agent_release.py) · [Windows-Updater austauschen](../deploy/windows/Replace-DevelopmentReleaseUpdater.ps1)
## Linux-Gerät per HTTPS aufnehmen
Unter **Verwaltung → Geräteaufnahme** den Kunden wählen und einen Einmalcode erstellen. Den gemeinsamen Linux-Befehl auf dem Zielgerät ausführen. `linux.sh` erkennt `x86_64` und `aarch64` und wählt den passenden signierten Agenten, Helper und Updater.
```sh
curl --fail --show-error --silent 'https://discovery.ease-technologies.de:8443/install/linux.sh' -o linux.sh && sudo sh linux.sh 'https://discovery.ease-technologies.de:8443' --enable-admin-scripts
```
Den Einmalcode auf Nachfrage eingeben. Bei direkter Root-Anmeldung entfällt `sudo`. `--enable-admin-scripts` aktiviert den lokalen Skript-Helper. Zusätzliche zentrale Geräterechte und die WebUI-Genehmigung gelten pro Auftrag.
Auf einem bereits aufgenommenen Linux-Gerät kann der gleiche HTTPS-Installer den Root-Skript-Helper nachträglich einrichten. Dazu `--enable-admin-scripts` verwenden. Der Installer übernimmt die vorhandene Geräteidentität. Für die WebUI muss die zentrale Geräteregel zusätzlich das berechtigte Konto und die authentisiert gemeldete Helper-Bindung enthalten. [Richtlinie und Berechtigungen](public-access.md#administrator-skripte-anfordern)
## Ergebnis prüfen
Die folgenden Rust-Tests laufen lokal mit Testidentitäten. Die Transporttests starten einen TLS-Server auf Loopback.
@@ -0,0 +1,40 @@
{
"date": "2026-11-18",
"host": "otrs.ease-technologies.de",
"reported_startup_traceback": "FileNotFoundError: /var/lib/rmm-discovery/certificates/current.json",
"before_correction": {
"device_certificate_present": true,
"identity_present": true,
"installer_token_file_removed": true,
"agent_active": true,
"update_timer_active": true,
"update_service_last_result": "success",
"authenticated_contact_at": "2026-11-18T17:08:51.835840Z",
"connection_generation": 1
},
"deployment": {
"revision": "4b3d0136e93e9b6d1de2ddaf3aae2cdade0dbb85",
"server_sha256": "e1001b2d82e7cde249cfbdfe1b559a8f33369fa78a49eb894dbc781a8e0c563d",
"web_index_sha256": "7486a20e0e570c8059d75be7b847ee35a39680d7bfa93498a8a9d26e6e3c4f87",
"image": "sha256:09a5dfb6750dce24f838f393b592bd41ceff22592710df9c308d6909faac49b6",
"api_health": "ok",
"gateway_health": "ok"
},
"native_retest": {
"version": "0.1.0-arm64.1",
"architecture": "aarch64-unknown-linux-gnu",
"last_authenticated_contact": "2026-11-18T17:12:57.212990Z",
"connection_generation": 2,
"identity_preserved": true,
"agent_process_preserved": true,
"update_timer_active": true,
"arm64_release_check": "available",
"installer_without_traceback": true,
"installer_without_new_code": true,
"wait_for_missing_certificate_native_probe": true,
"previous_lifecycle_sha256": "70fdfc00dc2126df6b6d6fc58fc0d727fa573f9838e703695c63dda126228e3d",
"new_lifecycle_sha256": "552a4ab7fb69b6c2f4a89b01145e866e53d2ae3f61c0db9536fe0b5d355d260c",
"public_installer_sha256": "09e1aef51362e2de9e6ce40721acc17df3cace0a9da01560c1ada0c5e1b906a2"
},
"linux_installer_regressions_passed": 15
}
@@ -0,0 +1,82 @@
{
"date": "2026-11-18",
"native_host": "otrs.ease-technologies.de",
"source": {
"client_revision": "247646a2ba74abe4f2d42a2c712e3d3166fd2187",
"server_revision": "6d15f1b3f524c11923ec1a5daba1c1903ff63a27",
"build_receipt_sha256": "35ee5c49744970e140f3e3156980e7dca950eb88a3346e92b8e497fe16265df7",
"maximum_required_glibc": "2.34"
},
"deployment": {
"database_schema": 40,
"server_sha256": "5681a387a1cdfe459185106e9bdf61c47103472a9051db4396262721d9d0a7eb",
"public_web_index_sha256": "7486a20e0e570c8059d75be7b847ee35a39680d7bfa93498a8a9d26e6e3c4f87",
"api_and_gateway_health": "ok",
"manifest_sha256": "d0f548d0d0a0c0e29e706089d958928d8e48cea29e61e52637be01938dd5df45",
"arm64_customer_starter_published": true,
"existing_devices_after_deploy": [
{
"platform": "linux-x86-64",
"version": "0.1.0-mvp.7",
"authenticated_contact_at": "2026-11-18T17:05:50.996252Z"
},
{
"platform": "windows-x64",
"version": "0.1.17",
"authenticated_contact_at": "2026-11-18T17:05:33.93606Z"
}
]
},
"public_https_native_probe": {
"machine": "aarch64",
"os": "Ubuntu 22.04.5 LTS",
"glibc": "glibc 2.35",
"origin": "https://discovery.ease-technologies.de:8443",
"installer_sha256": "f22a93fb33021f49ed9d9e56b89d040751c9505313611c73fd1490b1d0618c61",
"tenant_id": "11111111-1111-4111-8111-111111111111",
"release_id": "8f9c1558-a9b6-4338-84a7-4cd376201879",
"sequence": 1,
"version": "0.1.0-arm64.1",
"platform": "aarch64-unknown-linux-gnu",
"artifacts": [
{
"artifact_id": "2b8efd81-ed48-5d13-92b0-b5e0fdca49ac",
"bytes": 14786512,
"role": "agent",
"sha256": "67fac33df92bf88223c57d2c641518ce35fd7706f8d5031975b2227ad258c869"
},
{
"artifact_id": "bb4e1fb2-02fa-561c-993c-5c4792493a9a",
"bytes": 13055264,
"role": "helper",
"sha256": "8f64e88b11be8298fc216ccf827787a690419c9e9ebd948a9bd476f7a47dd5b6"
}
],
"updater": {
"bytes": 8755912,
"sha256": "3a3c272c59fd59ee8b994288372531f9bfb92b7174e31f71bd107542ec044934"
},
"published_platforms": [
"aarch64-unknown-linux-gnu",
"x86_64-pc-windows-gnu",
"x86_64-unknown-linux-gnu"
],
"x86_64_version": "0.1.0-mvp.7",
"native_checks": {
"installer_shell_syntax": true,
"invalid_code_stopped_before_install": true,
"manifest_and_pair_signature": true,
"modified_agent_rejected": true,
"wrong_platform_artifact_http_status": 404,
"missing_code_http_status": 401,
"enrollment_performed": false,
"all_three_elf_images_started_with_help": true
}
},
"passed_tests": {
"linux_installer": 13,
"release_contract_native_arm64": 10,
"webui_unit": 78,
"webui_enrollment_chromium": 1
}
}

Some files were not shown because too many files have changed in this diff Show More