Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
915e126f24 | ||
|
|
abad2022e2 | ||
|
|
6ed57c4d0b | ||
|
|
7006ce7420 | ||
|
|
b3fe35b86b |
@@ -10,6 +10,8 @@ Jede Installation ist an Kunde, Gerät und Agentenidentität gebunden. Die Stand
|
||||
|
||||
Der Agent meldet den Gerätetyp mit dem passiven Hardwareinventar. `physical` bezeichnet einen physischen Rechner, `virtual` ein virtualisiertes System und `hypervisor` eine erkannte Hostrolle. Proxmox VE erscheint als `hypervisor` mit `Proxmox VE` in der Spalte OS. Eine erkannte Hostrolle hat Vorrang, auch bei verschachtelter Virtualisierung. Bei fehlender Angabe zeigt die WebUI „Nicht gemeldet“. Die Geräteabfragen in WebUI und MCP verwenden denselben gespeicherten Typ. [Inventarprojektion](crates/server/src/devices.rs)
|
||||
|
||||
Die RAM-Spalte zeigt den für Anwendungen verfügbaren und den gesamten physischen Arbeitsspeicher in GiB. Wie beim Speicherplatz zeigt ein Balken den belegten Anteil. Die CPU-Spalte ergänzt die zuletzt gemessene Systemauslastung als Prozentwert. Die Agenten gewinnen sie aus den CPU-Zeitdifferenzen zwischen zwei Heartbeats im Abstand von ungefähr 30 Sekunden. Linux liefert systemweite Werte. Windows liefert den Wert, wenn alle logischen CPUs zu einer einzigen Prozessorgruppe gehören. Vor dem ersten aussagekräftigen Intervall bleibt die Auslastung offen. Die WebUI und MCP lesen dieselbe letzte Messung. Für Agenten ohne Ressourcenmeldung stammen verfügbare RAM-Werte aus dem vorhandenen Inventar. [Inventarprojektion](crates/server/src/devices.rs)
|
||||
|
||||
Kunden lassen sich in der WebUI anlegen und umbenennen. Beim Kundenwechsel erhält das Gerät eine neue Kopplung. Eine ausgeschiedene Installation kann nach Widerruf und persönlicher Bestätigung endgültig gelöscht werden. Ein noch gültiger offener Genehmigungsantrag oder ein nicht abgeschlossener Job einschließlich `unknown` sperrt die Löschung. Daten weiterhin beteiligter Zielgeräte bleiben erhalten. [Regeln zur Löschung](migrations/0036_delete_device_pairing.sql)
|
||||
|
||||
## Vom Skript zum Ergebnis
|
||||
@@ -24,7 +26,7 @@ Kunden lassen sich in der WebUI anlegen und umbenennen. Beim Kundenwechsel erhä
|
||||
|
||||
Linux führt Shellskripte unter dem Dienstkonto oder über den eingerichteten Helper als root aus. Windows führt PowerShell unter LocalService oder über den Helper als SYSTEM aus. Neustart und Installation von Windows-Updates sind Skriptaufträge. Bei einem Neustart kann das Skript bereits vor dem Herunterfahren erfolgreich enden. Der Betriebsnachweis verbindet den neuen Bootbezug mit dem nächsten authentisierten Kontakt und einem Folgeauftrag. [Skriptbibliothek](crates/server/src/script_library.rs) · [Auftragsverarbeitung](crates/server/src/jobs.rs)
|
||||
|
||||
Ein Abbruchwunsch wird am Auftrag gespeichert. Bei unterbrochener Rückmeldung gleichen Server, Agent und gegebenenfalls Helper den gespeicherten Versuch ab. Bei `unknown` bleibt die Wirkung ungeklärt. Die WebUI kann einen solchen Fall dokumentiert abschließen. Dieser Abschluss erteilt keine Freigabe für einen weiteren Versuch.
|
||||
Ein Abbruchwunsch wird am Auftrag gespeichert. Bei unterbrochener Rückmeldung gleichen Server, Agent und gegebenenfalls Helper den gespeicherten Versuch ab. Bei `unknown` bleibt die Wirkung ungeklärt. Die WebUI kann einen solchen Fall dokumentiert abschließen. Agent und Helper übernehmen den Abschluss im signierten Abgleich auch dann, wenn der Agent den Abbruch seiner eigenen Ausführung protokolliert hat und im unabhängig gebundenen Verbrauchsjournal des Helpers ein Ergebnis fehlt. Beide bewahren die ursprünglichen Belege. Der Abschluss erteilt keine Freigabe für einen weiteren Versuch.
|
||||
|
||||
## Öffentliche Verwaltung und Berechtigungen
|
||||
|
||||
|
||||
@@ -782,10 +782,10 @@ impl<V: Ed25519Verifier> HelperSync<V> {
|
||||
let d = decisions
|
||||
.get(&id)
|
||||
.context("ROOT_ATTEMPT_OMITTED_FROM_DECISION")?;
|
||||
validate_summary(row, local)?;
|
||||
validate_decision(row, local, d)?;
|
||||
let unreported = validate_summary(row, local)?;
|
||||
validate_decision(row, local, d, unreported)?;
|
||||
if d.unlock {
|
||||
let bytes = unlock_outcome(row, d, current_boot)?;
|
||||
let bytes = unlock_outcome(row, d, current_boot, unreported.then_some(local))?;
|
||||
if d.outcome_state == "closed_unknown" {
|
||||
crate::ledger::retain_closure_original(
|
||||
&tx,
|
||||
@@ -1636,22 +1636,33 @@ fn definitive(row: &LocalAttempt) -> bool {
|
||||
"succeeded" | "failed" | "cancelled" | "expired" | "closed_unknown"
|
||||
) && !row.outcome_bytes.is_empty()
|
||||
}
|
||||
fn validate_summary(row: &LocalAttempt, a: &AttemptView) -> Result<()> {
|
||||
fn validate_summary(row: &LocalAttempt, a: &AttemptView) -> Result<bool> {
|
||||
let unreported = row.state == "unknown"
|
||||
&& row.outcome_sha256.is_empty()
|
||||
&& row.outcome_bytes.is_empty()
|
||||
&& row.update_handoff_bytes.is_empty()
|
||||
&& crate::reconcile::uncertain_agent_outcome(&a.state, &a.outcome_bytes, &a.outcome_sha256);
|
||||
ensure!(
|
||||
a.job_payload_hash == row.job_payload_hash
|
||||
&& a.grant_id == row.grant_id.to_string()
|
||||
&& a.consumption == "consumed"
|
||||
&& a.state == row.state
|
||||
&& a.revision >= row.revision
|
||||
&& a.original_boot_id == row.original_boot_id.to_string()
|
||||
&& a.outcome_sha256 == row.outcome_sha256
|
||||
&& a.update_handoff_bytes == row.update_handoff_bytes
|
||||
&& (a.outcome_bytes.is_empty() || a.outcome_bytes == row.outcome_bytes),
|
||||
&& (unreported
|
||||
|| (a.state == row.state
|
||||
&& a.outcome_sha256 == row.outcome_sha256
|
||||
&& (a.outcome_bytes.is_empty() || a.outcome_bytes == row.outcome_bytes))),
|
||||
"ROOT_SNAPSHOT_REQUEST_MISMATCH"
|
||||
);
|
||||
Ok(())
|
||||
Ok(unreported)
|
||||
}
|
||||
fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -> Result<()> {
|
||||
fn validate_decision(
|
||||
row: &LocalAttempt,
|
||||
a: &AttemptView,
|
||||
d: &AttemptDecision,
|
||||
unreported: bool,
|
||||
) -> Result<()> {
|
||||
if !row.update_handoff_bytes.is_empty() {
|
||||
let proof =
|
||||
rmm_domain::agent_update::UpdateInflightProof::from_bytes(&row.update_handoff_bytes)
|
||||
@@ -1686,7 +1697,7 @@ fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -
|
||||
&& d.original_boot_id == Some(row.original_boot_id)
|
||||
&& (d.revision >= a.revision || (!d.unlock && d.outcome_state == "unknown"))
|
||||
&& d.previous_outcome_sha256.as_deref()
|
||||
== (!row.outcome_sha256.is_empty()).then_some(row.outcome_sha256.as_str()),
|
||||
== (!a.outcome_sha256.is_empty()).then_some(a.outcome_sha256.as_str()),
|
||||
"ROOT_SIGNED_ATTEMPT_BINDING"
|
||||
);
|
||||
ensure!(
|
||||
@@ -1697,11 +1708,26 @@ fn validate_decision(row: &LocalAttempt, a: &AttemptView, d: &AttemptDecision) -
|
||||
),
|
||||
"UNKNOWN_CANNOT_UNLOCK"
|
||||
);
|
||||
ensure!(
|
||||
!unreported
|
||||
|| (!d.unlock && d.outcome_state == "unknown")
|
||||
|| (d.unlock
|
||||
&& d.outcome_state == "closed_unknown"
|
||||
&& d.reason == "OPERATOR_CLOSED_UNKNOWN"),
|
||||
"ROOT_UNREPORTED_OUTCOME_CLOSURE_REQUIRED"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
fn unlock_outcome(row: &LocalAttempt, d: &AttemptDecision, current_boot: Uuid) -> Result<Vec<u8>> {
|
||||
fn unlock_outcome(
|
||||
row: &LocalAttempt,
|
||||
d: &AttemptDecision,
|
||||
current_boot: Uuid,
|
||||
unreported: Option<&AttemptView>,
|
||||
) -> Result<Vec<u8>> {
|
||||
let bytes = if d.outcome_bytes.is_empty() {
|
||||
row.outcome_bytes.clone()
|
||||
unreported
|
||||
.map_or(&row.outcome_bytes, |a| &a.outcome_bytes)
|
||||
.clone()
|
||||
} else {
|
||||
d.outcome_bytes.clone()
|
||||
};
|
||||
@@ -1712,9 +1738,7 @@ fn unlock_outcome(row: &LocalAttempt, d: &AttemptDecision, current_boot: Uuid) -
|
||||
&& o.state == d.outcome_state,
|
||||
"ROOT_CANONICAL_UNLOCK_OUTCOME"
|
||||
);
|
||||
if bytes != row.outcome_bytes
|
||||
&& crate::reconcile::operator_closure(&d.reason, &row.state, &o)
|
||||
{
|
||||
if bytes != row.outcome_bytes && crate::reconcile::operator_closure(&d.reason, &row.state, &o) {
|
||||
return Ok(bytes);
|
||||
}
|
||||
if bytes != row.outcome_bytes {
|
||||
|
||||
@@ -1019,9 +1019,9 @@ impl<V: Ed25519Verifier> ClientSync<V> {
|
||||
} else {
|
||||
for d in decision.attempts.iter().filter(|d| d.unlock) {
|
||||
let grant = d.grant_id.context("UNLOCK_GRANT_MISSING")?;
|
||||
let (agent,generation,job,attempt,previous):(String,u64,String,String,Option<Vec<u8>>)=tx.query_row(
|
||||
"SELECT agent,generation,job,attempt,COALESCE((SELECT outcome FROM execution_evidence WHERE execution_evidence.grant_id=consumed.grant_id),result) FROM consumed WHERE grant_id=?1",
|
||||
[grant.to_string()],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?,r.get(4)?)))?;
|
||||
let (agent,generation,job,attempt,previous,start):(String,u64,String,String,Option<Vec<u8>>,Option<Vec<u8>>)=tx.query_row(
|
||||
"SELECT c.agent,c.generation,c.job,c.attempt,COALESCE(e.outcome,c.result),e.start FROM consumed c LEFT JOIN execution_evidence e USING(grant_id) WHERE c.grant_id=?1",
|
||||
[grant.to_string()],|r|Ok((r.get(0)?,r.get(1)?,r.get(2)?,r.get(3)?,r.get(4)?,r.get(5)?)))?;
|
||||
ensure!(
|
||||
agent == self.binding.agent_id.to_string()
|
||||
&& generation == self.binding.enrollment_generation
|
||||
@@ -1034,13 +1034,61 @@ impl<V: Ed25519Verifier> ClientSync<V> {
|
||||
Some(&self.binding),
|
||||
grant,
|
||||
)?;
|
||||
let previous = resolved
|
||||
.or(previous)
|
||||
.context("LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISSING")?;
|
||||
let original = previous.clone();
|
||||
let prior = canonical_outcome_bytes(&serde_json::from_slice::<ReconciledOutcome>(
|
||||
&previous,
|
||||
)?)?;
|
||||
let previous = resolved.or(previous);
|
||||
let summary = gate
|
||||
.request
|
||||
.attempts
|
||||
.iter()
|
||||
.find(|a| a.job_id == job && a.attempt_id == attempt)
|
||||
.context("LOCAL_UNLOCK_ATTEMPT_MISSING")?;
|
||||
let closure = d.reason == "OPERATOR_CLOSED_UNKNOWN"
|
||||
&& d.outcome_state == "closed_unknown"
|
||||
&& summary.consumption == "consumed"
|
||||
&& uncertain_agent_outcome(
|
||||
&summary.state,
|
||||
&summary.outcome_bytes,
|
||||
&summary.outcome_sha256,
|
||||
);
|
||||
let prior = match &previous {
|
||||
Some(bytes) => {
|
||||
let canonical = canonical_outcome_bytes(&serde_json::from_slice(bytes)?)?;
|
||||
if closure
|
||||
&& summary.state == "closed_unknown"
|
||||
&& uncertain_agent_outcome("unknown", &canonical, &hash(&canonical))
|
||||
{
|
||||
summary.outcome_bytes.clone()
|
||||
} else {
|
||||
canonical
|
||||
}
|
||||
}
|
||||
None => {
|
||||
ensure!(closure, "LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISSING");
|
||||
let start: serde_json::Value = serde_json::from_slice(
|
||||
&start.context("LOCAL_UNLOCK_ORIGINAL_START_MISSING")?,
|
||||
)?;
|
||||
let request: crate::auth::Request =
|
||||
serde_json::from_value(start["request"].clone())?;
|
||||
let permit: rmm_domain::jobs::ExecutionPermitPayload =
|
||||
serde_json::from_slice(&request.permit_bytes)?;
|
||||
let (original, permit) = self.binding.verify_with(
|
||||
&request,
|
||||
permit.not_before,
|
||||
&self.verifier,
|
||||
)?;
|
||||
ensure!(
|
||||
permit.grant_id == grant
|
||||
&& original.job_id == d.job_id
|
||||
&& original.attempt_id == d.attempt_id
|
||||
&& hash(&request.job_bytes) == d.job_payload_hash
|
||||
&& Some(permit.authorization_id) == d.authorization_id
|
||||
&& Some(permit.plan_hash.as_str()) == d.plan_hash.as_deref()
|
||||
&& start["previous_boot_id"].as_str()
|
||||
== d.original_boot_id.map(|id| id.to_string()).as_deref(),
|
||||
"LOCAL_UNLOCK_ORIGINAL_START_BINDING"
|
||||
);
|
||||
summary.outcome_bytes.clone()
|
||||
}
|
||||
};
|
||||
ensure!(
|
||||
d.previous_outcome_sha256.as_deref() == Some(hash(&prior).as_str()),
|
||||
"LOCAL_UNLOCK_PREVIOUS_OUTCOME_MISMATCH"
|
||||
@@ -1055,7 +1103,8 @@ impl<V: Ed25519Verifier> ClientSync<V> {
|
||||
terminal(&outcome.state)
|
||||
&& outcome.state == d.outcome_state
|
||||
&& canonical_outcome_bytes(&outcome)? == bytes
|
||||
&& d.outcome_sha256.as_deref() == Some(hash(&bytes).as_str()),
|
||||
&& d.outcome_sha256.as_deref() == Some(hash(&bytes).as_str())
|
||||
&& (previous.is_some() || operator_closure(&d.reason, "unknown", &outcome)),
|
||||
"LOCAL_UNLOCK_CANONICAL_OUTCOME"
|
||||
);
|
||||
let preserve = crate::windows_system_nonstart::preserve_original_unlock(
|
||||
@@ -1065,7 +1114,12 @@ impl<V: Ed25519Verifier> ClientSync<V> {
|
||||
&bytes,
|
||||
)?;
|
||||
if outcome.state == "closed_unknown" {
|
||||
crate::ledger::retain_closure_original(&tx, grant, &original, &bytes)?;
|
||||
crate::ledger::retain_closure_original(
|
||||
&tx,
|
||||
grant,
|
||||
previous.as_deref().unwrap_or_default(),
|
||||
&bytes,
|
||||
)?;
|
||||
}
|
||||
tx.execute(
|
||||
"UPDATE consumed SET result=?2 WHERE grant_id=?1",
|
||||
@@ -1304,6 +1358,32 @@ pub(crate) fn operator_closure(
|
||||
&& outcome.evidence["closed_by_human"] == true
|
||||
&& outcome.evidence["retry"] == false
|
||||
}
|
||||
|
||||
/// An interrupted agent can record its own uncertainty while the independently
|
||||
/// consumed helper grant has no result. Neither witness establishes an effect.
|
||||
pub(crate) fn uncertain_agent_outcome(state: &str, bytes: &[u8], digest: &str) -> bool {
|
||||
if bytes.is_empty() || hash(bytes) != digest {
|
||||
return false;
|
||||
}
|
||||
let Ok(outcome) = serde_json::from_slice::<ReconciledOutcome>(bytes) else {
|
||||
return false;
|
||||
};
|
||||
canonical_outcome_bytes(&outcome).is_ok_and(|canonical| canonical == bytes)
|
||||
&& outcome.state == state
|
||||
&& !outcome.output_truncated
|
||||
&& !outcome.reboot_required
|
||||
&& (operator_closure("OPERATOR_CLOSED_UNKNOWN", "unknown", &outcome)
|
||||
|| (state == "unknown"
|
||||
&& matches!(
|
||||
outcome.error_code.as_str(),
|
||||
"AGENT_PROCESS_INTERRUPTED" | "HELPER_TRANSPORT_EFFECT_UNKNOWN"
|
||||
)
|
||||
&& outcome.exit_code.is_none()
|
||||
&& outcome.stdout.is_empty()
|
||||
&& outcome.stderr.is_empty()
|
||||
&& outcome.evidence["postconditions_passed"] == false))
|
||||
}
|
||||
|
||||
fn optional_uuid(s: &str) -> Result<Option<Uuid>> {
|
||||
if s.is_empty() {
|
||||
Ok(None)
|
||||
@@ -1452,6 +1532,24 @@ fn attempts(db: &Connection, ledger: &Connection, binding: &Binding) -> Result<V
|
||||
result_envelope = Some(envelope);
|
||||
}
|
||||
}
|
||||
// The signed closure projection can commit before either consumption
|
||||
// ledger. A fresh signed check may then finish the remaining writes.
|
||||
if state == "closed_unknown"
|
||||
&& consumption == "consumed"
|
||||
&& (canonical.is_empty()
|
||||
|| uncertain_agent_outcome("unknown", &canonical, &hash(&canonical)))
|
||||
{
|
||||
let closed: Vec<u8> = db.query_row(
|
||||
"SELECT outcome FROM reconcile_outcomes WHERE job=?1 AND attempt=?2",
|
||||
params![offer.job_id, offer.attempt_id],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
ensure!(
|
||||
uncertain_agent_outcome(&state, &closed, &hash(&closed)),
|
||||
"LOCAL_CLOSURE_OUTCOME_MISMATCH"
|
||||
);
|
||||
canonical = closed;
|
||||
}
|
||||
if !canonical.is_empty() {
|
||||
let o: ReconciledOutcome = serde_json::from_slice(&canonical)?;
|
||||
ensure!(o.state == state, "LOCAL_OUTCOME_STATE_MISMATCH");
|
||||
@@ -1746,6 +1844,15 @@ fn merge_helper(local: &mut [SyncAttempt], helper: &HelperSnapshot) -> Result<()
|
||||
"HELPER_ATTEMPT_BINDING"
|
||||
);
|
||||
if h.consumption != "unconsumed" {
|
||||
let unreported = h.state == "unknown"
|
||||
&& h.consumption == "consumed"
|
||||
&& h.outcome_bytes.is_empty()
|
||||
&& h.outcome_sha256.is_empty()
|
||||
&& h.update_handoff_bytes.is_empty()
|
||||
&& a.consumption == "consumed"
|
||||
&& a.grant_id == h.grant_id
|
||||
&& a.original_boot_id == h.original_boot_id
|
||||
&& uncertain_agent_outcome(&a.state, &a.outcome_bytes, &a.outcome_sha256);
|
||||
if !h.update_handoff_bytes.is_empty() {
|
||||
ensure!(
|
||||
h.state == "running" && a.state == "running" && a.outcome_bytes.is_empty(),
|
||||
@@ -1756,7 +1863,7 @@ fn merge_helper(local: &mut [SyncAttempt], helper: &HelperSnapshot) -> Result<()
|
||||
a.grant_id = h.grant_id.clone();
|
||||
a.consumption = h.consumption.clone();
|
||||
ensure!(
|
||||
a.outcome_sha256 == h.outcome_sha256,
|
||||
a.outcome_sha256 == h.outcome_sha256 || unreported,
|
||||
"HELPER_OUTCOME_MISMATCH"
|
||||
);
|
||||
if a.original_boot_id.is_empty() {
|
||||
|
||||
@@ -1485,6 +1485,207 @@ fn persist_original_fixture_result(
|
||||
fn completion_outcome() -> rmm_agent_core::auth::Outcome {
|
||||
serde_json::from_slice(&canonical_outcome_bytes(&success()).unwrap()).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn human_closure_reconciles_agent_interruption_and_a_helper_without_result() {
|
||||
use rmm_agent_core::helper_reconcile::{self, BootSource, ExecutorLedgerGuard, HelperSync};
|
||||
|
||||
struct TestGuard;
|
||||
impl ExecutorLedgerGuard for TestGuard {
|
||||
fn check(&self, db: &Connection) -> anyhow::Result<()> {
|
||||
anyhow::ensure!(
|
||||
db.is_autocommit(),
|
||||
"fixture ledger must be outside a transaction"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
struct TestBoot(Uuid);
|
||||
impl BootSource for TestBoot {
|
||||
fn current_boot(&self) -> anyhow::Result<Uuid> {
|
||||
Ok(self.0)
|
||||
}
|
||||
}
|
||||
fn map(snapshot: &helper_reconcile::Snapshot) -> HelperSnapshot {
|
||||
HelperSnapshot {
|
||||
journal_id: snapshot.journal_id,
|
||||
checkpoint: snapshot.checkpoint,
|
||||
previous_decision_id: snapshot.previous_decision_id,
|
||||
attempts: snapshot
|
||||
.attempts
|
||||
.iter()
|
||||
.map(|a| SyncAttempt {
|
||||
job_id: a.job_id.to_string(),
|
||||
attempt_id: a.attempt_id.to_string(),
|
||||
job_payload_hash: a.job_payload_hash.clone(),
|
||||
grant_id: a.grant_id.to_string(),
|
||||
consumption: a.consumption.clone(),
|
||||
state: a.state.clone(),
|
||||
revision: a.revision,
|
||||
original_boot_id: a.original_boot_id.to_string(),
|
||||
outcome_sha256: a.outcome_sha256.clone(),
|
||||
outcome_bytes: a.outcome_bytes.clone(),
|
||||
..Default::default()
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
let closed = canonical_outcome_bytes(&ReconciledOutcome {
|
||||
state: "closed_unknown".into(),
|
||||
stdout: vec![],
|
||||
stderr: vec![],
|
||||
exit_code: None,
|
||||
output_truncated: false,
|
||||
reboot_required: false,
|
||||
evidence: serde_json::json!({
|
||||
"postconditions_passed":false,"effect":"unknown","closed_by_human":true,"retry":false
|
||||
}),
|
||||
error_code: "OPERATOR_CLOSED_UNKNOWN".into(),
|
||||
})
|
||||
.unwrap();
|
||||
for interrupt_after_projection in [false, true] {
|
||||
let mut f = Fixture::new();
|
||||
reconcile::initialize(&f.db).unwrap();
|
||||
let (offer, grant, _) = f.add(
|
||||
"running",
|
||||
true,
|
||||
None,
|
||||
Action::SetServiceState {
|
||||
unit: "rmm-acceptance.service".into(),
|
||||
state: "running".into(),
|
||||
},
|
||||
);
|
||||
let grant = grant.unwrap();
|
||||
let archive = persist_original_fixture_result(
|
||||
&mut f,
|
||||
&offer,
|
||||
&grant,
|
||||
rmm_agent_core::auth::Outcome::unknown("AGENT_PROCESS_INTERRUPTED"),
|
||||
);
|
||||
rmm_agent_core::jobs::ack(&mut f.db, &archive.message_id).unwrap();
|
||||
let start: Vec<u8> = f
|
||||
.ledger
|
||||
.query_row("SELECT start FROM execution_evidence", [], |row| row.get(0))
|
||||
.unwrap();
|
||||
let start: serde_json::Value = serde_json::from_slice(&start).unwrap();
|
||||
let request: Request = serde_json::from_value(start["request"].clone()).unwrap();
|
||||
let boot = TestBoot(f.channel.boot_id);
|
||||
let guard = TestGuard;
|
||||
let mut helper_db = Connection::open_in_memory().unwrap();
|
||||
rmm_agent_core::ledger::initialize_ledger(&helper_db).unwrap();
|
||||
rmm_agent_core::ledger::begin_execution(
|
||||
&mut helper_db,
|
||||
&grant,
|
||||
&request,
|
||||
boot.0,
|
||||
f.clock.now,
|
||||
)
|
||||
.unwrap();
|
||||
let mut helper = HelperSync::new(f.binding.clone(), &f.clock).unwrap();
|
||||
let mut client = f.client();
|
||||
|
||||
for round in 0..if interrupt_after_projection { 3 } else { 2 } {
|
||||
let snapshot = map(&helper
|
||||
.snapshot(
|
||||
&mut helper_db,
|
||||
&guard,
|
||||
f.channel.connection_generation,
|
||||
&f.clock,
|
||||
&boot,
|
||||
)
|
||||
.unwrap());
|
||||
assert_eq!(snapshot.attempts[0].state, "unknown");
|
||||
assert!(snapshot.attempts[0].outcome_bytes.is_empty());
|
||||
let prepared = client
|
||||
.prepare(
|
||||
&mut f.db,
|
||||
&f.ledger,
|
||||
f.channel.clone(),
|
||||
Some(&snapshot),
|
||||
&f.clock,
|
||||
)
|
||||
.unwrap();
|
||||
let agent_attempt = &prepared.request.attempts[0];
|
||||
assert!(!agent_attempt.outcome_bytes.is_empty());
|
||||
let mut d = f.decision(&prepared, round != 0);
|
||||
d.issued_at = f.clock.now;
|
||||
if round != 0 {
|
||||
let attempt = &mut d.attempts[0];
|
||||
attempt.revision = 3;
|
||||
attempt.outcome_state = "closed_unknown".into();
|
||||
attempt.reason = "OPERATOR_CLOSED_UNKNOWN".into();
|
||||
attempt.unlock = true;
|
||||
attempt.outcome_sha256 = Some(hash(&closed));
|
||||
if agent_attempt.state == "unknown" {
|
||||
attempt.outcome_bytes = closed.clone();
|
||||
}
|
||||
}
|
||||
let signed = f.sign(&d);
|
||||
client.apply(&mut f.db, &signed, &f.clock).unwrap();
|
||||
if interrupt_after_projection && round == 1 {
|
||||
client = f.client();
|
||||
continue;
|
||||
}
|
||||
client.apply_consumption(&mut f.ledger, &f.clock).unwrap();
|
||||
let raw = prepared.request.encode_to_vec();
|
||||
let response = helper_reconcile::SignedResponse {
|
||||
decision_bytes: signed.decision_bytes,
|
||||
signature: signed.signature,
|
||||
};
|
||||
let applied = helper
|
||||
.apply(&mut helper_db, &guard, &response, &raw, &f.clock, &boot)
|
||||
.unwrap();
|
||||
if round == 0 {
|
||||
assert!(!applied.ready && applied.unlocked_grants.is_empty());
|
||||
assert!(!client.can_start(&f.channel, false, &f.clock));
|
||||
} else {
|
||||
assert!(applied.ready);
|
||||
assert_eq!(applied.unlocked_grants, vec![grant.grant_id]);
|
||||
assert!(
|
||||
helper
|
||||
.apply(&mut helper_db, &guard, &response, &raw, &f.clock, &boot)
|
||||
.unwrap()
|
||||
.unlocked_grants
|
||||
.is_empty()
|
||||
);
|
||||
client.apply_consumption(&mut f.ledger, &f.clock).unwrap();
|
||||
client.confirm_helper(&map(&applied.snapshot)).unwrap();
|
||||
assert!(client.can_start(&f.channel, true, &f.clock));
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
f.db.query_row::<String, _, _>("SELECT state FROM job_attempts", [], |rec| rec.get(0))
|
||||
.unwrap(),
|
||||
"closed_unknown"
|
||||
);
|
||||
assert_eq!(
|
||||
f.db.query_row::<Vec<u8>, _, _>("SELECT envelope FROM reconcile_results", [], |rec| {
|
||||
rec.get(0)
|
||||
})
|
||||
.unwrap(),
|
||||
archive.encode_to_vec()
|
||||
);
|
||||
for ledger in [&f.ledger, &helper_db] {
|
||||
assert_eq!(
|
||||
ledger
|
||||
.query_row::<Vec<u8>, _, _>("SELECT result FROM consumed", [], |rec| rec.get(0))
|
||||
.unwrap(),
|
||||
closed
|
||||
);
|
||||
assert_eq!(
|
||||
ledger
|
||||
.query_row::<(Vec<u8>, Vec<u8>), _, _>(
|
||||
"SELECT original_outcome,closure_outcome FROM unknown_closure_originals",
|
||||
[],
|
||||
|rec| Ok((rec.get(0)?, rec.get(1)?))
|
||||
)
|
||||
.unwrap(),
|
||||
(vec![], closed.clone())
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn crash_after_durable_outcome_before_result_recovers_original_then_signed_sync_closes() {
|
||||
let mut f = Fixture::new();
|
||||
|
||||
@@ -4,12 +4,12 @@ use crate::{
|
||||
actions::{ServiceUnit, reboot_required},
|
||||
process,
|
||||
};
|
||||
use rmm_domain::inventory_document::DeviceType;
|
||||
use rmm_domain::inventory_document::{CpuTimes, DeviceType};
|
||||
use std::{
|
||||
collections::BTreeMap,
|
||||
ffi::{CStr, CString},
|
||||
fs,
|
||||
io::Read,
|
||||
io::{BufRead, BufReader, Read},
|
||||
mem::MaybeUninit,
|
||||
net::{Ipv4Addr, Ipv6Addr},
|
||||
path::Path,
|
||||
@@ -432,6 +432,41 @@ pub fn operating_system() -> Observation<OperatingSystem> {
|
||||
Observation::unavailable("linux.os", Quality::Error, "OS_SOURCE_UNAVAILABLE")
|
||||
}
|
||||
}
|
||||
fn aggregate_jiffies(stat: &str) -> Option<Vec<u64>> {
|
||||
stat.lines()
|
||||
.find(|line| line.starts_with("cpu "))
|
||||
.and_then(|line| {
|
||||
line.split_whitespace()
|
||||
.skip(1)
|
||||
.map(str::parse)
|
||||
.collect::<Result<_, _>>()
|
||||
.ok()
|
||||
})
|
||||
}
|
||||
|
||||
fn linux_cpu_times(jiffies: &[u64]) -> Option<CpuTimes> {
|
||||
// Linux already includes guest and guest_nice in user and nice.
|
||||
let total = jiffies
|
||||
.iter()
|
||||
.take(8)
|
||||
.try_fold(0_u64, |total, ticks| total.checked_add(*ticks))?;
|
||||
let idle = jiffies
|
||||
.get(3)?
|
||||
.checked_add(jiffies.get(4).copied().unwrap_or_default())?;
|
||||
Some(CpuTimes { total, idle })
|
||||
}
|
||||
|
||||
pub fn cpu_times() -> Option<CpuTimes> {
|
||||
let mut line = String::new();
|
||||
BufReader::new(fs::File::open("/proc/stat").ok()?.take(4096))
|
||||
.read_line(&mut line)
|
||||
.ok()?;
|
||||
if !line.ends_with('\n') {
|
||||
return None;
|
||||
}
|
||||
linux_cpu_times(&aggregate_jiffies(&line)?)
|
||||
}
|
||||
|
||||
pub fn cpu() -> Observation<Cpu> {
|
||||
let info = read("/proc/cpuinfo", 1024 * 1024);
|
||||
let stat = read("/proc/stat", 256 * 1024);
|
||||
@@ -460,15 +495,7 @@ pub fn cpu() -> Observation<Cpu> {
|
||||
.filter(|(k, _)| k.trim() == "model name")
|
||||
.map(|(_, v)| v.trim().to_owned())
|
||||
});
|
||||
let aggregate_jiffies: Option<Vec<u64>> =
|
||||
stat.lines().find(|l| l.starts_with("cpu ")).and_then(|l| {
|
||||
l.split_whitespace()
|
||||
.skip(1)
|
||||
.map(str::parse)
|
||||
.collect::<Result<Vec<_>, _>>()
|
||||
.ok()
|
||||
});
|
||||
match aggregate_jiffies {
|
||||
match aggregate_jiffies(&stat) {
|
||||
Some(aggregate_jiffies) if logical_processors > 0 => Observation::valid(
|
||||
"linux.cpu:/proc/cpuinfo,/proc/stat",
|
||||
Cpu {
|
||||
@@ -1400,6 +1427,16 @@ mod tests {
|
||||
assert_eq!(request.since_filter, "@100.123457");
|
||||
assert_eq!(request.until_filter, "@101.987654");
|
||||
}
|
||||
#[test]
|
||||
fn cpu_time_excludes_double_counted_guest_ticks_and_io_wait() {
|
||||
let before = linux_cpu_times(&[10, 0, 5, 60, 10, 0, 0, 0, 4, 0]).unwrap();
|
||||
let after = linux_cpu_times(&[30, 0, 5, 120, 30, 0, 0, 0, 20, 0]).unwrap();
|
||||
assert_eq!(after.usage_since(before), Some(2_000));
|
||||
assert_eq!(after.usage_since(after), None);
|
||||
assert_eq!(before.usage_since(after), None);
|
||||
assert!(linux_cpu_times(&[10, 0, 0]).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn machine_types_distinguish_proxmox_guests_and_probe_failures() {
|
||||
assert_eq!(machine_type(true), Some(DeviceType::Hypervisor));
|
||||
|
||||
@@ -185,6 +185,7 @@ mod native {
|
||||
Payload::Heartbeat(Heartbeat {
|
||||
instance_id: Uuid::new_v4().to_string(),
|
||||
uptime_seconds: 1,
|
||||
system_usage: None,
|
||||
}),
|
||||
)?;
|
||||
let acknowledgement = send(client, &heartbeat).await?;
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
//! uninstall commands; registry entries are evidence, not verified MSI state.
|
||||
use anyhow::{Result, ensure};
|
||||
use chrono::Utc;
|
||||
use rmm_domain::inventory_document::DeviceType;
|
||||
use rmm_domain::inventory_document::{CpuTimes, DeviceType};
|
||||
use serde_json::{Value, json};
|
||||
use std::{
|
||||
mem::{size_of, zeroed},
|
||||
@@ -10,8 +10,12 @@ use std::{
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
use windows_sys::Win32::{
|
||||
Foundation::{ERROR_FILE_NOT_FOUND, ERROR_NO_MORE_ITEMS, ERROR_SUCCESS},
|
||||
System::{Registry::*, SystemInformation::*},
|
||||
Foundation::{ERROR_FILE_NOT_FOUND, ERROR_NO_MORE_ITEMS, ERROR_SUCCESS, FILETIME},
|
||||
System::{
|
||||
Registry::*,
|
||||
SystemInformation::*,
|
||||
Threading::{GetActiveProcessorGroupCount, GetSystemTimes},
|
||||
},
|
||||
};
|
||||
|
||||
struct Registry(HKEY);
|
||||
@@ -141,21 +145,53 @@ fn machine_type() -> Option<DeviceType> {
|
||||
)
|
||||
}
|
||||
|
||||
fn hardware() -> Result<Value> {
|
||||
let mut system: SYSTEM_INFO = unsafe { zeroed() };
|
||||
unsafe {
|
||||
GetNativeSystemInfo(&mut system);
|
||||
}
|
||||
fn memory_status() -> Result<MEMORYSTATUSEX> {
|
||||
let mut memory: MEMORYSTATUSEX = unsafe { zeroed() };
|
||||
memory.dwLength = size_of::<MEMORYSTATUSEX>() as u32;
|
||||
ensure!(
|
||||
unsafe { GlobalMemoryStatusEx(&mut memory) } != 0
|
||||
&& system.dwNumberOfProcessors > 0
|
||||
&& memory.ullTotalPhys > 0
|
||||
&& memory.ullAvailPhys <= memory.ullTotalPhys
|
||||
&& memory.dwMemoryLoad <= 100,
|
||||
"NATIVE_HARDWARE_QUERY"
|
||||
);
|
||||
Ok(memory)
|
||||
}
|
||||
|
||||
pub fn physical_memory() -> Option<(u64, u64)> {
|
||||
let memory = memory_status().ok()?;
|
||||
Some((memory.ullAvailPhys, memory.ullTotalPhys))
|
||||
}
|
||||
|
||||
pub fn cpu_times() -> Option<CpuTimes> {
|
||||
// GetSystemTimes on a multi-group machine only reports the caller's group.
|
||||
if unsafe { GetActiveProcessorGroupCount() } != 1 {
|
||||
return None;
|
||||
}
|
||||
let (mut idle, mut kernel, mut user) = (
|
||||
FILETIME::default(),
|
||||
FILETIME::default(),
|
||||
FILETIME::default(),
|
||||
);
|
||||
if unsafe { GetSystemTimes(&mut idle, &mut kernel, &mut user) } == 0 {
|
||||
return None;
|
||||
}
|
||||
let ticks =
|
||||
|time: FILETIME| (u64::from(time.dwHighDateTime) << 32) | u64::from(time.dwLowDateTime);
|
||||
// Windows includes idle time in the kernel counter.
|
||||
Some(CpuTimes {
|
||||
total: ticks(kernel).checked_add(ticks(user))?,
|
||||
idle: ticks(idle),
|
||||
})
|
||||
}
|
||||
|
||||
fn hardware() -> Result<Value> {
|
||||
let mut system: SYSTEM_INFO = unsafe { zeroed() };
|
||||
unsafe {
|
||||
GetNativeSystemInfo(&mut system);
|
||||
}
|
||||
ensure!(system.dwNumberOfProcessors > 0, "NATIVE_HARDWARE_QUERY");
|
||||
let memory = memory_status()?;
|
||||
let cpu_model = open(
|
||||
HKEY_LOCAL_MACHINE,
|
||||
r"HARDWARE\DESCRIPTION\System\CentralProcessor\0",
|
||||
|
||||
@@ -8,7 +8,7 @@ use rmm_agent_core::{
|
||||
reconcile::{self, SyncClock},
|
||||
};
|
||||
use rmm_agent_windows::certificate_rotation::{ProtectedKeyFiles, RotationStore};
|
||||
use rmm_agent_windows::{CAPABILITIES, job_binding::readonly_binding, journal::Journal, native};
|
||||
use rmm_agent_windows::{CAPABILITIES, inventory, job_binding::readonly_binding, journal::Journal, native};
|
||||
use rmm_domain::jobs::{Action, ExecutionPermitPayload};
|
||||
use rmm_protocol::*;
|
||||
use rusqlite::{Connection, OptionalExtension};
|
||||
@@ -2602,6 +2602,7 @@ impl Agent {
|
||||
let mut rotation_at = Instant::now();
|
||||
let mut rotation_after = Duration::from_secs(30);
|
||||
let mut heartbeat = Instant::now() - Duration::from_secs(60);
|
||||
let mut previous_cpu = None;
|
||||
let mut heartbeat_after = Duration::from_secs(30);
|
||||
let mut tick = tokio::time::interval(Duration::from_millis(20));
|
||||
tick.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
@@ -2776,9 +2777,19 @@ impl Agent {
|
||||
// inventory offline. Retry setup at the next heartbeat.
|
||||
eprintln!("PASSIVE_SIGNED_TIME_UNAVAILABLE");
|
||||
}
|
||||
let cpu = inventory::cpu_times();
|
||||
let cpu_usage_basis_points = cpu.zip(previous_cpu)
|
||||
.and_then(|(current, previous)| current.usage_since(previous));
|
||||
previous_cpu = cpu;
|
||||
let memory = inventory::physical_memory();
|
||||
let payload = Payload::Heartbeat(Heartbeat {
|
||||
instance_id: self.instance.clone(),
|
||||
uptime_seconds: self.started.elapsed().as_secs(),
|
||||
system_usage: Some(SystemUsage {
|
||||
cpu_usage_basis_points,
|
||||
ram_available_bytes: memory.map(|(available, _)| available),
|
||||
ram_total_bytes: memory.map(|(_, total)| total),
|
||||
}),
|
||||
});
|
||||
let h = self.envelope(generation, payload)?;
|
||||
channel.send(&h).await?;
|
||||
@@ -2788,6 +2799,7 @@ impl Agent {
|
||||
Payload::Heartbeat(Heartbeat {
|
||||
instance_id: self.instance.clone(),
|
||||
uptime_seconds: self.started.elapsed().as_secs(),
|
||||
system_usage: None,
|
||||
}),
|
||||
)?;
|
||||
channel.poll(poll).await?;
|
||||
|
||||
@@ -649,6 +649,7 @@ impl<C: reconcile::SyncClock> Agent<C> {
|
||||
let mut inventory_at = Instant::now();
|
||||
let mut update_at = Instant::now();
|
||||
let mut heartbeat_at = Instant::now();
|
||||
let mut previous_cpu = None;
|
||||
let mut heartbeat_after = Duration::from_millis(rand::random_range(27_000..33_000));
|
||||
let mut tick = tokio::time::interval(Duration::from_secs(5));
|
||||
let mut sync_at = Instant::now();
|
||||
@@ -809,11 +810,21 @@ impl<C: reconcile::SyncClock> Agent<C> {
|
||||
}
|
||||
if first_heartbeat || heartbeat_at.elapsed() >= heartbeat_after {
|
||||
first_heartbeat = false;
|
||||
let cpu = rmm_agent_linux::inventory::cpu_times();
|
||||
let cpu_usage_basis_points = cpu.zip(previous_cpu)
|
||||
.and_then(|(current, previous)| current.usage_since(previous));
|
||||
previous_cpu = cpu;
|
||||
let memory = rmm_agent_linux::inventory::memory().data;
|
||||
let heartbeat = self.envelope(
|
||||
generation,
|
||||
Payload::Heartbeat(Heartbeat {
|
||||
instance_id: self.instance.clone(),
|
||||
uptime_seconds: self.started.elapsed().as_secs(),
|
||||
system_usage: Some(SystemUsage {
|
||||
cpu_usage_basis_points,
|
||||
ram_available_bytes: memory.as_ref().and_then(|m| m.available_bytes),
|
||||
ram_total_bytes: memory.map(|m| m.total_bytes),
|
||||
}),
|
||||
}),
|
||||
)?;
|
||||
ws.send(&heartbeat).await?;
|
||||
@@ -823,6 +834,7 @@ impl<C: reconcile::SyncClock> Agent<C> {
|
||||
Payload::Heartbeat(Heartbeat {
|
||||
instance_id: self.instance.clone(),
|
||||
uptime_seconds: self.started.elapsed().as_secs(),
|
||||
system_usage: None,
|
||||
}),
|
||||
)?;
|
||||
ws.poll(poll).await?;
|
||||
|
||||
@@ -11,6 +11,22 @@ pub enum DeviceType {
|
||||
Hypervisor,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
pub struct CpuTimes {
|
||||
pub total: u64,
|
||||
pub idle: u64,
|
||||
}
|
||||
|
||||
impl CpuTimes {
|
||||
pub fn usage_since(self, previous: Self) -> Option<u32> {
|
||||
let total = self.total.checked_sub(previous.total)?;
|
||||
let idle = self.idle.checked_sub(previous.idle)?;
|
||||
let busy = total.checked_sub(idle)?;
|
||||
(total != 0)
|
||||
.then(|| ((u128::from(busy) * 10_000 + u128::from(total / 2)) / u128::from(total)) as u32)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
struct Observation {
|
||||
|
||||
@@ -81,6 +81,17 @@ pub struct Heartbeat {
|
||||
pub instance_id: String,
|
||||
#[prost(uint64, tag = "2")]
|
||||
pub uptime_seconds: u64,
|
||||
#[prost(message, optional, tag = "3")]
|
||||
pub system_usage: Option<SystemUsage>,
|
||||
}
|
||||
#[derive(Clone, PartialEq, Message)]
|
||||
pub struct SystemUsage {
|
||||
#[prost(uint32, optional, tag = "1")]
|
||||
pub cpu_usage_basis_points: Option<u32>,
|
||||
#[prost(uint64, optional, tag = "2")]
|
||||
pub ram_available_bytes: Option<u64>,
|
||||
#[prost(uint64, optional, tag = "3")]
|
||||
pub ram_total_bytes: Option<u64>,
|
||||
}
|
||||
#[derive(Clone, PartialEq, Message)]
|
||||
pub struct InventorySnapshot {
|
||||
@@ -156,6 +167,7 @@ mod tests {
|
||||
payload: Some(Payload::Heartbeat(Heartbeat {
|
||||
instance_id: "x".into(),
|
||||
uptime_seconds: 1,
|
||||
system_usage: None,
|
||||
})),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
@@ -4,6 +4,7 @@ use chrono::{DateTime, Utc};
|
||||
use prost::Message;
|
||||
use rmm_domain::MAX_MESSAGE;
|
||||
use rmm_protocol::*;
|
||||
use serde_json::{Value, json};
|
||||
use sha2::{Digest, Sha256};
|
||||
use sqlx::Row;
|
||||
use uuid::Uuid;
|
||||
@@ -146,7 +147,14 @@ pub async fn ingest(
|
||||
match &e.payload {
|
||||
Some(Payload::Heartbeat(h)) => {
|
||||
Uuid::parse_str(&h.instance_id)?;
|
||||
sqlx::query("UPDATE agent_installations SET last_contact_at=now() WHERE agent_id=$1 AND revoked_at IS NULL").bind(agent).execute(&mut *tx).await?;
|
||||
let usage = h.system_usage.as_ref().and_then(|sample| system_usage(sample, observed));
|
||||
sqlx::query("UPDATE agent_installations SET last_contact_at=now(),
|
||||
system_usage=CASE WHEN $2::jsonb IS NULL THEN system_usage
|
||||
ELSE COALESCE(system_usage,'{}'::jsonb) || $2 END,
|
||||
system_usage_observed_at=CASE WHEN $2::jsonb ? 'ram_available_bytes'
|
||||
THEN $3 ELSE system_usage_observed_at END
|
||||
WHERE agent_id=$1 AND revoked_at IS NULL")
|
||||
.bind(agent).bind(usage).bind(observed).execute(&mut *tx).await?;
|
||||
}
|
||||
Some(Payload::InventorySnapshot(s)) => {
|
||||
ensure!(
|
||||
@@ -197,3 +205,49 @@ pub async fn ingest(
|
||||
e.connection_generation,
|
||||
))
|
||||
}
|
||||
|
||||
fn system_usage(sample: &SystemUsage, observed: DateTime<Utc>) -> Option<Value> {
|
||||
let cpu = sample
|
||||
.cpu_usage_basis_points
|
||||
.filter(|&value| value <= 10_000);
|
||||
let memory = sample
|
||||
.ram_available_bytes
|
||||
.zip(sample.ram_total_bytes)
|
||||
.filter(|&(available, total)| total > 0 && available <= total);
|
||||
let mut usage = serde_json::Map::new();
|
||||
if let Some(cpu) = cpu {
|
||||
usage.insert("cpu_usage_percent".into(), json!(f64::from(cpu) / 100.0));
|
||||
usage.insert("cpu_usage_observed_at".into(), json!(observed));
|
||||
}
|
||||
if let Some((available, total)) = memory {
|
||||
usage.insert("ram_available_bytes".into(), json!(available));
|
||||
usage.insert("ram_total_bytes".into(), json!(total));
|
||||
}
|
||||
(!usage.is_empty()).then_some(Value::Object(usage))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn accepts_zero_usage_and_ignores_invalid_metrics_without_losing_the_heartbeat() {
|
||||
let mut sample = SystemUsage {
|
||||
cpu_usage_basis_points: Some(0),
|
||||
ram_available_bytes: Some(0),
|
||||
ram_total_bytes: Some(1),
|
||||
};
|
||||
let now = Utc::now();
|
||||
let measured = system_usage(&sample, now).unwrap();
|
||||
assert_eq!(measured["cpu_usage_percent"], json!(0.0));
|
||||
assert_eq!(measured["cpu_usage_observed_at"], json!(now));
|
||||
assert_eq!(measured["ram_available_bytes"], json!(0));
|
||||
assert_eq!(measured["ram_total_bytes"], json!(1));
|
||||
sample.cpu_usage_basis_points = Some(10_001);
|
||||
let partial = system_usage(&sample, now).unwrap();
|
||||
assert!(partial.get("cpu_usage_percent").is_none());
|
||||
assert!(partial.get("cpu_usage_observed_at").is_none());
|
||||
sample.ram_available_bytes = Some(2);
|
||||
assert_eq!(system_usage(&sample, now), None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ WITH classified AS (
|
||||
SELECT a.tenant_id,a.asset_id,a.agent_id,a.created_at,a.revoked_at,a.last_contact_at,
|
||||
a.last_authenticated_at,a.last_authenticated_certificate_sha256,a.certificate_sha256,
|
||||
a.enrollment_generation,a.assignment_version,a.capabilities,a.reported_agent_version,
|
||||
a.system_usage,a.system_usage_observed_at,
|
||||
asset.archived_at,t.name AS customer_name,
|
||||
CASE
|
||||
WHEN a.revoked_at IS NOT NULL THEN 'revoked'
|
||||
|
||||
@@ -352,6 +352,19 @@ fn project(row: &PgRow, now: DateTime<Utc>, full: bool) -> Value {
|
||||
let data = row.get::<Option<Value>, _>("data").unwrap_or(Value::Null);
|
||||
let cpu = observed(&data, "linux.cpu");
|
||||
let windows = observed(&data, "windows.hardware");
|
||||
let linux_memory = observed(&data, "linux.memory");
|
||||
let usage = row
|
||||
.get::<Option<Value>, _>("system_usage")
|
||||
.unwrap_or(Value::Null);
|
||||
let usage_at: Option<DateTime<Utc>> = row.get("system_usage_observed_at");
|
||||
let inventory_at: Option<DateTime<Utc>> = row.get("observed_at");
|
||||
let cpu_observed_at = usage["cpu_usage_observed_at"].as_str();
|
||||
let cpu_usage = cpu_observed_at.and_then(|_| usage["cpu_usage_percent"].as_f64());
|
||||
let ram_available = usage["ram_available_bytes"].as_u64();
|
||||
let ram_total = usage["ram_total_bytes"]
|
||||
.as_u64()
|
||||
.or_else(|| linux_memory["total_bytes"].as_u64())
|
||||
.or_else(|| windows["memory_total_bytes"].as_u64());
|
||||
let device_type =
|
||||
device_type(observed(&data, "linux.hardware")).or_else(|| device_type(windows));
|
||||
let last = row.get::<Option<DateTime<Utc>>, _>("last_contact_at");
|
||||
@@ -372,9 +385,14 @@ fn project(row: &PgRow, now: DateTime<Utc>, full: bool) -> Value {
|
||||
"disk_free_bytes":data["root_available_bytes"].as_u64(),"disk_total_bytes":data["root_total_bytes"].as_u64(),
|
||||
"cpu_model":cpu["model"].as_str().or_else(||windows["cpu_model"].as_str()),
|
||||
"cpu_logical_count":cpu["logical_processors"].as_u64().or_else(||windows["logical_processors_visible"].as_u64()),
|
||||
"ram_total_bytes":observed(&data,"linux.memory")["total_bytes"].as_u64().or_else(||windows["memory_total_bytes"].as_u64()),
|
||||
"cpu_usage_percent":cpu_usage,
|
||||
"cpu_usage_observed_at":cpu_observed_at,
|
||||
"ram_total_bytes":ram_total,
|
||||
"ram_available_bytes":ram_available.or_else(|| linux_memory["available_bytes"].as_u64().or_else(||windows["memory_available_bytes"].as_u64())
|
||||
.filter(|available| ram_total.is_some_and(|total| *available <= total))),
|
||||
"ram_observed_at":if ram_available.is_some() {usage_at} else {inventory_at},
|
||||
"last_contact_at":last,"agent_version":row.get::<Option<String>,_>("reported_agent_version"),
|
||||
"presence":presence,"observed_at":row.get::<Option<DateTime<Utc>>,_>("observed_at"),
|
||||
"presence":presence,"observed_at":inventory_at,
|
||||
"received_at":row.get::<Option<DateTime<Utc>>,_>("received_at"),
|
||||
"source_id":row.get::<Option<String>,_>("source_id"),"quality":if data.is_null(){"unknown"}else{"valid"},
|
||||
"completeness":"partial","health":"unknown"
|
||||
|
||||
@@ -27,10 +27,12 @@ Alle acht Versuche lieferten Exitcode 0. Bei beiden Neustarts wurde die Wiederan
|
||||
| Funktion | Beobachtung und Beleg |
|
||||
| --- | --- |
|
||||
| Öffentliche Verwaltung und Kundengrenzen | HTTPS-WebUI mit Passwort/TOTP, MCP mit widerrufbarem Token. Gerätebestand, Historie, eingeschränkter und kundenübergreifender Zugriff sowie Widerruf sind geprüft. [Öffentlicher Zugang](evidence/public-access-20261117.json) · [Kunden und Geräte](evidence/customer-fleet-20261117.json) |
|
||||
| RAM und CPU-Auslastung in der Übersicht | RAM als verfügbar / gesamt in GiB mit demselben Belegungsbalken wie beim Speicherplatz. Die CPU-Spalte zeigt die letzte gemessene Auslastung in Prozent. Am 18. November wurde `6ed57c4` mit Schema 41 produktiv aktiviert. Beide TLS-Healthchecks und der öffentliche WebUI-Index stimmten mit dem Build überein. Die öffentliche Übersicht zeigte die RAM-Balken auf Desktop und bei 390 px Breite für alle drei Geräte. Linux x64 mit `0.1.0-mvp.8` meldete in der finalen Ansicht um 17:53 UTC `3 / 3,7 GiB` und `1,5 %`. Die Folgemessung erschien ohne Neuladen. Windows meldete nach dem signierten Wechsel auf `0.1.18` zwischen 17:42 und 17:47 UTC native Werte. Beim abschließenden Browserdurchlauf war die Windows-Test-VM heruntergefahren. Der ARM64-Host konnte den signierten Kandidaten `0.1.0-arm64.2` abrufen. Sein Updater meldete wegen eines lokal als `unknown` geführten Versuchs `busy`. Dort steht die CPU-Messung aus und der RAM-Balken verwendet das vorhandene Inventar. [Geräteansicht](../web/src/devices.tsx) · [Geräteprojektion](../crates/server/src/devices.rs) |
|
||||
| Gerätetyp in der Übersicht | Spalte „Type“ nach „Hostname“ mit `physical`, `virtual` und `hypervisor`. Erkannte Proxmox-VE-Hosts zeigen `Proxmox VE` als OS. Am 18. November wurde `c599baa` auf dem öffentlichen Server aktiviert. Der Serverrunner bestätigte API, Gateway und öffentlichen API-Dienst mit Binärhash `4f9c8c60…`. Der öffentliche WebUI-Index stimmte mit dem gebauten Bundle überein. Die unveränderten Geräteidentitäten meldeten nach regulärem signierten Update `virtual` auf Linux mit `0.1.0-mvp.7` um 16:45 UTC und Windows mit `0.1.17` um 16:47 UTC. Geräteliste, Details, Mobilansicht mit 390 px und Historie sind in der öffentlichen WebUI geprüft. Beide Pakete sind in Development A und B und als Starter für leere Kataloge veröffentlicht. Die Typen `physical` und `hypervisor` sind mit Testdaten geprüft. Die Feldabnahme auf einem physischen Gerät und einem Proxmox-VE-Host steht aus. [Inventarprojektion](../crates/server/src/devices.rs) · [Geräteansicht](../web/src/devices.tsx) |
|
||||
| Von der Auswahl zur Ausgabe | Die WebUI bindet die ausgewählte Skriptversion, zeigt den Inhalt vor Genehmigung und verlinkt das Ergebnis je Gerät. Ein abgelehnter Antrag erzeugte null Jobs. [Auftragsablauf](evidence/webui-auftragsablauf-20261117.json) · [Ergebnislinks und Entscheidungen](evidence/webui-order-history-20261118.json) Auf OTRS ARM64 wurde der Root-Skript-Helper nachträglich mit dem HTTPS-Installer aktiviert. Die zentrale Geräteregel bindet das berechtigte WebUI-Konto an diese Installation und den signierten Helper. Die öffentliche WebUI-Vorschau akzeptierte Root für OTRS. Die Probe blieb bei einer Vorschau und erzeugte null Anträge, Jobs, Outbox-Einträge und Startgrants. [OTRS-Nachweis](evidence/linux-arm64-otrs-root-helper-20261118.json) |
|
||||
| Frische Installation und periodisches Agentupdate | Beide VMs wurden über HTTPS aufgenommen und übernahmen anschließend ein signiertes Folgepaket mit derselben Geräteidentität. Auf Windows ist der öffentliche GNU-Kanal einschließlich Prüfung des offiziellen Schlüssels belegt. [Linux](evidence/linux-public-installer-update-20261117.json) · [Windows GNU](evidence/windows-gnu-official-vm-20261118.json). ARM64 erweitert denselben HTTPS-Installer mit automatischer Architekturerkennung. Release `0.1.0-arm64.1` ist veröffentlicht und als Starter verfügbar. Auf Ubuntu 22.04.5 ARM64 wurden die öffentlichen Downloads, die Signaturprüfung, die Ablehnung einer veränderten Agentdatei und der native Start aller drei Dateien im Prüfmodus beobachtet. OTRS ist mit Gerätezertifikat aufgenommen und meldet authentisierte Heartbeats. Die erste Kontaktprüfung wartet seit der Korrektur auf das Zertifikat. Der öffentliche Installer lief auf OTRS erneut ohne Traceback und ohne neuen Einmalcode. Der ARM64-Katalogabruf ist bestätigt. Ein späterer Versionswechsel auf ARM64 steht aus. [ARM64-Release](evidence/linux-arm64-installer-20261118.json) · [Geräteaufnahme und Nachtest](evidence/linux-arm64-enrollment-heartbeat-20261118.json) |
|
||||
| Betriebssystemupdate per PowerShell | Die Windows-Updatevorlage installierte KB890830 v5.147 als SYSTEM. Ergebnis und Windows-Ereignis 19 bestätigen die Installation. [Installationsbeleg](evidence/windows-update-installation-20261117.json) |
|
||||
| Wiederanlauf | Nach einem Gatewaywechsel wurde der Windows-Agent ohne manuellen Dienstneustart wieder bereit. Der Helper glich seine bereits quittierte Ergebnismeldung ab. [Gatewaywechsel](evidence/windows-gateway-clock-recovery-20261117.json) · [Helper-Abgleich](evidence/windows-helper-result-recovery-20261118.json) |
|
||||
| WebUI-Abschluss eines ungeklärten Versuchs | Auf OTRS ARM64 war derselbe Versuch zentral als `closed_unknown` abgeschlossen und lokal wegen unterschiedlicher Befunde noch blockiert. Der Agent hatte eine archivierte Unterbrechungsmeldung, beide Verbrauchsjournale hatten den ursprünglichen Start und kein Ergebnis. Die Regression prüft den signierten Abschluss in Agent und Helper, den Erhalt der Originale und den Wiederanlauf nach einer Unterbrechung zwischen den Journalaktualisierungen. Die Bereitstellung und der native Nachweis stehen aus. [Geräteabgleich](../crates/agent-core/tests/reconcile.rs) |
|
||||
|
||||
[Produktvertrag](../Rust_RMM_Implementierungsspezifikation.md) · [Übergabe für die Weiterentwicklung](handover.md)
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
ALTER TABLE agent_installations
|
||||
ADD COLUMN system_usage jsonb,
|
||||
ADD COLUMN system_usage_observed_at timestamptz;
|
||||
|
||||
GRANT UPDATE(system_usage, system_usage_observed_at) ON agent_installations TO rmm_app;
|
||||
@@ -9,6 +9,8 @@ Linux- und Windows-Agenten bauen die Verbindung zum Gateway ausgehend auf. Eine
|
||||
| `POST /agent/v1/messages` | Ein `ClientHello` eröffnet die Verbindung. Weitere `MessageBatch`-Anfragen übertragen unter anderem Inventar, Abgleich, Auftragsstatus und Ergebnisse. | Ein `MessageBatch` enthält die Antworten des Servers. |
|
||||
| `POST /agent/v1/poll` | Ein `PollRequest` enthält einen Heartbeat und die gewünschte Wartezeit von 0 bis 25 Sekunden. | Das Gateway bestätigt den Heartbeat und liefert vorhandene Arbeit mit derselben Antwort. Im Leerlauf genügt die Bestätigung. |
|
||||
|
||||
Heartbeats können eine passive Ressourcenmessung übermitteln. `cpu_usage_basis_points` liegt zwischen 0 und 10.000, `ram_available_bytes` und `ram_total_bytes` bilden gemeinsam einen Messwert. Das Gateway speichert je authentisierter Installation den letzten gültigen CPU-Wert und das letzte gültige RAM-Wertepaar mit ihrem jeweiligen Zeitpunkt. Ein zusätzlicher Poll-Heartbeat ohne Ressourcendaten lässt die gespeicherten Messwerte bestehen.
|
||||
|
||||
Beide Endpunkte verwenden `application/x-protobuf` und Protokollversion 1. Eine Nachricht trägt unter anderem die Gerätekennung, die Generationen von Einschreibung und Verbindung, die Boot-ID, Sequenznummer und Nachrichten-ID. Für gespeicherte Meldungen wie ein Skriptergebnis bestätigt der Server die ursprüngliche Nachrichten-ID. [HTTP-Verarbeitung](../crates/server/src/agent_http.rs) · [Speicherung und Bestätigung](../crates/server/src/agent_service.rs)
|
||||
|
||||
## Vom Antrag bis zum Ergebnis
|
||||
|
||||
+10
-1
@@ -31,7 +31,16 @@ message Envelope {
|
||||
}
|
||||
message ClientHello { string agent_version = 1; repeated string capabilities = 2; }
|
||||
message ServerHello { uint64 connection_generation = 1; uint32 max_message_bytes = 2; }
|
||||
message Heartbeat { string instance_id = 1; uint64 uptime_seconds = 2; }
|
||||
message Heartbeat {
|
||||
string instance_id = 1;
|
||||
uint64 uptime_seconds = 2;
|
||||
SystemUsage system_usage = 3;
|
||||
}
|
||||
message SystemUsage {
|
||||
optional uint32 cpu_usage_basis_points = 1;
|
||||
optional uint64 ram_available_bytes = 2;
|
||||
optional uint64 ram_total_bytes = 3;
|
||||
}
|
||||
// Single-chunk root inventory scope only; JSON is schema rmm.linux.base/1.
|
||||
message InventorySnapshot {
|
||||
string snapshot_id = 1;
|
||||
|
||||
@@ -56,7 +56,9 @@ test('creates and renames a customer, opens its inventory, and hides editing for
|
||||
if (!customer) throw new Error('Kunde fehlt im Testbestand');
|
||||
return route.fulfill({ json: listing([{
|
||||
tenant_id: customerId, customer_name: customer.name, asset_id: 'device-1', agent_id: 'agent-1',
|
||||
lifecycle: 'current', status: 'online', hostname: 'NORD-PC', device_type: 'hypervisor', os: 'Proxmox VE'
|
||||
lifecycle: 'current', status: 'online', hostname: 'NORD-PC', device_type: 'hypervisor', os: 'Proxmox VE',
|
||||
ram_available_bytes: 6 * 2 ** 30, ram_total_bytes: 16 * 2 ** 30, cpu_usage_percent: 0,
|
||||
disk_free_bytes: 24 * 2 ** 30, disk_total_bytes: 32 * 2 ** 30
|
||||
}]) });
|
||||
}
|
||||
if (method === 'POST' || method === 'PATCH') {
|
||||
@@ -108,6 +110,13 @@ test('creates and renames a customer, opens its inventory, and hides editing for
|
||||
await expect(headers.nth(1)).toHaveText('Type');
|
||||
await expect(device.getByRole('cell').nth(2)).toHaveText('hypervisor');
|
||||
await expect(device.getByRole('cell').nth(3)).toHaveText('Proxmox VE');
|
||||
const memory = device.locator('[data-label="RAM frei / gesamt"]');
|
||||
await expect(memory).toHaveText('6 / 16 GiB');
|
||||
await expect(memory.locator('.capacity-track')).toHaveAttribute('title', '37,5 % frei');
|
||||
await expect(memory.locator('.capacity-track i')).toHaveAttribute('style', 'width: 62.5%;');
|
||||
await expect(device.locator('[data-label="Speicher frei / gesamt"] .capacity-track i'))
|
||||
.toHaveAttribute('style', 'width: 25%;');
|
||||
await expect(device.getByText('Auslastung 0 %', { exact: true })).toBeVisible();
|
||||
const navigation = page.getByRole('navigation', { name: 'Hauptnavigation' });
|
||||
await navigation.getByRole('link', { name: 'Aufträge', exact: true }).click();
|
||||
await expect(page.getByRole('combobox', { name: 'Kunde', exact: true })).toHaveValue(customerId);
|
||||
|
||||
+21
-9
@@ -43,6 +43,15 @@ function capacity(free: unknown, total: unknown) {
|
||||
t = gib(total);
|
||||
return f === null && t === null ? unknown : `${f ?? '–'} / ${t ?? '–'} GiB`;
|
||||
}
|
||||
function cpuUsage(value: unknown) {
|
||||
return typeof value === 'number' && Number.isFinite(value) && value >= 0 && value <= 100
|
||||
? `${number.format(value)} %`
|
||||
: unknown;
|
||||
}
|
||||
function measuredAt(value: unknown) {
|
||||
const observed = time(value);
|
||||
return observed === unknown ? undefined : `Gemessen am ${observed}`;
|
||||
}
|
||||
function cpu(d: RecordValue) {
|
||||
const model = label(d.cpu_model),
|
||||
count = d.cpu_logical_count;
|
||||
@@ -76,14 +85,13 @@ function Presence({ d }: { d: RecordValue }) {
|
||||
</span>
|
||||
);
|
||||
}
|
||||
function Storage({ d }: { d: RecordValue }) {
|
||||
const free = d.disk_free_bytes,
|
||||
total = d.disk_total_bytes;
|
||||
function Capacity({ free, total }: { free: unknown; total: unknown }) {
|
||||
return (
|
||||
<>
|
||||
<span>{capacity(free, total)}</span>
|
||||
{typeof free === 'number' &&
|
||||
typeof total === 'number' &&
|
||||
Number.isFinite(total) &&
|
||||
free >= 0 &&
|
||||
free <= total &&
|
||||
total > 0 && (
|
||||
@@ -171,8 +179,9 @@ function DeviceDetail({
|
||||
['Kunde', label(d.customer_name)],
|
||||
['Type', label(d.device_type)],
|
||||
['Betriebssystem', label(d.os)],
|
||||
['RAM', gib(d.ram_total_bytes) === null ? unknown : gib(d.ram_total_bytes) + ' GiB'],
|
||||
['RAM frei / gesamt', capacity(d.ram_available_bytes, d.ram_total_bytes)],
|
||||
['CPU', [cpu(d).primary, cpu(d).secondary].filter(Boolean).join(' · ')],
|
||||
['CPU-Auslastung', [cpuUsage(d.cpu_usage_percent), measuredAt(d.cpu_usage_observed_at)].filter(Boolean).join(' · ')],
|
||||
['Speicher frei / gesamt', capacity(d.disk_free_bytes, d.disk_total_bytes)],
|
||||
['Letzter Kontakt', time(d.last_contact_at)],
|
||||
['Inventar erhoben', time(d.observed_at)],
|
||||
@@ -534,7 +543,7 @@ export function Devices({
|
||||
<span className="sr-only">Ziel auswählen</span>
|
||||
</th>
|
||||
)}
|
||||
{['Hostname', 'Type', 'OS', 'Status', 'Speicher frei / gesamt', 'RAM', 'CPU', 'Kunde'].map(
|
||||
{['Hostname', 'Type', 'OS', 'Status', 'Speicher frei / gesamt', 'RAM frei / gesamt', 'CPU', 'Kunde'].map(
|
||||
(h) => (
|
||||
<th scope="col" key={h}>
|
||||
{h}
|
||||
@@ -591,15 +600,18 @@ export function Devices({
|
||||
<small>Zuletzt {time(d.last_contact_at)}</small>
|
||||
)}
|
||||
</td>
|
||||
<td data-label="Speicher frei / gesamt" className="storage">
|
||||
<Storage d={d} />
|
||||
<td data-label="Speicher frei / gesamt" className="capacity-cell">
|
||||
<Capacity free={d.disk_free_bytes} total={d.disk_total_bytes} />
|
||||
</td>
|
||||
<td data-label="RAM">
|
||||
{gib(d.ram_total_bytes) === null ? unknown : gib(d.ram_total_bytes) + ' GiB'}
|
||||
<td data-label="RAM frei / gesamt" className="capacity-cell" title={measuredAt(d.ram_observed_at)}>
|
||||
<Capacity free={d.ram_available_bytes} total={d.ram_total_bytes} />
|
||||
</td>
|
||||
<td data-label="CPU">
|
||||
{model.primary}
|
||||
{model.secondary && <small>{model.secondary}</small>}
|
||||
<small title={measuredAt(d.cpu_usage_observed_at)}>
|
||||
Auslastung {cpuUsage(d.cpu_usage_percent)}
|
||||
</small>
|
||||
</td>
|
||||
<td data-label="Kunde">{label(d.customer_name)}</td>
|
||||
</tr>
|
||||
|
||||
+1
-1
@@ -98,7 +98,7 @@ section.fleet { padding:0; overflow:hidden; }
|
||||
.fleet-table td[data-label="Type"] { overflow-wrap:anywhere; }
|
||||
.fleet-table input[type=checkbox] { margin:0; }
|
||||
.host-link { padding:0; background:none; color:var(--accent); text-align:left; font-weight:680; margin:0; overflow-wrap:anywhere; }
|
||||
.fleet-table .storage { font-variant-numeric:tabular-nums; }
|
||||
.fleet-table .capacity-cell { font-variant-numeric:tabular-nums; }
|
||||
.capacity-track { display:block; width:min(90px,100%); border-radius:2px; background:#e1ebe6; height:4px; margin-top:6px; overflow:hidden; }
|
||||
.capacity-track i { display:block; height:100%; background:#659d84; }
|
||||
.presence { display:inline-flex; align-items:center; gap:5px; border-radius:20px; padding:2px 8px; background:#edf1ef; color:#4d5f59; font-size:.8rem; font-weight:650; }
|
||||
|
||||
Reference in New Issue
Block a user