Skip to content

Config Drift Detection

Background scanner that fingerprints VM/CT, network, storage and cluster-options config across every cluster every 6 hours and diffs the current state against an admin-set baseline. When a diff appears, a drift event is raised; the admin can acknowledge it or promote the change into a new baseline so it stops being flagged. "Why did vmbr3 lose its VLAN tag last Friday?" becomes a one-line audit-log query.

What's fingerprinted

  • VMs / CTs - boot order, CPU/RAM, disk list, NIC list, cloud-init config, tags, HA group, start-on-boot.
  • Networks - bridges, bonds, slaves, VLAN tagging, MTU.
  • Storage - type, content list, target nodes, max-files / prune settings.
  • Cluster options - datacenter-level configuration.

False-positive avoidance

Volatile fields are normalized before hashing. CSV-style fields (tags, content, nodes) are sorted alphabetically - Proxmox returns these in non-deterministic order, so naïve hashing would flip on every read. Live counters (uptime, mem_used) are excluded entirely.

From 1.3:

  • Offline nodes: when an object is missing because its node is offline, it is not reported as removed. The outage is recorded once, marked as node offline, without an alert, and the row is cleared when the node is back. A real removal still shows up after the node returns.
  • Reads that failed: when the guest list, storage, network or cluster options cannot be read (for example a timeout on a large cluster), the scan skips the removals for what it could not read and lists the unread kinds in its result. Resetting the baseline keeps what it could not read.

Permissions

Reading drift status is open to every user who sees the cluster; the event list needs a whole-cluster user. Acknowledging events and running a scan need admin.audit. From 1.3, changing the baseline (reset, a scan that seeds it, acknowledging with promote) also needs cluster.config, so a read-only auditor cannot hide drift.

Configuration

  • Cadence: default 6h, configurable 1h–24h.
  • Severity: drift event severity per object type (info / warning / critical).
  • Per-cluster on/off toggle.

API

Method Endpoint Description
GET /api/clusters/{id}/drift/status Open-event counts by kind + severity
GET /api/clusters/{id}/drift/events Drift events with diff
POST /api/clusters/{id}/drift/scan Trigger an out-of-band scan (admin)
POST /api/clusters/{id}/drift/baseline Set / promote the baseline (admin)
POST /api/drift/events/{eid}/acknowledge Acknowledge one event (admin)