feat: log and store redacted machine config diffs #1305

utkuozdemir · 2025-07-04T19:18:07Z

Change the RedactedClusterMachineConfig controller to also compute diffs between each config change and store them in a new resource.

Additionally, log this diff and include its creation in the audit logs.

Clean up old diffs with both size (count) and time-based retention. Include these diffs in the support bundles.

The resource ID follows the following pattern: <machine-id>-<timestamp>, e.g., 34bafa44-e994-4911-9c1a-609cccefee93-2025-07-04T19:05:40.181Z.

Closes #1119.

Sample Diff Resource

get machineconfigdiff -oyaml
metadata:
    namespace: default
    type: MachineConfigDiffs.omni.sidero.dev
    id: 34bafa44-e994-4911-9c1a-609cccefee93-2025-07-04T19:05:40.181Z
    version: 1
    owner: RedactedClusterMachineConfigController
    phase: running
    created: 2025-07-04T19:05:40Z
    updated: 2025-07-04T19:05:40Z
    labels:
        omni.sidero.dev/cluster: talos-default
        omni.sidero.dev/machine: 34bafa44-e994-4911-9c1a-609cccefee93
        omni.sidero.dev/machine-set: talos-default-control-planes
        omni.sidero.dev/role-controlplane:
    annotations:
        omni.sidero.dev/modified-at: 2025-07-04T19:05:40.181685358Z
spec:
    diff: |-
        @@ -49,6 +49,7 @@
                     enabled: true
                     forwardKubeDNSToHost: true
             nodeLabels:
        +        aaa: bbb
                 node.kubernetes.io/exclude-from-external-load-balancers: ""
         cluster:
             id: Dyt3vOPZ26gywnEh4PQgf7n37Dmvp4KR4pAnu8RP33w=

utkuozdemir · 2025-07-04T19:18:57Z

client/pkg/omni/resources/omni/machine_config_diff.go

+)
+
+// MachineConfigDiff is the diff between two redacted machine configurations.
+type MachineConfigDiff = typed.Resource[MachineConfigDiffSpec, MachineConfigDiffExtension]


Didn't name this as RedactedClusterMachineConfigDiff to avoid over-verbosity.

utkuozdemir · 2025-07-04T19:19:48Z

internal/backend/grpc/support.go

+		{
+			rt:          infra.InfraMachineType,
+			listOptions: clusterQuery,
+		},
+		{
+			rt:          infra.InfraMachineStatusType,
+			listOptions: clusterQuery,
+		},


Added these two into bundles as well, good to have.

Copilot

Pull Request Overview

This PR introduces a new MachineConfigDiff resource to capture, store, and audit diffs of redacted machine configs, and adds logic to periodically clean up old diffs.

Controller changes to compute diffs on each config change, save them, and clean up by age/count
Runtime updates to register caches and controllers for the new diff type
Audit, support bundles, frontend, and client code extended to recognize and handle the new diff resource

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
internal/backend/runtime/omni/state_access.go	Added `MachineConfigDiffType` to access filters
internal/backend/runtime/omni/omni.go	Registered cache/controllers for `MachineConfigDiff`
internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go	Implemented diff computation, saving, and cleanup logic
internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config_test.go	Extended tests for diff creation and retention
internal/backend/runtime/omni/audit/hooks/hooks.go	Added audit hook for diff creation
internal/backend/runtime/omni/audit/data.go	Extended audit `Data` struct with `MachineConfigDiff`
internal/backend/grpc/support.go	Included diffs in support bundle collection
frontend/src/api/resources.ts	Defined `MachineConfigDiffType` constant
frontend/src/api/omni/specs/omni.pb.ts	Added `MachineConfigDiffSpec` type
client/pkg/omni/resources/omni/omni.go	Registered `MachineConfigDiff` resource
client/pkg/omni/resources/omni/machine_config_diff.go	Implemented client type for `MachineConfigDiff`
client/api/omni/specs/omni.proto	Added `MachineConfigDiffSpec` message
client/api/omni/specs/omni.pb.go	Generated code for `MachineConfigDiffSpec`
client/api/omni/specs/omni_vtproto.pb.go	Added VT clone/marshal implementations for diffs

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go

utkuozdemir · 2025-07-04T19:21:13Z

internal/backend/runtime/omni/omni.go

-		clusterValidationOptions(st.Default(), config.Config.EtcdBackup, config.Config.Services.EmbeddedDiscoveryService),
+		clusterValidationOptions(cachedState, config.Config.EtcdBackup, config.Config.Services.EmbeddedDiscoveryService),
 		relationLabelsValidationOptions(),
 		accessPolicyValidationOptions(),
-		authorizationValidationOptions(st.Default()),
+		authorizationValidationOptions(cachedState),
 		roleValidationOptions(),
-		machineSetNodeValidationOptions(st.Default()),
-		machineSetValidationOptions(st.Default(), storeFactory),
-		machineClassValidationOptions(st.Default()),
+		machineSetNodeValidationOptions(cachedState),
+		machineSetValidationOptions(cachedState, storeFactory),
+		machineClassValidationOptions(cachedState),
 		identityValidationOptions(config.Config.Auth.SAML),
 		exposedServiceValidationOptions(),
-		configPatchValidationOptions(st.Default()),
+		configPatchValidationOptions(cachedState),


Some refactor here to use the cached state in more places.

utkuozdemir · 2025-07-04T19:21:52Z

internal/backend/runtime/omni/state_access.go

+		infra.InfraMachineType,
+		infra.InfraMachineStatusType,


These two types also got cluster labels with my recent changes, so I add them here as well.

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go

Change the RedactedClusterMachineConfig controller to also compute diffs between each config change and store them in a new resource. Additionally, log this diff and include its creation in the audit logs. Clean up old diffs with both size (count) and time-based retention. Include these diffs in the support bundles. The resource ID follows the following pattern: `<machine-id>-<timestamp>`, e.g., `34bafa44-e994-4911-9c1a-609cccefee93-2025-07-04T19:05:40.181Z`. Signed-off-by: Utku Ozdemir <[email protected]>

utkuozdemir · 2025-07-10T14:47:57Z

/m

utkuozdemir requested review from oguzkilcan and Copilot July 4, 2025 19:18

utkuozdemir self-assigned this Jul 4, 2025

utkuozdemir added the status/ok-to-test label Jul 4, 2025

utkuozdemir commented Jul 4, 2025

View reviewed changes

Copilot AI reviewed Jul 4, 2025

View reviewed changes

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go Show resolved Hide resolved

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go Show resolved Hide resolved

utkuozdemir commented Jul 4, 2025

View reviewed changes

utkuozdemir force-pushed the feat/redacted-machine-config-diff-store branch from 84df002 to d6a87f0 Compare July 4, 2025 19:22

utkuozdemir marked this pull request as ready for review July 4, 2025 19:25

utkuozdemir force-pushed the feat/redacted-machine-config-diff-store branch 3 times, most recently from 9a435f3 to 39f0cee Compare July 7, 2025 11:30

Unix4ever reviewed Jul 7, 2025

View reviewed changes

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go Show resolved Hide resolved

utkuozdemir force-pushed the feat/redacted-machine-config-diff-store branch 2 times, most recently from 60d4eb1 to 85e1296 Compare July 9, 2025 11:39

Unix4ever approved these changes Jul 10, 2025

View reviewed changes

Unix4ever reviewed Jul 10, 2025

View reviewed changes

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go Outdated Show resolved Hide resolved

Unix4ever reviewed Jul 10, 2025

View reviewed changes

internal/backend/runtime/omni/controllers/omni/redacted_cluster_machine_config.go Outdated Show resolved Hide resolved

utkuozdemir force-pushed the feat/redacted-machine-config-diff-store branch from 85e1296 to f1b47f0 Compare July 10, 2025 12:49

talos-bot added the status/ok-to-merge label Jul 10, 2025

talos-bot merged commit f1b47f0 into siderolabs:main Jul 10, 2025
32 of 33 checks passed

talos-bot removed status/ok-to-test status/ok-to-merge labels Jul 10, 2025

utkuozdemir deleted the feat/redacted-machine-config-diff-store branch July 10, 2025 14:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: log and store redacted machine config diffs #1305

feat: log and store redacted machine config diffs #1305

utkuozdemir commented Jul 4, 2025 •

edited

Loading

Uh oh!

utkuozdemir Jul 4, 2025

Uh oh!

utkuozdemir Jul 4, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

utkuozdemir Jul 4, 2025 •

edited

Loading

Uh oh!

utkuozdemir Jul 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

utkuozdemir commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

feat: log and store redacted machine config diffs #1305

feat: log and store redacted machine config diffs #1305

Conversation

utkuozdemir commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sample Diff Resource

Uh oh!

utkuozdemir Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

utkuozdemir Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

utkuozdemir Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utkuozdemir Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

utkuozdemir commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

utkuozdemir commented Jul 4, 2025 •

edited

Loading

utkuozdemir Jul 4, 2025 •

edited

Loading