Node Reboot triggered NMC reconciler, but not seeing the drivers getting installed on the host kernel.

[quic-vmellemp]

Scenario:

1. triggered a node reboot around: 04:13:13
2. ---Node rebooted---
3. After reboot, NMC last transition time around: 04:16
4. from the logs, I see NMC reconciler getting triggered a couple of times after the node reboot.
5. Module is in Ready status now.
   Issue: When I login to the node's terminal and try lsmod, I don't see the modules on the host kernel.

KMM version in use: 2.3.0(have observed similar issue with 2.2.0 as well).

Attaching logs of KMM Controller before and after node reboot.

Before Node Reboot: kmm-operator-controller-564dc74655-b2gxc-manager.log

After Node Reboot: kmm-operator-controller-564dc74655-b2gxc-manager (1).log

Recovery steps: I had to delete/create the module instance to recover.

[ybettan]

Thank you for this issue @quic-vmellemp

Can you please share the following information:

1. The node object after the reboot (oc get node/<node name> -o yaml)
2. The pods in the modules namespace before you delete/recreate the module to recover (and their describe, logs)
3. The module object (oc get module/<modulename> -n <namespace> -o yaml)
4. The NMC object (oc get nmc/<name> -n <namespace> -o yaml)
5. The MIC object (oc get moduleimagesconfig/<module-name> -n <namespace> -o yaml)
6. The MBSC object (oc get modulebuildsignconfig/<module-name> -n <namespace> -o yaml)