[feat, multicast] Multicast Group Support

[zeeshanlakhani]

Note: Review IP Pool extensions first.

Introduces end-to-end multicast group support across control plane and sled-agent, integrated with IP pool extensions required for supporting multicast workflows. This work enables project-scoped multicast groups with lifecycle-driven dataplane programming and exposes an API for operating multicast groups over instances.

This currently points to #9084, which is needed for this to work.

Highlights:

• DB: new multicast_group tables; member lifecycle management
• API: multicast group/member CRUD; source IP validation; VPC/project hierarchy integration with default VNI fallback
• Control plane: RPW reconcilers for groups/members; sagas for dataplane updates atomically at the group level; instance lifecycle hooks and piggybacking
• Dataplane: Dendrite DPD switch programming via trait abstraction; DPD client used in tests
• Sled agent: multicast-aware instance management; network interface configuration for multicast traffic; cross-version testing; OPTE stubs present
• Tests: comprehensive integration suites under nexus/tests/integration_tests/multicast/

Components:

• Database schema: external and underlay multicast groups; member/instance association tables
• Control plane modules: multicast group management, member lifecycle, dataplane abstraction; RPW reconcilers to ensure convergence
• API layer: endpoints and validation; default-VNI semantics when VPC not provided
• Sled agent: OPTE stubs and compatibility shims for older agents

Workflows Implemented:

1. Instance lifecycle integration:

   • "Create" -> resolve VPC/VNI (or default), validate source IPs, create memberships, enqueue group ensure RPW
   • "Start" -> program dataplane via ensure/update sagas; activate member flows after switch ack
   • "Stop" -> deactivate dataplane membership; retain DB membership for fast restart
   • "Delete" -> remove instance memberships; group deletion is explicit
   • "Migrate" -> deactivate on source sled; activate on target; idempotent with ordering guarantees
   • Restart/recovery -> RPWs reconcile desired state; compensations clean up partial programming

2. RPW reconciliation:

   • ensure dataplane switches match database state
   • handle sled migrations and state transitions - Eventual consistency with retry logic

Migrations:

• Apply schema changes in schema/crdb/multicast-group-support/up01.sql (and update dbinit.sql)
• Bump schema versions accordingly

API/Compatibility:

• OpenAPI updated: openapi/nexus.json, openapi/sled-agent/sled-agent-5.0.0-89f1f7.json
• Contains a version change (to v5) as InstanceEnsureBody has been modified to
  include multicast_groups associated with an instance in the underlying sled config
• Regenerate clients where applicable

References:

• RFD 488: https://rfd.shared.oxide.computer/rfd/488
• IP Pool extensions: [feat, multicast] Add IP pool multicast support #9084
• Dendrite PRs (based on recency):
  • [fix] minor: make ssm validation consistent with Omicron expectations dendrite#132
  • [mcast] Lifecycle + API changes for Omicron impl dendrite#109
  • [feature] ASIC-focused multicast replication and dendrite API dendrite#14

Follow-ups include:

• OPTE integration
• commtest extension
• omdb commands are tracked in issues
• pool and group stats

[zeeshanlakhani]

@internet-diglett, others, I added "feature-gating" to this PR, as well as "experimental" tagging for the new entrypoints.