feat : Added Event Streams DA (Standard and Security-Enforced Variation) #397

Khuzaima05 · 2025-05-05T15:53:38Z

Description

Added Event Streams DA (Standard and Security-Enforced Variation)

issue: https://github.ibm.com/GoldenEye/issues/issues/13014 , https://github.ibm.com/GoldenEye/issues/issues/13484,
https://github.ibm.com/GoldenEye/issues/issues/13582

Release required?

No release
Patch release (x.x.X)
Minor release (x.X.x)
Major release (X.x.x)

Release notes content

Run the pipeline

If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.

Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:

/run pipeline

Checklist for reviewers

If relevant, a test for the change is included or updated with this PR.
If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

Khuzaima05 · 2025-05-14T07:07:07Z

/run pipeline

Khuzaima05 · 2025-05-14T07:47:32Z

/run pipeline

Khuzaima05 · 2025-05-14T08:55:39Z

The upgrade test is failing because it clones the main branch and looks for the standard DA repo, which doesn’t exist in main yet—we’re introducing it in our current branch. The main branch only contains the quickstart and enterprise DAs, which we’ve are updating. Hence, we are skipping the upgrade test.

Khuzaima05 · 2025-05-14T08:58:00Z

/run pipeline

Khuzaima05 · 2025-05-14T14:10:19Z

/run pipeline

Khuzaima05 · 2025-05-14T16:39:27Z

/run pipeline

Khuzaima05 · 2025-05-14T17:03:25Z

Catalog Tile Overview and Features:

Security Enforced Variant:

Standard Variant:

Khuzaima05 · 2025-05-14T17:03:52Z

/run pipeline

Khuzaima05 · 2025-05-14T17:08:49Z

/run pipeline

Khuzaima05 · 2025-05-15T07:57:10Z

/run pipeline

Khuzaima05 · 2025-05-19T07:02:30Z

/run pipeline

Khuzaima05 · 2025-05-19T07:50:08Z

/run pipeline

Khuzaima05 · 2025-05-22T06:53:57Z

/run pipeline

ocofaigh · 2025-05-26T09:18:30Z

@Khuzaima05 Can we re-discuss the variation name "Standard" on tomorrow deep dive? I would really like to find a better name, but as we dicsussed "Fully configurable" doesn't make sense in this case

ocofaigh · 2025-05-27T12:50:41Z

As per deep dive, 2 variation names should be:

Quickstart
Security-enforced

Khuzaima05 · 2025-05-29T12:25:21Z

/run pipeline

Ak-sky

Left few comments, also need to update the URL in variables in exisitng code to use security-enforced instead of enterprise.
Few typos below needs to be fixed, PR in progress to add codespell to check for typos.

modules/fscloud/variables.tf:10: Steams ==> Streams
modules/fscloud/variables.tf:16: Steams ==> Streams
solutions/quickstart/README.md:45: Steams ==> Streams
solutions/quickstart/README.md:46: Steams ==> Streams
examples/basic/variables.tf:27: Steams ==> Streams
examples/fscloud/variables.tf:27: Steams ==> Streams
modules/fscloud/README.md:29: Steams ==> Streams
modules/fscloud/README.md:47: Steams ==> Streams
solutions/quickstart/main.tf:37: intance ==> instance, intense
README.md:164: Steams ==> Streams
solutions/security-enforced/variables.tf:66: Steams ==> Streams
solutions/security-enforced/variables.tf:72: Steams ==> Streams
solutions/security-enforced/README.md:5: wan't ==> want, wasn't
solutions/security-enforced/README.md:12: instace ==> instance
solutions/security-enforced/README.md:57: Steams ==> Streams
solutions/security-enforced/README.md:58: Steams ==> Streams
variables.tf:27: Steams ==> Streams
examples/complete/variables.tf:27: Steams ==> Streams
examples/complete/variables.tf:33: Steams ==> Streams
solutions/security-enforced/main.tf:169: intance ==> instance, intense
solutions/quickstart/variables.tf:76: Steams ==> Streams
solutions/quickstart/variables.tf:82: Steams ==> Streams

Ak-sky · 2025-06-04T14:46:35Z

solutions/quickstart/variables.tf

 variable "topics" {
  type = list(object(
    {
      name       = string
      partitions = number
-      config     = map(string)
+      config     = object({})


Suggested change

config = object({})

config = map(string)

Refer this.

Ak-sky · 2025-06-04T21:01:53Z

solutions/security-enforced/catalogValidationValues.json.template

  "prefix": $PREFIX,
-  "resource_group_name": $PREFIX,
+  "existing_resource_group_name": $PREFIX,


Suggested change

"existing_resource_group_name": $PREFIX,

"existing_resource_group_name": "geretain-test-event-streams",

Ak-sky · 2025-06-04T21:09:08Z

solutions/security-enforced/README.md

+
+## Prerequisites
+- An existing resource group
+- An existing KMS instance (or key) if you wan't to encrypt the Event Streams instance.


Here it should not be if as this is security-enforced variation.

Ak-sky · 2025-06-05T18:10:56Z

cra-config.yaml

      TF_VAR_provider_visibility: "public"
-      TF_VAR_use_existing_resource_group: false
-      TF_VAR_kms_endpoint_type: "public"
+      TF_VAR_prefix: "test"


Suggested change

TF_VAR_prefix: "test"

TF_VAR_prefix: "test-se"

Ak-sky · 2025-06-05T18:12:22Z

cra-config.yaml

    CRA_IGNORE_RULES_FILE: "cra-tf-validate-ignore-rules.json" # CRA Ignore file to use. If not provided, it checks the repo root directory for `cra-tf-validate-ignore-rules.json`
    PROFILE_ID: "fe96bd4d-9b37-40f2-b39f-a62760e326a3"         # SCC profile ID (currently set to 'IBM Cloud Framework for Financial Services' '1.7.0' profile).
    CRA_ENVIRONMENT_VARIABLES:
      TF_VAR_existing_kms_instance_crn: "crn:v1:bluemix:public:hs-crypto:us-south:a/abac0df06b644a9cabc6e44f55b3880e:e6dce284-e80f-46e1-a3c1-830f7adff7a9::"
-      TF_VAR_resource_group_name: "test-event-s-cra"
+      TF_VAR_existing_resource_group_name: "Default"


Suggested change

TF_VAR_existing_resource_group_name: "Default"

TF_VAR_existing_resource_group_name: "geretain-test-event-streams"

Ak-sky · 2025-06-05T19:15:05Z

solutions/quickstart/variables.tf

  default     = []
-  description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/quickstart/DA-types.md#service-credential-secrets)."
+  nullable    = false
+  description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#service-credential-secrets)."


Suggested change

description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#service-credential-secrets)."

description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-types.md#service-credential-secrets)."

Ak-sky · 2025-06-05T19:17:53Z

solutions/security-enforced/README.md

+- Topics to apply to resources.
+- Schemas to apply to resources.
+- Mirroring of existing event stream instace.
+


Missing points on

Quotas

Metrices

Ak-sky · 2025-06-05T19:21:59Z

solutions/security-enforced/variables.tf

+      })))
+    })
+  }))
+  description = "The list of schema objects. Include the `schema_id`, `type` and `name` of the schema in the `schema` object. Learn more: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-schemas."


Suggested change

description = "The list of schema objects. Include the `schema_id`, `type` and `name` of the schema in the `schema` object. Learn more: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-schemas."

description = "The list of schema objects. Include the `schema_id`, `type` and `name` of the schema in the `schema` object. Learn more: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-schemas-topics-cbr.md#options-with-schemas."

Ak-sky · 2025-06-05T19:22:34Z

solutions/security-enforced/variables.tf

@@ -139,6 +134,36 @@ variable "quotas" {
  default     = []
 }

+variable "service_credential_names" {
+  description = "The mapping of names and roles for service credentials that you want to create for the Event streams.[Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#svc-credential-name)"


Suggested change

description = "The mapping of names and roles for service credentials that you want to create for the Event streams.[Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#svc-credential-name)"

description = "The mapping of names and roles for service credentials that you want to create for the Event streams.[Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-types.md#svc-credential-name)"

Ak-sky · 2025-06-05T19:24:28Z

solutions/security-enforced/variables.tf

+    }))) }))
+    enforcement_mode = string
+  }))
+  description = "A single context-based restriction rule to create. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-cbr)."


Suggested change

description = "A single context-based restriction rule to create. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-cbr)."

description = "A single context-based restriction rule to create. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-schemas-topics-cbr.md#options-with-cbr)."

Khuzaima-Shakeel added 2 commits May 5, 2025 14:10

add : fully configurable version

431986d

updated json and DA

3ac6a6f

Khuzaima05 self-assigned this May 5, 2025

Khuzaima-Shakeel added 8 commits May 7, 2025 11:53

updated json

0bb63c1

updated json

5b66874

added security and standard DA

05e774d

Merge branch 'main' into issue_13014

1337681

added security and standard DA

f81428f

updated PR

1028852

updated PR

9b69d3a

fix conflict

84e265f

fix pipeline

6811dc4

Khuzaima05 changed the title ~~[WIP] feat : Added Event Streams DA (Fully configurable variations)~~ [WIP] feat : Added Event Streams DA (Standard and Security-Enforced Variation) May 14, 2025

fix ref_arch

b81e550

SKIP UPGRADE TEST

c50fcda

update catalog.json

c9d3c8e

Khuzaima05 marked this pull request as ready for review May 14, 2025 16:39

Khuzaima05 requested review from Ak-sky and akocbek as code owners May 14, 2025 16:39

Khuzaima05 changed the title ~~[WIP] feat : Added Event Streams DA (Standard and Security-Enforced Variation)~~ feat : Added Event Streams DA (Standard and Security-Enforced Variation) May 14, 2025

fix ibm_catalog

059c16f

update endpoint

ffc76d5

Khuzaima-Shakeel added 2 commits May 15, 2025 13:13

updated output

d86f7f6

updated account infra dependency

06c6090

Merge branch 'main' into issue_13014

008faf1

fix pre-commit

73eea2c

fix json

bd2d28e

Merge branch 'main' into issue_13014

a891041

updated variation name

fe695f1

Ak-sky requested changes Jun 5, 2025

View reviewed changes

	"existing_resource_group_name": $PREFIX,
	"existing_resource_group_name": "geretain-test-event-streams",

	TF_VAR_existing_resource_group_name: "Default"
	TF_VAR_existing_resource_group_name: "geretain-test-event-streams"

	description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#service-credential-secrets)."
	description = "Service credential secrets configuration for Event Streams. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-types.md#service-credential-secrets)."

	description = "The list of schema objects. Include the `schema_id`, `type` and `name` of the schema in the `schema` object. Learn more: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-schemas."
	description = "The list of schema objects. Include the `schema_id`, `type` and `name` of the schema in the `schema` object. Learn more: https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-schemas-topics-cbr.md#options-with-schemas."

	description = "The mapping of names and roles for service credentials that you want to create for the Event streams.[Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-types.md#svc-credential-name)"
	description = "The mapping of names and roles for service credentials that you want to create for the Event streams.[Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-types.md#svc-credential-name)"

	description = "A single context-based restriction rule to create. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/enterprise/DA-schemas-topics-cbr.md#options-with-cbr)."
	description = "A single context-based restriction rule to create. [Learn more](https://github.com/terraform-ibm-modules/terraform-ibm-event-streams/tree/main/solutions/security-enforced/DA-schemas-topics-cbr.md#options-with-cbr)."

feat : Added Event Streams DA (Standard and Security-Enforced Variation) #397

Are you sure you want to change the base?

feat : Added Event Streams DA (Standard and Security-Enforced Variation) #397

Conversation

Khuzaima05 commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Release required?

Release notes content

Run the pipeline

Checklist for reviewers

For mergers

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 14, 2025

Uh oh!

Khuzaima05 commented May 15, 2025

Uh oh!

Khuzaima05 commented May 19, 2025

Uh oh!

Khuzaima05 commented May 19, 2025

Uh oh!

Khuzaima05 commented May 22, 2025

Uh oh!

ocofaigh commented May 26, 2025

Uh oh!

ocofaigh commented May 27, 2025

Uh oh!

Khuzaima05 commented May 29, 2025

Uh oh!

Ak-sky left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Khuzaima05 commented May 5, 2025 •

edited

Loading

Ak-sky left a comment •

edited

Loading