Skip to content

Commit ea221ca

Browse files
mohitjha-elasticmohitjha-elastic
authored
azure_blob_storage,azure_network_watcher_nsg,azure_network_watcher_vnet,symantec_enpoint_security: add RBAC-based authentication for azure blob storage input
Implemented RBAC-based authentication support for the azure blob storage input across the following packages: azure_blob_storage, azure_network_watcher_nsg, azure_network_watcher_vnet, and symantec_endpoint_security. This enhancement enables secure data collection using azure active directory RBAC credentials instead of access keys.
1 parent 1a9aba6 commit ea221ca

File tree

21 files changed

+299
-27
lines changed

21 files changed

+299
-27
lines changed

‎packages/azure_blob_storage/agent/input/abs.yml.hbs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,12 @@ pipeline: {{pipeline}}
66
{{#if account_name}}
77
account_name: {{account_name}}
88
{{/if}}
9+
{{#if oauth2}}
10+
auth.oauth2:
11+
client_id: {{client_id}}
12+
client_secret: {{client_secret}}
13+
tenant_id: {{tenant_id}}
14+
{{/if}}
915
{{#if service_account_key}}
1016
auth.shared_credentials.account_key: {{service_account_key}}
1117
{{/if}}

‎packages/azure_blob_storage/changelog.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,12 @@
11
# newer versions go on top
2+
- version: "2.3.0"
3+
changes:
4+
- description: Add RBAC based authentication for azure blob storage input.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/14396
7+
- description: Update the Kibana Version to 8.16.0 to support RABC based authentication for azure blob storage input.
8+
type: enhancement
9+
link: https://github.com/elastic/integrations/pull/14396
210
- version: "2.2.0"
311
changes:
412
- description: Update Kibana constraint to support 9.0.0.

‎packages/azure_blob_storage/manifest.yml‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@ name: azure_blob_storage
33
title: Custom Azure Blob Storage Input
44
description: Collect log data from configured Azure Blob Storage Container with Elastic Agent.
55
type: input
6-
version: "2.2.0"
6+
version: "2.3.0"
77
conditions:
88
kibana:
9-
version: "^8.13.0 || ^9.0.0"
9+
version: "^8.16.0 || ^9.0.0"
1010
categories:
1111
- azure
1212
- observability
@@ -20,13 +20,42 @@ policy_templates:
2020
template_path: abs.yml.hbs
2121
description: Collect log data from configured Azure Blob Storage Container with Elastic Agent.
2222
vars:
23+
- name: oauth2
24+
required: true
25+
show_user: true
26+
title: Collect logs using OAuth2 authentication
27+
description: To collect logs using OAuth2 authentication enable the toggle switch. By default, it will collect logs using service account key or URI.
28+
type: bool
29+
multi: false
30+
default: false
2331
- name: account_name
2432
type: text
2533
title: Account Name
2634
description: |
2735
This attribute is required for various internal operations with respect to authentication, creating service clients and blob clients which are used internally for various processing purposes.
2836
required: true
2937
show_user: true
38+
- name: client_id
39+
type: text
40+
title: Client ID (OAuth2)
41+
description: Client ID of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
42+
required: false
43+
show_user: true
44+
secret: true
45+
- name: client_secret
46+
type: password
47+
title: Client Secret (OAuth2)
48+
description: Client Secret of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
49+
required: false
50+
show_user: true
51+
secret: true
52+
- name: tenant_id
53+
type: text
54+
title: Tenant ID (OAuth2)
55+
description: Tenant ID of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
56+
multi: false
57+
required: false
58+
show_user: true
3059
- name: service_account_key
3160
type: password
3261
title: Service Account Key

‎packages/azure_network_watcher_nsg/_dev/build/docs/README.md‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,15 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
2121
3. Click on **Show** keys to show your **access keys** and **connection strings** and to enable buttons to copy the values.
2222
4. Under key1, find the Key value. Click on the Copy button to copy the **account key**. Same way you can copy the **storage account name** shown above keys.
2323
5. Go to **Containers** under **Data storage** in your storage account to copy the **container name**.
24+
6. Configure the integration using either Service Account Credentials or Microsoft Entra ID RBAC with OAuth2 options. For OAuth2 (Entra ID RBAC), you'll need the Client ID, Client Secret, and Tenant ID. For Service Account Credentials, you'll need either the Service Account Key or the URI to access the data.
2425

25-
**Note**: Enable virtual network flow logs using the steps provided in [reference](https://learn.microsoft.com/en-us/azure/network-watcher/nsg-flow-logs-portal).
26+
- How to setup the `auth.oauth2` credentials can be found in the Azure documentation [here](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app).
27+
- For more details about the Azure Blob Storage input settings, check the [Filebeat documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-azure-blob-storage.html).
28+
29+
Note:
30+
- Enable virtual network flow logs using the steps provided in [reference](https://learn.microsoft.com/en-us/azure/network-watcher/nsg-flow-logs-portal).
31+
- The service principal must be granted the appropriate permissions to read blobs. Ensure that the necessary role assignments are in place for the service principal to access the storage resources. For more information, please refer to the [Azure Role-Based Access Control (RBAC) documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage).
32+
- We recommend assigning either the **Storage Blob Data Reader** or **Storage Blob Data Owner** role. The **Storage Blob Data Reader** role provides read-only access to blob data and is aligned with the principle of least privilege, making it suitable for most use cases. The **Storage Blob Data Owner** role grants full administrative access — including read, write, and delete permissions — and should be used only when such elevated access is explicitly required.
2633

2734
### Enabling the integration in Elastic:
2835

@@ -31,9 +38,18 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
3138
3. Select the "Azure Network Watcher NSG" integration from the search results.
3239
4. Select "Add Azure Network Watcher NSG" to add the integration.
3340
5. While adding the integration, to collect logs via Azure Blob Storage, keep **Collect NSG logs via Azure Blob Storage** toggle on and then configure following parameters:
34-
- account name
35-
- containers
36-
- service account key/service account uri
41+
For OAuth2 (Microsoft Entra ID RBAC):
42+
- Toggle on **Collect logs using OAuth2 authentication**
43+
- Account Name
44+
- Client ID
45+
- Client Secret
46+
- Tenant ID
47+
- Container Details.
48+
49+
For Service Account Credentials:
50+
- Service Account Key or the URI
51+
- Account Name
52+
- Container Details
3753
6. Save the integration.
3854

3955
## Logs Reference

‎packages/azure_network_watcher_nsg/changelog.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,12 @@
11
# newer versions go on top
2+
- version: "1.5.0"
3+
changes:
4+
- description: Add RBAC based authentication for azure blob storage input.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/14396
7+
- description: Update the Kibana Version to 8.16.0 to support RABC based authentication for azure blob storage input.
8+
type: enhancement
9+
link: https://github.com/elastic/integrations/pull/14396
210
- version: "1.4.0"
311
changes:
412
- description: Remove redundant installation instructions.

‎packages/azure_network_watcher_nsg/data_stream/log/agent/stream/abs.yml.hbs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
{{#if account_name}}
22
account_name: {{account_name}}
33
{{/if}}
4+
{{#if oauth2}}
5+
auth.oauth2:
6+
client_id: {{client_id}}
7+
client_secret: {{client_secret}}
8+
tenant_id: {{tenant_id}}
9+
{{/if}}
410
{{#if service_account_key}}
511
auth.shared_credentials.account_key: {{service_account_key}}
612
{{/if}}

‎packages/azure_network_watcher_nsg/data_stream/log/manifest.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,27 @@ streams:
1313
This attribute is required for various internal operations with respect to authentication, creating service clients and blob clients which are used internally for various processing purposes.
1414
required: true
1515
show_user: true
16+
- name: client_id
17+
type: text
18+
title: Client ID (OAuth2)
19+
description: Client ID of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
20+
required: false
21+
show_user: true
22+
secret: true
23+
- name: client_secret
24+
type: password
25+
title: Client Secret (OAuth2)
26+
description: Client Secret of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
27+
required: false
28+
show_user: true
29+
secret: true
30+
- name: tenant_id
31+
type: text
32+
title: Tenant ID (OAuth2)
33+
description: Tenant ID of Azure Account. This is required if 'Collect logs using OAuth2 authentication' is enabled.
34+
multi: false
35+
required: false
36+
show_user: true
1637
- name: service_account_key
1738
type: password
1839
title: Service Account Key

‎packages/azure_network_watcher_nsg/docs/README.md‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,15 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
2121
3. Click on **Show** keys to show your **access keys** and **connection strings** and to enable buttons to copy the values.
2222
4. Under key1, find the Key value. Click on the Copy button to copy the **account key**. Same way you can copy the **storage account name** shown above keys.
2323
5. Go to **Containers** under **Data storage** in your storage account to copy the **container name**.
24+
6. Configure the integration using either Service Account Credentials or Microsoft Entra ID RBAC with OAuth2 options. For OAuth2 (Entra ID RBAC), you'll need the Client ID, Client Secret, and Tenant ID. For Service Account Credentials, you'll need either the Service Account Key or the URI to access the data.
2425

25-
**Note**: Enable virtual network flow logs using the steps provided in [reference](https://learn.microsoft.com/en-us/azure/network-watcher/nsg-flow-logs-portal).
26+
- How to setup the `auth.oauth2` credentials can be found in the Azure documentation [here](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app).
27+
- For more details about the Azure Blob Storage input settings, check the [Filebeat documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-azure-blob-storage.html).
28+
29+
Note:
30+
- Enable virtual network flow logs using the steps provided in [reference](https://learn.microsoft.com/en-us/azure/network-watcher/nsg-flow-logs-portal).
31+
- The service principal must be granted the appropriate permissions to read blobs. Ensure that the necessary role assignments are in place for the service principal to access the storage resources. For more information, please refer to the [Azure Role-Based Access Control (RBAC) documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage).
32+
- We recommend assigning either the **Storage Blob Data Reader** or **Storage Blob Data Owner** role. The **Storage Blob Data Reader** role provides read-only access to blob data and is aligned with the principle of least privilege, making it suitable for most use cases. The **Storage Blob Data Owner** role grants full administrative access — including read, write, and delete permissions — and should be used only when such elevated access is explicitly required.
2633

2734
### Enabling the integration in Elastic:
2835

@@ -31,9 +38,18 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
3138
3. Select the "Azure Network Watcher NSG" integration from the search results.
3239
4. Select "Add Azure Network Watcher NSG" to add the integration.
3340
5. While adding the integration, to collect logs via Azure Blob Storage, keep **Collect NSG logs via Azure Blob Storage** toggle on and then configure following parameters:
34-
- account name
35-
- containers
36-
- service account key/service account uri
41+
For OAuth2 (Microsoft Entra ID RBAC):
42+
- Toggle on **Collect logs using OAuth2 authentication**
43+
- Account Name
44+
- Client ID
45+
- Client Secret
46+
- Tenant ID
47+
- Container Details.
48+
49+
For Service Account Credentials:
50+
- Service Account Key or the URI
51+
- Account Name
52+
- Container Details
3753
6. Save the integration.
3854

3955
## Logs Reference

‎packages/azure_network_watcher_nsg/manifest.yml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
format_version: 3.1.2
22
name: azure_network_watcher_nsg
33
title: Azure Network Watcher NSG
4-
version: "1.4.0"
4+
version: "1.5.0"
55
description: Collect logs from Azure Network Watcher NSG with Elastic Agent.
66
type: integration
77
categories:
@@ -10,7 +10,7 @@ categories:
1010
- security
1111
conditions:
1212
kibana:
13-
version: "^8.13.0 || ^9.0.0"
13+
version: "^8.16.0 || ^9.0.0"
1414
elastic:
1515
subscription: basic
1616
screenshots:
@@ -31,6 +31,15 @@ policy_templates:
3131
- type: azure-blob-storage
3232
title: Collect Azure Network Watcher NSG logs via Azure Blob Storage Input
3333
description: Collecting Azure Network Watcher NSG logs via Azure Blob Storage Input.
34+
vars:
35+
- name: oauth2
36+
required: true
37+
show_user: true
38+
title: Collect logs using OAuth2 authentication
39+
description: To collect logs using OAuth2 authentication enable the toggle switch. By default, it will collect logs using service account key or URI.
40+
type: bool
41+
multi: false
42+
default: false
3443
owner:
3544
github: elastic/security-service-integrations
3645
type: elastic

‎packages/azure_network_watcher_vnet/_dev/build/docs/README.md‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,18 +21,34 @@ Elastic Agent must be installed. For more details, check the Elastic Agent [inst
2121
3. Click **Show keys** to show your **access keys** and **connection strings** to enable buttons to copy the values.
2222
4. Under **key1**, find the key value. Click **Copy** to copy the **account key**. In the same way, copy the storage account name shown above the keys.
2323
5. In your storage account, go to **Data storage** > **Containers** to copy the container name.
24+
6. Configure the integration using either Service Account Credentials or Microsoft Entra ID RBAC with OAuth2 options. For OAuth2 (Entra ID RBAC), you'll need the Client ID, Client Secret, and Tenant ID. For Service Account Credentials, you'll need either the Service Account Key or the URI to access the data.
2425

25-
**Note**: Follow [these steps](https://learn.microsoft.com/en-us/azure/network-watcher/vnet-flow-logs-portal) to enable virtual network flow logs.
26+
- How to setup the `auth.oauth2` credentials can be found in the Azure documentation [here](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app).
27+
- For more details about the Azure Blob Storage input settings, check the [Filebeat documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-azure-blob-storage.html).
28+
29+
Note:
30+
- Follow [these steps](https://learn.microsoft.com/en-us/azure/network-watcher/vnet-flow-logs-portal) to enable virtual network flow logs.
31+
- The service principal must be granted the appropriate permissions to read blobs. Ensure that the necessary role assignments are in place for the service principal to access the storage resources. For more information, please refer to the [Azure Role-Based Access Control (RBAC) documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage).
32+
- We recommend assigning either the **Storage Blob Data Reader** or **Storage Blob Data Owner** role. The **Storage Blob Data Reader** role provides read-only access to blob data and is aligned with the principle of least privilege, making it suitable for most use cases. The **Storage Blob Data Owner** role grants full administrative access — including read, write, and delete permissions — and should be used only when such elevated access is explicitly required.
2633

2734
### Enable the integration in Elastic
2835

2936
1. In Kibana navigate to **Management** > **Integrations**.
3037
2. In the search top bar, type **Azure Network Watcher VNet**.
3138
3. Select the **Azure Network Watcher VNet** integration and add it.
3239
5. To collect logs via Azure Blob Storage, select **Collect VNet logs via Azure Blob Storage** and configure the following parameters:
33-
- account name
34-
- containers
35-
- service account key/service account uri
40+
For OAuth2 (Microsoft Entra ID RBAC):
41+
- Toggle on **Collect logs using OAuth2 authentication**
42+
- Account Name
43+
- Client ID
44+
- Client Secret
45+
- Tenant ID
46+
- Container Details.
47+
48+
For Service Account Credentials:
49+
- Service Account Key or the URI
50+
- Account Name
51+
- Container Details
3652
6. Save the integration.
3753

3854
## Limitations

0 commit comments

Comments
 (0)