Sysops-15175: Forwarder upgrade #6
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mplate Add integration test note to template
Add aws_account tag to enhanced metrics and Lambda logs
This extends "extractResourceId" function to extract "source" tag from applications that are deployed at Tenant level. The existing behavior is to discard all resourceId that don't begin with "/subscriptions/" thus it doesn't handle Tenant level resourceId that begin with "/tenants/". Adds a else if block to handle resourceIds with Tenant level deployemnts(Like Active Directory) I have kept the source and tag extraction logic similar to what currently exists.
…d-warnings Fixes DataDog#230 as cfn-lint now passes with no errors
…s-and-S3-to-invoke-Forwarder Fixes DataDog#207 by allowing CloudWatch Logs & S3 to invoke the Forwarder
Run cfn-lint against the AWS Datadog Forwarder CloudFormation Template
Exclude unuseful files from zip
…-installation Add manual installation steps for the AWS Datadog forwarder
Co-Authored-By: Stephen Pinkerton <[email protected]>
…rmissions Terraform installation and permissions
Support sending data through a proxy
Co-authored-by: stroem <[email protected]>
…nd (DataDog#350) * update azure tests * add and update binary tests * lint * rename and cleanup * fix setUp
* handle malformed metrics * fix tests * fix formatting
* Use s3 for tags cache * Update unit tests * Update snapshot for integration test * Case when s3 cache is expired * Set ddfetchlambdatags to true by default * Update label * Move ddfetchlambdatags out of experimental * Allow additional tags for internal metrics * Add unit test for client error case * New line at EOF * Patch aws boto3 pagination calls * Import env vars from settings * Script to run unittests (sets env vars) * Add env vars to github workflow * Try quotes * Update description for DDFetchLambdaTags * Fix env var name * Lambda function for cache test * Remove lambda functions after external lambdas test * Integration test for s3 cache * Update events with cache lambda name * Update snapshots * Run cache integration test by default * Remove guids from snapshots * Update snapshots * Turn cache tests off by default * Snapshots for when cache tests are on * Snapshots for when cache test is off * Undo lambda function name change * Enable locking for S3 cache * Add try catch for cache lock creation * Update unit tests * Update snapshot for cache integration test * Fix concat issue * Fix operand type issue * Add deletion permission for s3 bucket * Group DdFetchLambdaTags under advanced * Add debug logs
Had to check the source code to confirm this was supported https://github.com/DataDog/datadog-serverless-functions/blob/e9180b1bbdd9ccef14d4c498d47d2901c28bbea5/aws/logs_monitoring/lambda_function.py#L624 There is sort of/kind of a reference to this, but it's worded in a very strange way. Logging isn't supported, but it is, by picking an ARN that you "want to use". I'm assuming what it really means, is select *the* datadog logs forwarder, you set up here<link to cloudformation forwarder documentation>
…taDog#337) * feat: add KMS permission to get log from Encrypted S3 Bucket * fix: remove none necessary permissiong If you use a customer managed CMK, the Forwarder only needs kms:Decrypt for downloading, typically CloudTrail SSE require CMK and this permission will work for the case. if you use the AWS managed CMK (i.e., key aws/s3), no extra permissions are required for the Forwarder to download the object.
* Add support for S3 events triggered via SNS - When S3 events are configured to push to SNS topics - The DD Lambda Forwarder subscribes to the SNS topic - Then in DD console the event shows up as SNS with SNS event data being logged rather than the S3 log being ingested - This change extracts the S3 event from the SNS topic and then extract and ingests the S3 log from it * removing redundant checks Co-authored-by: Tian Chu <[email protected]> * simpler validations using try-catch Co-authored-by: Tian Chu <[email protected]> Co-authored-by: Tian Chu <[email protected]>
* inital commit for new sources * update tests and replace allSettled * make empty string checking a bit more clear * handle trailing slash
…s-functions into DataDog-master # Conflicts: # aws/logs_monitoring/lambda_function.py
mpuaplacester
requested review from
nedpesca,
tjcmoldes and
melaguinaldoplacester
melaguinaldoplacester
approved these changes
Sep 29, 2020
tjcmoldes
approved these changes
Sep 29, 2020
nedpesca
approved these changes
Sep 29, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.