Merge pull request #21 from igfasouza/master

added new samples for Datadog and Syniverse

Author: sachin-pikle

samples/basic-observability/README.md

@@ -0,0 +1,6 @@
+# Basic Observability Guidance
+
+This example shows how you can get basic observability using out-of-the-box service metrics and alarms for
+* [Functions](functions.md)
+* [Notifications Service](notifications.md)
+* [Service Connector Hub](service-connector-hub.md)

samples/basic-observability/functions.md

@@ -0,0 +1,47 @@
+# Basic Guidance for Monitoring your Functions
+
+This guide shows how you can use a simple email alert to monitor the status of your functions.
+
+For more details, refer to the [technical documentation](https://docs.oracle.com/en-us/iaas/Content/Functions/Reference/functionsmetrics.htm).
+
+
+## Create a topic and a subscription for the Notification Service
+
+1. From the [OCI Console](https://cloud.oracle.com) navigation menu, select **Application Integration**, and then select **Notifications**
+
+2. Click **Create Topic** and create a topic with **my_function_status** name
+
+3. Choose your topic, click **Create Subscription** and create a subscription with your email:
+    * **Protocol**: Select Email 
+    * **Email**: Enter your email address
+
+4. The subscription will be created in "Pending" status. You will receive a confirmation email 
+and will need to click on the link in the email to confirm your email address.
+
+
+## Check Metrics and create an alarm definition from Metrics
+
+1. From the navigation menu, select **Developer Services**, and then select **Functions**
+
+2. Choose the application and the function that you want to monitor
+
+3. From the Metrics page, go to the "Functions Errors" chart, click on **Options** and **Create an Alarm on this Query**
+
+4. Add a **name** and under **Notifications** select **Destination service** as notification service, select the compartment **your_compartment**, and select **Topic** as my_function_status
+
+
+
+## Create an alarm definition from scratch
+
+1. From the navigation menu, select **Monitoring**, and then select **Alarm Definitions**
+
+2. Click **Create Alarm** and use the following example:
+    * **Alarm Name**: my_function_status
+    * **Alarm severity**: Critical
+    * **Alarm body**: OCI Alarm: my function 
+
+3. On **Metric description** select the compartment **your_compartment**, select **Metric namespace** as oci_faas, **Metric name** as FunctionResponseCount, **Interval** as 1m, and **Statistic** as Mean
+
+4. On **Trigger rule** select Value greater than 1
+
+6. On **Notification** select **Destination service** as notification service, select the compartment **your_compartment**, and select **Topic** as my_function_status

samples/basic-observability/notifications.md

@@ -0,0 +1,45 @@
+# Basic Guidance for Monitoring your Notifications Topics
+
+This guide shows how you can use a simple email alert to monitor the status of your Oracle Notifications Service (ONS) Topics.
+
+For more details, refer to the [technical documentation](https://docs.oracle.com/en-us/iaas/Content/Notification/Reference/notificationmetrics.htm).
+
+## Create a topic and a subscription for the Notification Service
+
+1. From the [OCI Console](https://cloud.oracle.com) navigation menu, select **Application Integration**, and then select **Notifications**
+
+2. Click **Create Topic** and create a topic with **my_ons_status** name
+
+3. Choose your topic, click **Create Subscription** and use the following example:
+    * **Protocol**: Select Email 
+    * **Email**: Enter your email address
+
+4. The subscription will be created in "Pending" status. You will receive a confirmation email 
+and will need to click on the link in the email to confirm your email address.
+
+
+## Check Metrics and create an alarm definition from Metrics
+
+1. From the navigation menu, select **Application Integration**, and then select **Notification**
+
+2. Choose the notification topic that you want to monitor and click on **Metrics** link under **Resources** in the left navigation pane
+
+3. From the chart that you want to add the alarm e.g. "Failed Messages Count", click on **Options** and **Create an Alarm on this Query**
+
+4. Add a **name** and on **Notification** select **Destination service** as Notifications service, select the compartment **your_compartment**, and select **Topic** as my_ons_status
+
+
+## Create an alarm definition from scratch
+
+1. From the navigation menu, select **Monitoring**, and then select **Alarm definitions**
+
+2. Click **Create Alarm** and use the following example:
+    * **Alarm Name**: my_ons_status
+    * **Alarm severity**: Critical
+    * **Alarm body**: OCI Alarm: my notification 
+
+3. On **Metric description** select the compartment **your_compartment**, select **Metric namespace** as oci_notification, **Metric name** as FailedMessagesSize, **Interval** as 1m, and **Statistic** as Sum
+
+4. On **Trigger rule** select Value greater than 1
+
+6. On **Notification** select **Destination service** as notification service, select the compartment **your_compartment**, and select **Topic** as my_ons_status

samples/basic-observability/service-connector-hub.md

@@ -0,0 +1,45 @@
+# Basic Guidance for Monitoring your Service Connectors
+
+This guide shows how you can use a simple email alert to monitor the status of your Service Connectors.
+
+For more details, refer to the [technical documentation](https://docs.oracle.com/en-us/iaas/Content/service-connector-hub/metrics.htm).
+
+## Create a topic and a subscription for the Notification Service
+
+1. From the [OCI Console](https://cloud.oracle.com) navigation menu, select **Application Integration**, and then select **Notifications**
+
+2. Click **Create Topic** and create a topic with **my_sch_status** name
+
+3. Choose your topic, click **Create Subscription** and use the following example:
+    * **Protocol**: Select Email 
+    * **Email**: Enter your email address
+
+4. The subscription will be created in "Pending" status. You will receive a confirmation email 
+and will need to click on the link in the email to confirm your email address.
+
+
+## Check Metrics and create an alarm definition from Metrics
+
+1. From the navigation menu, select **Logging**, and then select **Service Connectors**
+
+2. Choose the connector that you want to monitor and click on **Metrics** link under **Resources** in the left navigation pane
+
+3. From the metrics chart that you want to add the alarm e.g., "Service Connector Hub Errors", click on **Options** and **Create an Alarm on this Query**
+
+4. Add a **name** and on **Notification** select **Destination service** as notification service, select the compartment **your_compartment**, and select **Topic** as my_sch_status
+
+
+## Create an alarm definition from scratch
+
+1. From the navigation menu, select **Monitoring**, and then select **Alarm definitions**
+
+2. Click **Create Alarm** and use the following example:
+    * **Alarm Name**: my_sch_status
+    * **Alarm severity**: Critical
+    * **Alarm body**: OCI Alarm: my sch 
+
+3. On **Metric description** select the compartment **your_compartment**, select **Metric namespace** as oci_service_connector_hub, **Metric name** as ServiceConnectorHubErros, **Interval** as 1m, and **Statistic** as Mean
+
+4. On **Trigger rule** select Value greater than 1
+
+6. On **Notification** select **Destination service** as notification service, select the compartment **your_compartment**, and select **Topic** as my_sch_status

samples/oci-logs-datadog/README.md

@@ -0,0 +1,100 @@
+## Monitor Oracle Cloud Infrastructure with Datadog
+
+Move logs from Oracle Cloud Infrastructure Logging service to Datadog for the following processes: Running analytics on service, audit, and custom logs in Datadog; Consolidating logs from multicloud and hybrid environments;
+
+
+![workflow](./images/workflow.png)
+
+More
+- [Detailed Steps](https://docs.oracle.com/en/learn/logs_oci_datadog/#introduction)
+
+
+As you make your way through this tutorial, look out for this icon ![user input icon](./images/userinput.png).
+Whenever you see it, it's time for you to perform an action.
+
+
+## Prerequisites
+
+Before you deploy this sample function, make sure you have run steps A, B 
+and C of the [Oracle Functions Quick Start Guide for Cloud Shell](https://www.oracle.com/webfolder/technetwork/tutorials/infographics/oci_functions_cloudshell_quickview/functions_quickview_top/functions_quickview/index.html)
+* A - Set up your tenancy
+* B - Create application
+* C - Set up your Cloud Shell dev environment
+
+
+## List Applications 
+
+Assuming you have successfully completed the prerequisites, you should see your 
+application in the list of applications.
+
+```
+fn ls apps
+```
+
+
+## Review and customize the function
+
+Review the following files in the current folder:
+* the code of the function, [func.py](./func.py)
+* its dependencies, [requirements.txt](./requirements.txt)
+* the function metadata, [func.yaml](./func.yaml)
+
+
+## Create a Datadog API key
+
+* In your Datadog account, select Integrations, and under the APIs, click create API key. This will generate your API key.
+* Replace the value of the DATADOG_TOKEN in the [func.yaml](./func.yaml)
+
+
+## Deploy the function
+
+In Cloud Shell, run the `fn deploy` command to build *this* function and its dependencies as a Docker image, 
+push the image to the specified Docker registry, and deploy *this* function to Oracle Functions 
+in the application created earlier:
+
+![user input icon](./images/userinput.png)
+```
+fn -v deploy --app <app-name>
+```
+e.g.,
+```
+fn -v deploy --app myapp
+```
+
+
+### Test
+
+In Cloud Shell, run the following `fn invoke` command to unit test this function:
+
+![user input icon](./images/userinput.png)
+```
+fn invoke <app-name> <function-name> < test-logs.json
+```
+e.g., 
+```
+fn invoke myapp oci-logs-to-datadog-python < test-logs.json
+```
+
+You should see the logs appear in Datadog.
+
+
+## Configure the logs you want to capture
+
+1. From the [OCI Console](https://cloud.oracle.com) navigation menu, select **Logging**, and then select **Log Groups**.
+
+2. Click Create Log Group, select your compartment, add a Name and Description
+
+3. Select Logs in the left menu, click Enable Service Log, select your compartment, select Log Category on Service and fill the rest of the fields appropriately.
+
+
+## Create a Service Connector for reading logs from Logging and send to Functions
+
+1. From the navigation menu, select **Logging**, and then select **Service Connectors**.
+
+2. Click Create Connector, add a Name, Description, select the compartment, select the Source as **Logging** and Target as **Functions**.
+    
+3. On Configure Source connection, select the compartment, select the Log Group created earlier.
+
+4. On Configure Target connection, select the compartment and select the Function. If prompted to create a policy for writing to functions, click Create.
+
+

samples/oci-logs-datadog/func.py

@@ -0,0 +1,40 @@
+#
+# oci-logs-datadog version 1.0.
+#
+# Copyright (c) 2020 Oracle, Inc.  All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
+#
+
+import io
+import oci
+import os
+import json
+import requests
+import logging
+from fdk import response
+
+
+"""
+This Function receives the logging json and invokes the Datadog endpoint for ingesting logs.
+Logging Format Overview https://docs.cloud.oracle.com/en-us/iaas/Content/Logging/Reference/top_level_logging_format.htm#top_level_logging_format
+If this Function is invoked with more than one log the function go over each log and invokes the Datadog endpoint for ingesting one by one.
+"""
+def handler(ctx, data: io.BytesIO=None):
+    try:
+        logs = json.loads(data.getvalue())
+
+#       go over each log and invokes the Datadog endpoint for ingesting one by one.
+        for item in logs: 
+       
+#           Datadog endpoint URL and token to call the REST interface. These values are defined in func.yaml
+            datadoghost = os.environ['DATADOG_HOST']
+            datadogtoken = os.environ['DATADOG_TOKEN']
+
+#           Call the Datadog with the payload.
+            headers = {'Content-type': 'application/json', 'DD-API-KEY': datadogtoken}
+            x = requests.post(datadoghost, data = json.dumps(item), headers=headers)
+            logging.getLogger().info(x.text)
+       
+    except (Exception, ValueError) as ex:
+        logging.getLogger().info(str(ex))
+        return

samples/oci-logs-datadog/func.yaml

@@ -0,0 +1,8 @@
+schema_version: 20180708
+name: oci-logs-to-datadog-python
+version: 0.0.48
+runtime: python
+entrypoint: /python/bin/fdk /function/func.py handler
+config:
+  DATADOG_HOST: https://http-intake.logs.datadoghq.com/v1/input
+  DATADOG_TOKEN: ZZZZZzzzzz

samples/oci-logs-datadog/images/userinput.png

@@ -0,0 +1,3 @@
+fdk
+requests
+oci

samples/oci-logs-datadog/images/workflow.png

@@ -0,0 +1 @@
+{"results":[{"data":{"logContent":{"data":{"additionalDetails":{"bucketName":"bucket-test","eTag":"e34d66e8-ac09-4XXXxxxXXX9e885299be7","namespace":"yourTenancyName","publicAccessType":"NoPublicAccess","versioning":"Disabled"},"availabilityDomain":"PHX-AD-2","compartmentId":"ocid1.compartment.oc1..ZZZzzz","compartmentName":"syniverse-compartmment","definedTags":{},"eventGroupingId":"phx-1:hJX79im5JP-ZZZzzz","eventName":"CreateBucket","freeformTags":{},"identity":{"authType":"natv","callerId":null,"callerName":null,"consoleSessionId":null,"credentials":"ZZZzzz","ipAddress":"37.XXX.XXX.XXX","principalId":"ocid1.user.oc1..ZZZzzz","principalName":"ZZZzzz","tenantId":"ocid1.tenancy.oc1..ZZZzzz","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0"},"message":"Bucket created.","request":{"action":"POST","headers":{"Accept":["/"],"Accept-Encoding":["gzip, deflate, br"],"Accept-Language":["en"],"Connection":["keep-alive"],"Content-Length":["250"],"Origin":["https://object-storage.plugins.oci.oraclecloud.com"],"Referer":["https://object-storage.plugins.oci.oraclecloud.com/buckets"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0"],"authorization":["Signature keyId=\"XXXXXxxxdI-7ew\",version=\"1\",algorithm=\"rsa-sha256\",headers=\"(request-target) host content-length accept-language content-type opc-request-id x-content-sha256 x-date\",signature=\"*****\""],"content-type":["application/json"],"opc-request-id":["ZZZzzz"],"x-content-sha256":["1ZZZzzz="],"x-date":["Wed, 16 Dec 2020 11:44:39 GMT"]},"id":"phx-1:ZZZzzz","parameters":{"param0":["yourTenancyName"]},"path":"/n/yourTenancyName/b/"},"resourceId":"/n/yourTenancyName/b/","response":{"headers":{"Content-Length":["765"],"Content-Type":["application/json"],"access-control-allow-credentials":["true"],"access-control-allow-methods":["POST,PUT,GET,HEAD,DELETE,OPTIONS"],"access-control-allow-origin":["*"],"access-control-expose-headers":["access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,content-type,date,etag,location,opc-client-info,opc-request-id,x-api-id"],"date":["Wed, 16 Dec 2020 11:44:40 GMT"],"etag":["e34d66e8-ax5299be7"],"location":["https://objectstorage.us-phoenix-1.oraclecloud.com/n/tenancy_name/b/bucket-name-test"],"opc-request-id":["phx-1:ZZZzzz"],"x-api-id":["native"]},"message":null,"payload":{"id":"/n/yourTenancyName/b/","resourceName":"/n/test/b/"},"responseTime":"2020-12-16T11:44:40.003Z","status":"200"},"stateChange":{"current":{"bucketName":"bucket-_test","eTag":"ZZZzzz"},"previous":{}}},"dataschema":"2.0","id":"1158fbdexxxx0","oracle":{"compartmentid":"ocid1.compartment.oc1..aaaaaaaxxga","ingestedtime":"2020-12-16T11:44:48.648Z","loggroupid":"_Audit","tenantid":"ocid1.tenancy.oc1..ZZZzzz"},"source":"bucket-test","specversion":"1.0","time":"2020-12-16T11:44:40.003Z","type":"com.oraclecloud.objectstorage.createbucket"},"datetime":1608119080003,"oracle.tenantid":"ocid1.tenancy.oc1..ZZZzzz","oracle.compartmentid":"ocid1.compartment.oc1..ZZZzzz","oracle.loggroupid":"_Audit","specversion":"1.0","type":"com.oraclecloud.objectstorage.createbucket","source":"bucket-test","subject":null,"id":"1158x","time":"2020-12-16T11:44:40.003Z"}}],"summary":{"resultCount":1}}