router now supports flexible s3 event handling (spring-media#18)

* router now supports flexible s3 event handling

Author: chgohlke

README.md

@@ -5,7 +5,9 @@
 
 # aws-lambda-router
 
-A small library for [AWS Lambda](https://aws.amazon.com/lambda/details) providing routing for [API Gateway](https://aws.amazon.com/api-gateway) [Proxy Integrations](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html) and [SNS](https://aws.amazon.com/sns).
+A small library for [AWS Lambda](https://aws.amazon.com/lambda/details) providing routing for [API Gateway](https://aws.amazon.com/api-gateway),
+[Proxy Integrations](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html), [SNS](https://aws.amazon.com/sns) 
+and [S3 Events](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html). 
 
 ## Features
 
@@ -14,7 +16,7 @@ A small library for [AWS Lambda](https://aws.amazon.com/lambda/details) providin
 * Lambda Proxy Resource support for AWS API Gateway
 * Enable CORS for requests
 * No external dependencies
-* Currently there are two `processors` (callers for Lambda) implemented: API Gateway ANY method (called proxyIntegration), SNS and SQS. 
+* Currently there are four `processors` (callers for Lambda) implemented: API Gateway ANY method (called proxyIntegration), SNS, SQS and S3. 
 
 ## Installation
 Install via npm
@@ -122,6 +124,8 @@ and the http response then contains the configured value as response code and th
 
 ## SNS to Lambda Integrations
 
+SNS Event Structure: https://docs.aws.amazon.com/sns/latest/dg/sns-message-and-json-formats.html
+
 For handling calls in Lambdas initiated from AWS-SNS you can use the following code snippet:
 
 ```js
@@ -173,6 +177,80 @@ The `action` method gets all body elements from the router as an array.
 
 If more than one route matches, only the **first** is used!
 
+## S3 to Lambda Integrations
+
+
+Lambdas can be triggered by S3 events. The router now supports these events.
+With the router it is very easy and flexible to connect a lambda to different s3 sources (different buckets). The following possibilities are available:
+
+- bucketName: By specifying a fixed _bucketName_ all s3 events with this bucket name are forwarded to a certain action. Instead of a fixed bucket a _RegExp_ is also possible.
+- eventName: By configuring the [S3 event name](https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#supported-notification-event-types) the routing can be further restricted. A _RegExp_ is also possible here.
+- objectKeyPrefix: fixed string as an prefix of an object key (but not an RegExp). Is useful if you want to organize your bucket in subfolder. 
+
+A combination of bucketName, eventName and objectKeyPrefix is possible. If no _bucketName_, _eventName_ and _objectKeyPrefix_ is configured, all s3 events are forwarded to the action.
+
+The action method will be called with the [S3Event Structure](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html)
+
+The following examples demonstrates the most use cases:
+
+```js
+const router = require('aws-lambda-router');
+
+exports.handler = router.handler({
+    s3: {
+        routes: [
+            {
+                //match every s3 record to this action 
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            {
+                //match s3 events which created, bucket name is whitelisted here
+                eventName: 'ObjectCreated:Put',
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            {
+                //event name is an regex: match 'ObjectCreated:Put' or 'ObjectCreated:Copy'
+                eventName: /ObjectCreated:*/,
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            {
+                //exact name of bucket 'myBucket', event name is whitelisted and will not be checked
+                bucketName: 'myBucket',
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            {
+                //regex of bucket name (all buckets started with 'bucket-dev-' will be machted
+                bucketName: /^bucket-dev-.*/,
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            { 
+                //action only will be called if bucket and event matched to the given regex
+                bucketName: /bucket-dev-.*/,
+                eventName: /ObjectCreated:*/,
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            { 
+                //action only will be called if bucket and event matched to the given fixed string
+                bucketName: 'bucket',
+                eventName: 'ObjectRemoved:Delete',
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            },
+            { 
+                //match if s3 events comes from Bucket 'bucket' with event 'ObjectRemoved:Delete' 
+                // and the object key starts with /upload
+                objectKeyPrefix: '/upload',
+                bucketName: 'bucket',
+                eventName: 'ObjectRemoved:Delete',
+                action: (event, context) => console.log(event.s3.object.key, event.eventName)
+            }
+        ],
+        debug: true
+    }
+});
+```
+
+Per s3 event there can be several entries per event. Than the action methods are called one after the other. The result is an array with objects insides.
+
 ### Custom response
 
 Per default a status code 200 will be returned. This behavior can be overridden.

gulpfile.js

@@ -1,6 +1,4 @@
 const gulp = require('gulp');
-const del = require('del');
-const install = require('gulp-install');
 const jasmine = require('gulp-jasmine');
 
 gulp.task('test', () =>

index.d.ts

@@ -37,8 +37,21 @@ export interface SqsConfig {
     debug?: boolean;
 }
 
+export interface S3Route {
+    bucketName?: string | RegExp;
+    eventName?: string | RegExp;
+    objectKeyPrefix?: string;
+    action: (s3Event: any[], context: any) => any;
+}
+
+export interface S3Config {
+    routes: S3Route[];
+    debug?: boolean;
+}
+
 export interface RouteConfig {
     proxyIntegration?: ProxyIntegrationConfig;
     sns?: SnsConfig;
     sqs?: SqsConfig;
+    s3?: S3Config;
 }

lib/s3.js

@@ -0,0 +1,89 @@
+"use strict";
+
+function validateArguments(s3Config, event) {
+
+    if (!Array.isArray(event.Records) || event.Records.length < 1 || event.Records[0].eventSource !== 'aws:s3') {
+        console.log('Event does not look like S3');
+        return false;
+    }
+
+    //validate config
+    if (!Array.isArray(s3Config.routes) || s3Config.routes.length < 1) {
+        throw new Error('s3Config.routes must not be empty');
+    }
+
+    //validate config
+    for (const route of s3Config.routes) {
+        if (!route.action) {
+            throw new Error('s3Config.routes.action must not be empty');
+        }
+    }
+
+    return true;
+
+}
+
+function matchConfigToEventValue(config, eventValue) {
+
+    if (!config) {
+        return true;
+    }
+
+    if (config instanceof RegExp) {
+        if (config.test(eventValue)) {
+            return true;
+        }
+    } else {
+        if (config === eventValue) {
+            return true;
+        }
+    }
+
+    return false;
+}
+
+function matchObjectKeyWithPrefix(config, eventValue) {
+    if (!config || eventValue.startsWith(config)) {
+        return true;
+    }
+
+    return false;
+}
+
+function process(s3Config, event, context) {
+
+    if (s3Config.debug) {
+        console.log('s3:Event', JSON.stringify(event));
+        console.log('s3:context', context);
+    }
+
+    if (!validateArguments(s3Config, event)) {
+        return null;
+    }
+
+    const results = [];
+    for (const record of event.Records) {
+        for (const routeConfig of s3Config.routes) {
+            const bucketNameMatched = matchConfigToEventValue(routeConfig.bucketName, record.s3.bucket.name);
+            const eventNameMatched = matchConfigToEventValue(routeConfig.eventName, record.eventName);
+            const objectKeyPrefixMatched = matchObjectKeyWithPrefix(routeConfig.objectKeyPrefix, record.s3.object.key);
+
+            if (s3Config.debug) {
+                console.log(`match record '${record.eventName}'/'${record.s3.bucket.name}'/'${record.s3.object.key}': bucketMatch '
+                    ${bucketNameMatched}', eventMatch '${eventNameMatched}', key '${objectKeyPrefixMatched}' for route '${JSON.stringify(routeConfig)}'`);
+            }
+
+            if (bucketNameMatched && eventNameMatched && objectKeyPrefixMatched) {
+                const result = routeConfig.action(record, context);
+                if (result) {
+                    results.push(result);
+                }
+                break;
+            }
+        }
+    }
+
+    return results;
+}
+
+module.exports = process;

package.json

@@ -27,11 +27,9 @@
   },
   "homepage": "https://github.com/spring-media/aws-lambda-router#readme",
   "devDependencies": {
-    "del": "2.2.2",
     "gulp": "4.0",
-    "gulp-install": "1.1.0",
     "gulp-jasmine": "4.0.0",
-    "gulp-zip": "4.2.0",
+    "jasmine": "3.3.1",
     "proxyquire": "2.1.0"
   }
 }