Skip to content

Commit 7ecc4c7

Browse files
alexellisalexellis
committed
Initial version of SQS queue
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <[email protected]>
1 parent b0f6dd0 commit 7ecc4c7

File tree

8 files changed

+328
-0
lines changed

8 files changed

+328
-0
lines changed

‎chart/sqs-connector/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/values-ae.yaml

‎chart/sqs-connector/.helmignore

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*~
18+
# Various IDEs
19+
.project
20+
.idea/
21+
*.tmproj
22+
.vscode/

‎chart/sqs-connector/Chart.yaml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
apiVersion: v1
2+
description: Connect OpenFaaS functions to SQS topics
3+
name: sqs-connector
4+
version: 0.0.1
5+
sources:
6+
- https://github.com/openfaas/faas-netes
7+
home: https://www.openfaas.com
8+
icon: https://raw.githubusercontent.com/openfaas/media/master/OpenFaaS_logo_stacked_opaque.png
9+
keywords:
10+
- openfaaspro
11+
- serverless
12+
- sqs
13+
- events
14+
maintainers:
15+
- name: alexellis
16+
email: [email protected]

‎chart/sqs-connector/README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# OpenFaaS PRO SQS Connector
2+
3+
The SQS connector can be used to invoke functions from an AWS SQS queue.
4+
5+
## Prerequisites
6+
7+
- Obtain a license or trial
8+
9+
You will need an OpenFaaS Premium subscription to access PRO features.
10+
11+
Contact us to find out more and to start a free trial at: [openfaas.com/support](https://www.openfaas.com/support)
12+
13+
- Install OpenFaaS
14+
15+
You must have a working OpenFaaS installed.
16+
17+
## Install the Chart
18+
19+
- Create the required secret with your OpenFaaS PRO license code:
20+
21+
```bash
22+
$ kubectl create secret generic \
23+
-n openfaas \
24+
openfaas-license \
25+
--from-file license=$HOME/.openfaas/LICENSE
26+
```
27+
28+
- Add the OpenFaaS chart repo and deploy the `sqs-connector` PRO chart. We recommend installing it in the same namespace as the rest of OpenFaaS
29+
30+
```sh
31+
$ helm repo add openfaas https://openfaas.github.io/faas-netes/
32+
$ helm upgrade sqs-connector openfaas/sqs-connector \
33+
--install \
34+
--namespace openfaas
35+
```
36+
37+
> The above command will also update your helm repo to pull in any new releases.
38+
39+
## Install a development version
40+
```sh
41+
$ helm upgrade sqs-connector ./chart/sqs-connector \
42+
--install \
43+
--namespace openfaas
44+
-f ./values.yaml
45+
```
46+
47+
## Configuration
48+
49+
Additional sqs-connector options in `values.yaml`.
50+
51+
| Parameter | Description | Default |
52+
| ------------------------ | -------------------------------------------------------------------------------------- | ------------------------------ |
53+
| `asyncInvocation` | For long running or slow functions, offload to asychronous function invocations and carry on processing the stream | `false` |
54+
| `upstreamTimeout` | Maximum timeout for upstream function call, must be a Go formatted duration string. | `2m` |
55+
| `rebuildInterval` | Interval for rebuilding function to topic map, must be a Go formatted duration string. | `30s` |
56+
| `gatewayURL` | The URL for the API gateway. | `http://gateway.openfaas:8080` |
57+
| `printResponse` | Output the response of calling a function in the logs. | `true` |
58+
| `printResponseBody` | Output to the logs the response body when calling a function. | `false` |
59+
| `printRequestBody` | Output to the logs the request body when calling a function. | `false` |
60+
| `fullnameOverride` | Override the name value used for the Connector Deployment object. | `` |
61+
| `contentType` | Set a HTTP Content Type during function invocation. | `""` |
62+
63+
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. See `values.yaml` for the default configuration.
64+
65+
## Removing the sqs-connector
66+
67+
All control plane components can be cleaned up with helm:
68+
69+
```sh
70+
$ helm delete --purge sqs-connector
71+
```

‎chart/sqs-connector/templates/NOTES.txt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Thanks for installing kafka-connector. Please follow the instructions below to get you started.
2+
3+
You can watch the Connector logs to see it invoke your functions:
4+
5+
$ kubectl logs -n {{ .Release.Namespace }} deploy/sqs-connector -f
6+

‎chart/sqs-connector/templates/_helpers.tpl

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
{{/* vim: set filetype=mustache: */}}
2+
{{/*
3+
Expand the name of the chart.
4+
*/}}
5+
{{- define "connector.name" -}}
6+
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
7+
{{- end -}}
8+
9+
{{/*
10+
Create a default fully qualified app name.
11+
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
12+
If release name contains chart name it will be used as a full name.
13+
*/}}
14+
{{- define "connector.fullname" -}}
15+
{{- if .Values.fullnameOverride -}}
16+
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
17+
{{- else -}}
18+
{{- $name := default .Chart.Name .Values.nameOverride -}}
19+
{{- if contains $name .Release.Name -}}
20+
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
21+
{{- else -}}
22+
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
23+
{{- end -}}
24+
{{- end -}}
25+
{{- end -}}
26+
27+
{{/*
28+
Create chart name and version as used by the chart label.
29+
*/}}
30+
{{- define "connector.chart" -}}
31+
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
32+
{{- end -}}
33+

‎chart/sqs-connector/templates/deployment.yml

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
labels:
5+
app: {{ template "connector.name" . }}
6+
component: sqs-connector
7+
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
8+
heritage: {{ .Release.Service }}
9+
release: {{ .Release.Name }}
10+
app.kubernetes.io/name: {{ template "connector.name" . }}
11+
app.kubernetes.io/instance: {{ .Release.Name }}
12+
app.kubernetes.io/version: {{ .Chart.Version }}
13+
app.kubernetes.io/component: sqs-connector
14+
app.kubernetes.io/part-of: openfaas
15+
app.kubernetes.io/managed-by: {{ .Release.Service }}
16+
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
17+
name: {{ template "connector.fullname" . }}
18+
namespace: {{ .Release.Namespace | quote }}
19+
spec:
20+
replicas: {{ .Values.replicas }}
21+
selector:
22+
matchLabels:
23+
app: {{ template "connector.name" . }}
24+
component: sqs-connector
25+
26+
template:
27+
metadata:
28+
annotations:
29+
prometheus.io/scrape: "false"
30+
labels:
31+
app: {{ template "connector.name" . }}
32+
component: sqs-connector
33+
spec:
34+
volumes:
35+
- name: openfaas-license
36+
secret:
37+
secretName: openfaas-license
38+
{{- if .Values.basic_auth }}
39+
- name: auth
40+
secret:
41+
secretName: basic-auth
42+
{{- end }}
43+
{{- if .Values.awsCredentialsSecret }}
44+
- name: aws-credentials
45+
secret:
46+
secretName: {{ .Values.awsCredentialsSecret }}
47+
{{- end}}
48+
containers:
49+
- name: connector
50+
image: {{ .Values.image }}
51+
command:
52+
- "/usr/bin/sqs-connector"
53+
- "-license-file=/var/secrets/license/license"
54+
- "--queue-url={{ .Values.queueURL }}"
55+
- "--visibility-timeout={{.Values.visibilityTimeout }}"
56+
- "--wait-time={{.Values.waitTime}}"
57+
- "--max-messages={{.Values.maxMessages}}"
58+
env:
59+
- name: gateway_url
60+
value: {{ .Values.gatewayURL | quote }}
61+
- name: topics
62+
value: {{ .Values.topics | quote }}
63+
- name: print_response
64+
value: {{ .Values.printResponse | quote }}
65+
- name: print_response_body
66+
value: {{ .Values.printResponseBody | quote }}
67+
- name: print_request_body
68+
value: {{ .Values.printRequestBody | quote }}
69+
- name: asynchronous_invocation
70+
value: {{ .Values.asyncInvocation | quote }}
71+
{{- if .Values.basic_auth }}
72+
- name: basic_auth
73+
value: "true"
74+
- name: secret_mount_path
75+
value: "/var/secrets"
76+
{{- end }}
77+
{{- if .Values.upstreamTimeout }}
78+
- name: upstream_timeout
79+
value: {{ .Values.upstreamTimeout | quote }}
80+
{{- end }}
81+
{{- if .Values.rebuildInterval }}
82+
- name: rebuild_interval
83+
value: {{ .Values.rebuildInterval | quote }}
84+
{{- end }}
85+
- name: content_type
86+
value: {{ .Values.contentType | quote }}
87+
{{- if .Values.awsCredentialsSecret }}
88+
- name: AWS_SHARED_CREDENTIALS_FILE
89+
value: /var/secrets/aws-credentials/{{ .Values.awsCredentialsSecret }}
90+
{{- end }}
91+
{{- if .Values.awsDefaultRegion }}
92+
- name: AWS_DEFAULT_REGION
93+
value: {{ .Values.awsDefaultRegion }}
94+
{{- end }}
95+
volumeMounts:
96+
- name: openfaas-license
97+
readOnly: true
98+
mountPath: "/var/secrets/license"
99+
{{- if .Values.basic_auth }}
100+
- name: auth
101+
readOnly: true
102+
mountPath: "/var/secrets"
103+
{{- end }}
104+
{{- if .Values.awsCredentialsSecret }}
105+
- name: aws-credentials
106+
readOnly: true
107+
mountPath: "/var/secrets/aws-credentials"
108+
{{- end }}
109+
110+
{{- with .Values.nodeSelector }}
111+
nodeSelector:
112+
{{ toYaml . | indent 8 }}
113+
{{- end }}
114+
{{- with .Values.affinity }}
115+
affinity:
116+
{{ toYaml . | indent 8 }}
117+
{{- end }}
118+
{{- with .Values.tolerations }}
119+
tolerations:
120+
{{ toYaml . | indent 8 }}
121+
{{- end }}

‎chart/sqs-connector/values.yaml

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# You will need to create a license named "openfaas-license" - see the
2+
# chart README for detailed instructions.
3+
4+
image: alexellis/sqs-connector:0.0.1-4-g30ef18d-amd64
5+
6+
replicas: 1
7+
8+
queueURL: "https://"
9+
10+
# kubectl create secret generic -n openfaas \
11+
# aws-sqs-credentials --from-file aws-sqs-credentials=$HOME/sqs-credentials.txt
12+
awsCredentialsSecret: aws-sqs-credentials
13+
14+
awsDefaultRegion: eu-west-1
15+
16+
# Maximum time to keep message hidden from other processors whilst
17+
# executing function
18+
visibilityTimeout: 30s
19+
20+
# Time to wait between polling SQS queue for messages.
21+
waitTime: 20s
22+
23+
# Maximum messages to fetch at once - between 1-10
24+
maxMessages: 1
25+
26+
# Max timeout for a function
27+
upstreamTimeout: 30s
28+
29+
# interval for rebuilding the map of functions and topics
30+
rebuildInterval: 30s
31+
32+
# Use with slow consumers or long running functions
33+
asyncInvocation: false
34+
35+
# HTTP content-type for invoking functions
36+
contentType: text/plain
37+
38+
# Print the data read from the Kafka topic before invoking functions
39+
printRequestBody: false
40+
41+
# Print the HTTP status of invoked functions
42+
printResponse: false
43+
44+
# Print the data received from invoked functions
45+
printResponseBody: false
46+
47+
# Gateway URL to access API and for invocations
48+
gatewayURL: http://gateway.openfaas:8080
49+
50+
# Basic auth for the gateway
51+
basic_auth: true
52+
53+
nodeSelector: {}
54+
55+
tolerations: []
56+
57+
affinity: {}
58+

0 commit comments

Comments
 (0)