Add dataplane performance test

tests/dataplane-performance/manifests/cafe-routes.yaml

@@ -0,0 +1,60 @@
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: HTTPRoute
+metadata:
+  name: coffee
+spec:
+  parentRefs:
+  - name: cafe
+  hostnames:
+  - cafe.example.com
+  rules:
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /coffee
+    backendRefs:
+    - name: coffee-svc
+      port: 80
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /coffee
+      headers:
+      - name: version
+        value: v2
+    - path:
+        type: PathPrefix
+        value: /coffee
+      queryParams:
+      - name: TEST
+        value: v2
+    backendRefs:
+    - name: coffee-svc
+      port: 80
+---
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: HTTPRoute
+metadata:
+  name: tea
+spec:
+  parentRefs:
+  - name: cafe
+  hostnames:
+  - cafe.example.com
+  rules:
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /tea
+      method: POST
+    backendRefs:
+    - name: coffee-svc
+      port: 80
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /tea
+      method: GET
+    backendRefs:
+    - name: coffee-svc
+      port: 80

tests/dataplane-performance/manifests/coffee.yaml

@@ -0,0 +1,32 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: coffee
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: coffee
+  template:
+    metadata:
+      labels:
+        app: coffee
+    spec:
+      containers:
+      - name: coffee
+        image: nginxdemos/nginx-hello:plain-text
+        ports:
+        - containerPort: 8080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: coffee-svc
+spec:
+  ports:
+  - port: 80
+    targetPort: 8080
+    protocol: TCP
+    name: http
+  selector:
+    app: coffee

tests/dataplane-performance/manifests/gateway.yaml

@@ -0,0 +1,10 @@
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: Gateway
+metadata:
+  name: cafe
+spec:
+  gatewayClassName: nginx
+  listeners:
+  - name: http
+    port: 80
+    protocol: HTTP

tests/dataplane-performance/results/1.0.0.md

@@ -0,0 +1,105 @@
+# Results
+
+## Test environment
+
+GKE cluster:
+
+- Node count: 3
+- Instance Type: e2-medium
+- k8s version: 1.27.4-gke.900
+- Zone: europe-west2-b
+- Total vCPUs: 6
+- Total RAM: 12GB
+- Max pods per node: 110
+
+NGF deployment:
+
+- NGF version: edge - git commit a41e9e46d72788bceea8ba44e01d4062afec75fd
+- NGINX Version: 1.25.2
+
+## Test1: Running coffee path based routing
+
+Running 30s test @ http://cafe.example.com/coffee
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    32.38ms   10.50ms 155.88ms   95.86%
+    Req/Sec   157.97     23.28   210.00     84.73%
+  Latency Distribution
+     50%   31.47ms
+     75%   34.70ms
+     90%   36.45ms
+     99%   79.92ms
+  9441 requests in 30.07s, 3.36MB read
+Requests/sec:    314.00
+Transfer/sec:    114.38KB
+
+## Test2: Running coffee header based routing
+
+Running 30s test @ http://cafe.example.com/coffee
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    31.44ms    4.30ms  68.62ms   72.09%
+    Req/Sec   159.35     17.03   202.00     73.91%
+  Latency Distribution
+     50%   31.61ms
+     75%   34.61ms
+     90%   36.24ms
+     99%   41.03ms
+  9533 requests in 30.04s, 3.39MB read
+Requests/sec:    317.38
+Transfer/sec:    115.61KB
+
+## Test3: Running coffee query based routing
+
+Running 30s test @ http://cafe.example.com/coffee?TEST=v2
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    31.93ms    4.14ms  57.80ms   67.35%
+    Req/Sec   156.82     16.03   202.00     76.76%
+  Latency Distribution
+     50%   32.46ms
+     75%   35.12ms
+     90%   36.78ms
+     99%   39.99ms
+  9382 requests in 30.04s, 3.41MB read
+Requests/sec:    312.34
+Transfer/sec:    116.21KB
+
+## Test4: Running tea GET method based routing
+
+Running 30s test @ http://cafe.example.com/tea
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    31.69ms    4.14ms  52.77ms   70.05%
+    Req/Sec   158.02     14.69   202.00     82.11%
+  Latency Distribution
+     50%   31.96ms
+     75%   34.80ms
+     90%   36.42ms
+     99%   41.32ms
+  9457 requests in 30.03s, 3.34MB read
+Requests/sec:    314.87
+Transfer/sec:    113.77KB
+
+## Test5: Running tea POST method based routing
+
+Running 30s test @ http://cafe.example.com/tea
+  2 threads and 10 connections
+  Thread Stats   Avg      Stdev     Max   +/- Stdev
+    Latency    31.75ms    4.79ms 186.53ms   78.68%
+    Req/Sec   157.90     15.58   212.00     81.77%
+  Latency Distribution
+     50%   31.86ms
+     75%   34.86ms
+     90%   36.53ms
+     99%   41.55ms
+  9451 requests in 30.05s, 3.33MB read
+Requests/sec:    314.54
+Transfer/sec:    113.65KB
+
+## Observations
+
+- Average latency per request appears stable across all routing methods.
+- Average requests per second appears stable across all routing methods.
+- Interesting to note that the basic path based routing method has the lowest performance in terms of latency and
+  requests per second.

tests/dataplane-performance/scripts/post.lua

@@ -0,0 +1 @@
+wrk.method = "POST"

tests/dataplane-performance/scripts/wrk-latency.sh

@@ -0,0 +1,16 @@
+#!/bin/bash
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+echo -e "# Results\n"
+
+echo -e "## Test1: Running coffee path based routing\n"
+wrk -t2 -c10 -d30 http://cafe.example.com/coffee --latency
+echo -e "\n## Test2: Running coffee header based routing\n"
+wrk -t2 -c10 -d30 http://cafe.example.com/coffee -H "version:v2" --latency
+echo -e "\n## Test3: Running coffee query based routing\n"
+wrk -t2 -c10 -d30 http://cafe.example.com/coffee?TEST=v2 --latency
+echo -e "\n## Test4: Running tea GET method based routing\n"
+wrk -t2 -c10 -d30 http://cafe.example.com/tea --latency
+echo -e "\n## Test5: Running tea POST method based routing\n"
+wrk -t2 -c10 -d30 http://cafe.example.com/tea -s ${SCRIPT_DIR}/post.lua --latency

tests/dataplane-performance/setup.md

@@ -0,0 +1,74 @@
+# Dataplane Performance Testing
+
+## Goals
+
+- To capture the average latency of requests being proxied through NGINX using a variety of routing rules, so that we
+  can see if different routing rules provide different results, and so that we can know if any future work has an impact
+  on data plane performance.
+- A route is created and tested for each routing method below:
+  - path based routing
+  - header based routing
+  - query param based routing
+  - method based routing
+
+## Test Environment
+
+ The following cluster will be sufficient:
+
+- A Kubernetes cluster with 3 nodes on GKE
+  - Node: e2-medium (2 vCPU, 4GB memory)
+
+## Setup
+
+1. Create cloud cluster
+2. Deploy CRDs:
+
+   ```bash
+   kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.8.1/standard-install.yaml
+   ```
+
+3. Deploy NGF from edge using Helm install:
+
+   ```console
+   helm install my-release oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric  --version 0.0.0-edge \
+      --create-namespace --wait -n nginx-gateway
+   ```
+
+## Tests
+
+1. First create the test resources. The manifests provided will deploy a single application and service, and two
+   HTTPRoutes with matches for path based, header based, query based, and method based routing.
+
+   ```console
+   kubectl apply -f manifests/gateway.yaml
+   kubectl apply -f manifests/coffee.yaml
+   kubectl apply -f manifests/cafe-routes.yaml
+   ```
+
+2. Get the external IP of the nginx-gateway service and add an entry in /etc/hosts for `<GW_IP> cafe.example.com`.
+
+   ```console
+   kubectl get service my-release-nginx-gateway-fabric \
+   --namespace nginx-gateway \
+   --output jsonpath='{.status.loadBalancer.ingress[0].ip}'
+   ```
+
+3. Run tests using `wrk` (change the file output name to version under test). The tests use `wrk` to send requests to
+   coffee application using each of the configured routing rules, capturing the number of requests and the average
+   latency.
+
+   ```console
+   bash scripts/wrk-latency.sh > results/1.0.0.md
+   ```
+
+4. Analyse the results and check for anomolies. Append any findings or observations to the generated results document.
+   Add test environment information to the generated document.
+
+5. Cleanup the deployed resources
+
+   ```console
+   kubectl delete -f manifests/cafe-routes.yaml
+   kubectl delete -f manifests/coffee.yaml
+   kubectl delete -f manifests/gateway.yaml
+   helm uninstall my-release -n nginx-gateway
+   ```