Add support for backup directive for VS and TS (#4653)

Co-authored-by: Alan Dooley <[email protected]>
Co-authored-by: Venktesh Shivam Patel <[email protected]>
Co-authored-by: shaun-nx <[email protected]>

Author: 4 people

config/crd/bases/k8s.nginx.org_transportservers.yaml

@@ -103,6 +103,10 @@ spec:
                 items:
                   description: TransportServerUpstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     failTimeout:
                       type: string
                     healthCheck:
@@ -251,6 +255,10 @@ spec:
                 items:
                   description: TransportServerUpstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     failTimeout:
                       type: string
                     healthCheck:

config/crd/bases/k8s.nginx.org_virtualserverroutes.yaml

@@ -498,6 +498,10 @@ spec:
                 items:
                   description: Upstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     buffer-size:
                       type: string
                     buffering:

config/crd/bases/k8s.nginx.org_virtualservers.yaml

@@ -599,6 +599,10 @@ spec:
                 items:
                   description: Upstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     buffer-size:
                       type: string
                     buffering:

deploy/crds.yaml

@@ -612,6 +612,10 @@ spec:
                 items:
                   description: TransportServerUpstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     failTimeout:
                       type: string
                     healthCheck:
@@ -760,6 +764,10 @@ spec:
                 items:
                   description: TransportServerUpstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     failTimeout:
                       type: string
                     healthCheck:
@@ -1321,6 +1329,10 @@ spec:
                 items:
                   description: Upstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     buffer-size:
                       type: string
                     buffering:
@@ -2102,6 +2114,10 @@ spec:
                 items:
                   description: Upstream defines an upstream.
                   properties:
+                    backup:
+                      type: string
+                    backupPort:
+                      type: integer
                     buffer-size:
                       type: string
                     buffering:

docs/content/configuration/transportserver-resource.md

@@ -158,6 +158,8 @@ loadBalancingMethod: least_conn
 |``failTimeout`` | Sets the [time](https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout) during which the specified number of unsuccessful attempts to communicate with the server should happen to consider the server unavailable and the period of time the server will be considered unavailable. The default is ``10s``. | ``string`` | No |
 |``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check) directive. Note: this feature is supported only in NGINX Plus. | [healthcheck](#upstreamhealthcheck) | No |
 |``loadBalancingMethod`` | The method used to load balance the upstream servers. By default, connections are distributed between the servers using a weighted round-robin balancing method. See the [upstream](http://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#upstream) section for available methods and their details. | ``string`` | No |
+|``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No |
+|``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No |
 {{% /table %}}
 
 ### Upstream.Healthcheck

docs/content/configuration/virtualserver-and-virtualserverroute-resources.md

@@ -372,6 +372,8 @@ tls:
 |``buffer-size`` | Sets the size of the buffer used for reading the first part of a response received from the upstream server. See the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directive. The default is set in the ``proxy-buffer-size`` ConfigMap key. | ``string`` | No |
 |``ntlm`` | Allows proxying requests with NTLM Authentication. See the [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm) directive. In order for NTLM authentication to work, it is necessary to enable keepalive connections to upstream servers using the ``keepalive`` field. Note: this feature is supported only in NGINX Plus.| ``boolean`` | No |
 |``type`` |The type of the upstream. Supported values are ``http`` and ``grpc``. The default is ``http``. For gRPC, it is necessary to enable HTTP/2 in the [ConfigMap](/nginx-ingress-controller/configuration/global-configuration/configmap-resource/#listeners) and configure TLS termination in the VirtualServer. | ``string`` | No |
+|``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No |
+|``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No |
 {{% /table %}}
 
 ### Upstream.Buffers

examples/custom-resources/backup-directive/transport-server/README.md

@@ -0,0 +1,108 @@
+# Support for Backup Directive in Transport Server
+
+F5 NGINX Ingress Controller supports routing requests to a service called `backup`.
+`backup` is an [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) service.
+
+> [!NOTE]
+> The [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) service is only
+available with NGINX Plus.
+
+For this example, we will use two [tls-passthrough](/examples/custom-resources/tls-passthrough) configurations.
+One will be deployed in the `default` namespace, and the other in the `external-ns` namespace.
+
+The application in the `external-ns` namespace will respond to our requests when main application is unavailable.
+
+## Prerequisites
+
+1. Configure the F5 NGINX Ingress Controller deployment with the following flags:
+
+   ```shell
+   -enable-custom-resources
+   -enable-tls-passthrough
+   -watch-namespace=nginx-ingress,default
+   ```
+
+   We configure the `-watch-namespace` flag to only watch the `nginx-ingress` and `default` namespaces.
+   This ensures that NGINX Ingress Controller will treat our service in the `external-ns` namespace
+   as an external service.
+
+2. Follow the [installation](https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/)
+   instructions to deploy NGINX Ingress Controller.
+
+3. Save the public IP address of the F5 NGINX Ingress Controller into a shell variable:
+
+    ```shell
+    IC_IP=XXX.YYY.ZZZ.III
+    ```
+
+4. Save the HTTPS port of NGINX Ingress Controller into a shell variable:
+
+    ```shell
+    IC_HTTPS_PORT=<port number>
+    ```
+
+## Deployment
+
+### 1. Deploy ConfigMap with defined resolver
+
+   ```shell
+   kubectl create -f nginx-config.yaml
+   ```
+
+### 2. Deploy Backup ExternalName service
+
+   ```shell
+   kubectl create -f backup-svc.yaml
+   ```
+
+### 3. Deploy the tls-passthrough application
+
+   ```shell
+   kubectl create -f secure-app.yaml
+   ```
+
+### 4. Deploy TransportServer
+
+   ```shell
+   kubectl create -f transport-server-passthrough.yaml
+   ```
+
+### 5. Test the Configuration
+
+   Run the below curl command to get a response from your application:
+
+   ```shell
+   curl --resolve app.example.com:$IC_HTTPS_PORT:$IC_IP https://app.example.com:$IC_HTTPS_PORT --insecure
+   ```
+
+   ```shell
+   hello from pod secure-app-694bc784b-qh8ng
+   ```
+
+### 6. Deploy the second tls-passthrough application to the external namespace
+
+   ```shell
+   kubectl apply -f external-secure-app.yaml
+   ```
+
+### 7. Test the configuration using the backup service
+
+1. Scale down `secure-app` deployment to 0.
+   This is done to ensure that the external `backup` service will respond to our requests.
+
+    ```shell
+    kubectl scale deployment secure-app --replicas=0
+    ```
+
+2. Verify if the application is working by sending a request and check if the response is coming from the "external
+   backend pod"
+
+    ```shell
+    curl --resolve app.example.com:$IC_HTTPS_PORT:$IC_IP https://app.example.com:$IC_HTTPS_PORT --insecure
+    ```
+
+3. Check response from the backup service
+
+    ```shell
+    HELLO FROM EXTERNAL APP pod secure-app-backup-7d98dd8d78-p8q7d
+    ```

examples/custom-resources/backup-directive/transport-server/backup-svc.yaml

@@ -0,0 +1,7 @@
+kind: Service
+apiVersion: v1
+metadata:
+  name: backup-svc-ts
+spec:
+  type: ExternalName
+  externalName: secure-app-backup.external-ns.svc.cluster.local

examples/custom-resources/backup-directive/transport-server/external-secure-app.yaml

@@ -0,0 +1,85 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: external-ns
+  namespace: external-ns
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: secure-app-backup
+  namespace: external-ns
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: secure-app-backup
+  template:
+    metadata:
+      labels:
+        app: secure-app-backup
+    spec:
+      containers:
+        - name: secure-app-backup
+          image: nginxdemos/nginx-hello:plain-text
+          ports:
+            - containerPort: 8443
+          volumeMounts:
+            - name: secret
+              mountPath: /etc/nginx/ssl
+              readOnly: true
+            - name: config-volume
+              mountPath: /etc/nginx/conf.d
+      volumes:
+        - name: secret
+          secret:
+            secretName: app-tls-secret
+        - name: config-volume
+          configMap:
+            name: secure-config
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: secure-app-backup
+  namespace: external-ns
+spec:
+  ports:
+    - port: 8443
+      targetPort: 8443
+      protocol: TCP
+      name: https
+  selector:
+    app: secure-app-backup
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: secure-config
+  namespace: external-ns
+data:
+  app.conf: |-
+    server {
+      listen 8443 ssl;
+      listen [::]:8443 ssl;
+
+      server_name app.example.com;
+
+      ssl_certificate /etc/nginx/ssl/tls.crt;
+      ssl_certificate_key /etc/nginx/ssl/tls.key;
+
+      default_type text/plain;
+
+      location / {
+        return 200 "HELLO FROM EXTERNAL APP pod $hostname\n";
+      }
+    }
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: app-tls-secret
+  namespace: external-ns
+data:
+  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURGRENDQWZ3Q0NRQ3EzQWxhdnJiaWpqQU5CZ2txaGtpRzl3MEJBUXNGQURCTU1Rc3dDUVlEVlFRR0V3SlYKVXpFTE1Ba0dBMVVFQ0F3Q1EwRXhGakFVQmdOVkJBY01EVk5oYmlCR2NtRnVZMmx6WTI4eEdEQVdCZ05WQkFNTQpEMkZ3Y0M1bGVHRnRjR3hsTG1OdmJUQWVGdzB5TURBek1qTXlNekl3TkROYUZ3MHlNekF6TWpNeU16SXdORE5hCk1Fd3hDekFKQmdOVkJBWVRBbFZUTVFzd0NRWURWUVFJREFKRFFURVdNQlFHQTFVRUJ3d05VMkZ1SUVaeVlXNWoKYVhOamJ6RVlNQllHQTFVRUF3d1BZWEJ3TG1WNFlXMXdiR1V1WTI5dE1JSUJJakFOQmdrcWhraUc5dzBCQVFFRgpBQU9DQVE4QU1JSUJDZ0tDQVFFQTJCRXhZR1JPRkhoN2VPMVlxeCtWRHMzRzMrVEhyTEZULzdEUFFEQlkza3pDCi9oZlprWCt3OW1NNkQ1RU9uK2lpVlNhUWlQMm1aNFA3N29pR0dmd3JrNjJ0eEQ5cHphODM5NC9aSjF5Q0dXZ1QKK2NWUEVZbkxjQktzSTRMcktJZ21oWVIwUjNzWWRjR1JkSXJWUFZlNUVUQlk1Z1U0RGhhMDZOUEIraitmK0krWgphWGIvMlRBekJhNHozMWpIQzg2amVQeTFMdklGazFiY3I2cSsxRGR5eklxcWxkRDYvU3Q4Q2t3cDlOaDFCUGFhCktZZ1ZVd010UVBib2s1cFFmbVMrdDg4NHdSM0dTTEU4VkxRbzgyYnJhNUR3emhIamlzOTlJRGhzbUt0U3lWOXMKaWNJbXp5dHBnSXlhTS9zWEhRQU9KbVFJblFteWgyekd1WFhTQ0lkRGtRSURBUUFCTUEwR0NTcUdTSWIzRFFFQgpDd1VBQTRJQkFRQ0tsVkhOZ1k5VHZLaW9Xb0tvdllCdnNRMmYrcmFOOEJwdWNDcnRvRm15NUczcGIzU2lPTndaCkF2cnhtSm4vR3lsa3JKTHBpQVA1eUNBNGI2Y2lYMnRGa3pQRmhJVFZKRTVBeDlpaEF2WWZwTUFSdWVqM29HN2UKd0xwQk1iUnlGbHJYV29NWUVBMGxsV0JueHRQQXZYS2Y4SVZGYTRSSDhzV1JJSDB4M2hFdjVtQ3VUZjJTRTg0QwpiNnNjS3Z3MW9CQU5VWGxXRVZVYTFmei9rWWZBa1lrdHZyV2JUcTZTWGxodXRJYWY4WEYzSUMrL2x1b3gzZThMCjBBcEFQVE5sZ0JwOTkvcXMrOG9PMWthSmQ1TmV6TnlJeXhSdUtJMzlDWkxuQm9OYmkzdlFYY1NzRCtYU2lYT0cKcEVnTjNtci8xRms4OVZMSENhTnkyKzBqMjZ0eWpiclcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
+  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRRFlFVEZnWkU0VWVIdDQKN1Zpckg1VU96Y2JmNU1lc3NWUC9zTTlBTUZqZVRNTCtGOW1SZjdEMll6b1BrUTZmNktKVkpwQ0kvYVpuZy92dQppSVlaL0N1VHJhM0VQMm5OcnpmM2o5a25YSUlaYUJQNXhVOFJpY3R3RXF3amd1c29pQ2FGaEhSSGV4aDF3WkYwCml0VTlWN2tSTUZqbUJUZ09GclRvMDhINlA1LzRqNWxwZHYvWk1ETUZyalBmV01jTHpxTjQvTFV1OGdXVFZ0eXYKcXI3VU4zTE1pcXFWMFByOUszd0tUQ24wMkhVRTlwb3BpQlZUQXkxQTl1aVRtbEIrWkw2M3p6akJIY1pJc1R4VQp0Q2p6WnV0cmtQRE9FZU9LejMwZ09HeVlxMUxKWDJ5SndpYlBLMm1Bakpveit4Y2RBQTRtWkFpZENiS0hiTWE1CmRkSUloME9SQWdNQkFBRUNnZ0VCQUxYaW16ODZrT1A0bkhBcTFPYVEyb2l3dndhQTczbTNlUytZSm84eFk4NFcKcmxyNXRzUWR5dGxPcEhTd05yQjBSQnNNTU1XeFNPQ0JJWlltUlVVZ200cGd2Uk9rRWl2OG9VOThQMkE4SnFTKwprWHBFRjVCNi84K2pXRmM0Z1Q4SWhlMEZtR0VJQllvelhYL08wejBsV0h4WXg2MHluWUoycU9vS1FKT3A5YjlsCmpiUVBkaC9mN2ErRWF0RzZNUFlrNG5xSEY3a0FzcmNsRXo2SGUvaEx6NmRkSTJ1N2RMRjB6QlN0QjM5WDFRZysKZ1JzTittOXg1S1FVTXYxMktvajdLc2hEelozOG5hSjd5bDgycGhBV1lGZzBOZHlzRlBRbmt0WmlNSUxOblFjNwpOeUt0cHNQaUxIRE9ha05hdEZLU2lOaUJrUk1lY1ZUMlJNMzMzUG54bFVFQ2dZRUEvYTY5MEEralU4VFJNbVZyCk4vRnlYWkxYa1c5b2NxVjBRbTA0TDMrSExybFNCTlRWSzk2U1pVT203VjViTzIxNmd4S2dJK3IwYm5kdE5GTUQKLzFncDhsdlJNcUlIeGZTeUo4SHpsSzViT0lnaUpxRGhzK3BKWTZmLytIVzZ1QkZyN3NGS3lxbVlIQlA0SC9BdApsT3lLeEVjMHFXazFlT2tCMWNNSGx0WDRwemtDZ1lFQTJncDhDVDVYWjNMSWRQN2M1SHpDS1YwczBYS1hGNmYyCkxzclhPVlZaTmJCN1NIS1NsOTBIU2VWVGx3czdqSnNxcC9yWFY2aHF0eUdEaTg4aTFZekthcEF6dXl3b0U3TnEKMUJpd2ZYSURQeTlPNUdGNXFYNXFUeENzSWNIcmo2Z21XMEZVQWhoS1lQcDRxd1JMdzFMZkJsd3U1VmhuN3I3ego0SkZBTEFpdlp4a0NnWUJicnpuKzVvZjdFSmtqQTdDYWlYTHlDczVLUzkrTi8rcGl6NktNMkNSOWFKRVNHZkhwClp3bTErNXRyRXIwYVgxajE0bGRxWTlKdjBrM3ZxVWs2a2h5bThUUk1mbThjeG5GVkdTMzF3SVpMaWpmOWlndkkKd0paQnBFaEkvaE83enVBWmJGYWhwR1hMVUJSUFJyalNxQ01IQ1UwcEpWTWtIZUtCNVhqcXRPNm5VUUtCZ0NJUAp6VHlzYm44TW9XQVZpSEJ4Uk91dFVKa1BxNmJZYUU3N0JSQkIwd1BlSkFRM1VjdERqaVh2RzFYWFBXQkR4VEFrCnNZdFNGZ214eEprTXJNWnJqaHVEbDNFLy9xckZOb1VYcmtxS2l4Tk4wcWMreXdDOWJPSVpHcXJUWG5jOHIzRkcKRFZlZWI5QWlrTU0ya3BkYTFOaHJnaS8xMVphb1lmVE0vQmRrNi9IUkFvR0JBSnFzTmFZYzE2clVzYzAzUEwybApXUGNzRnZxZGI3SEJyakVSRkhFdzQ0Vkt2MVlxK0ZWYnNNN1FTQVZ1V1llcGxGQUpDYzcrSEt1YjRsa1hRM1RkCndSajJLK2pOUzJtUXp1Y2hOQnlBZ1hXVnYveHhMZEE3NnpuWmJYdjl5cXhnTVVjTVZwZGRuSkxVZm9QVVZ1dTcKS0tlVVU3TTNIblRKUStrcldtbUxraUlSCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K

examples/custom-resources/backup-directive/transport-server/nginx-config.yaml

@@ -0,0 +1,7 @@
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: nginx-config
+  namespace: nginx-ingress
+data:
+  resolver-addresses: "kube-dns.kube-system.svc.cluster.local"