executors: add manifests for native executors (sourcegraph#4255)

* executors: add manifests for native executors

* Finish readme, implement -1 sentinel values

Author: danieldides

configure/executors/README.md

@@ -8,11 +8,42 @@ This directory contains manifests for the optional deployment of Sourcegraph Exe
 
 It is expected that all components contained in this directory and any subdirectories are deployed to ensure full functionality and best performance.
 
+There are two distribution methods supported:
+
+### Native Kubernetes Executors (Recommended)
+Requirements: RBAC, persistent volumes
+
+This distribution method makes use of native Kubernetes Deployments, Services, and Jobs to execute workloads. It is suitable for clusters that meet Sourcegraph's minimum requirements.
+
 The following components will deployed:
+- [Executor Deployment](./executor/k8s/executor.Deployment.yaml) An Executor replica with a Docker sidecar to run isolated batch changes and auto-indexing jobs. This deployment requires a [privileged security context](https://kubernetes.io/docs/concepts/security/pod-security-standards/).
+- [Executor Service](./executor/k8s/executor.Service.yaml) A headless service for executor metrics access. Executors are not externally accessible.
+- [Executor ConfigMap](./executor/k8s/executor.ConfigMap.yaml) configuration for the Executor deployment
+- RBAC
+  - [Role](./executor/k8s/rbac/executor.Role.yaml) 
+  - [RoleBinding](./executor/k8s/rbac/executor.RoleBinding.yaml) 
+  - [ServiceAccount](./executor/k8s/rbac/executor.ServiceAccount.yaml) 
+- [Private docker registory]
+  - [Registry Deployment](./private-docker-registry/private-docker-registry.Deployment.yaml) A private docker registry configured as a pull-through cache to avoid docker hub rate limiting.
+  - [Registry Service](./private-docker-registry/private-docker-registry.Service.yaml) A service to access the private-docker-registry.
+  - [Registry Persistent Volume](./private-docker-registry/private-docker-registry.PersistentVolumeClaim.yaml) A volume to store images in the private-docker-registry.
 
-- [Executor Deployment](./executor/executor.Deployment.yaml) An Executor replica with a Docker sidecar to run isolated batch changes and auto-indexing jobs. This deployment requires a [privileged security context](https://kubernetes.io/docs/concepts/security/pod-security-standards/).
-- [Executor Service](./executor/executor.Service.yaml) A headless service for executor metrics access. Executors are not externally accessible.
-- [Docker ConfigMap](./executor/docker-daemon.ConfigMap.yaml) configuration for the docker sidecar to use the pull-through cache.
+To apply these manifests, run the following command:
+
+```bash
+kubectl apply -f . --recursive private-docker-registry
+kubectl apply -f . --recursive k8s
+```
+
+### Docker-in-Docker Kubernetes Executors
+Requirements: elevated permissions, persistent volumes
+
+This distribution method makes use of a docker-in-docker sidecar container to execute the workloads. It is suitable for clusters that meet Sourcegraph's minimum requirements that cannot utilize native Kubenretes executors.
+
+The following components will deployed:
+- [Executor Deployment](./executor/dind/executor.Deployment.yaml) An Executor replica with a Docker sidecar to run isolated batch changes and auto-indexing jobs. This deployment requires a [privileged security context](https://kubernetes.io/docs/concepts/security/pod-security-standards/).
+- [Executor Service](./executor/dind/executor.Service.yaml) A headless service for executor metrics access. Executors are not externally accessible.
+- [Docker ConfigMap](./executor/dind/docker-daemon.ConfigMap.yaml) configuration for the docker sidecar to use the pull-through cache.
 - [Private docker registory]
   - [Registry Deployment](./private-docker-registry/private-docker-registry.Deployment.yaml) A private docker registry configured as a pull-through cache to avoid docker hub rate limiting.
   - [Registry Service](./private-docker-registry/private-docker-registry.Service.yaml) A service to access the private-docker-registry.
@@ -21,6 +52,7 @@ The following components will deployed:
 To apply these manifests, run the following command:
 
 ```bash
-kubectl apply -f . --recursive
+kubectl apply -f . --recursive private-docker-registry
+kubectl apply -f . --recursive dind
 ```
 

configure/executors/executor/docker-daemon.ConfigMap.yaml renamed to configure/executors/dind/docker-daemon.ConfigMap.yaml

@@ -0,0 +1,30 @@
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: executor-config
+  labels:
+    app: executor
+    deploy: sourcegraph
+    sourcegraph-resource-requires: no-cluster-admin
+    app.kubernetes.io/component: executor
+# Refer to https://docs.sourcegraph.com/admin/executors/deploy_executors_binary#step-2-setup-environment-variables on how to populate these variables
+data:
+  EXECUTOR_USE_FIRECRACKER: "false"
+  EXECUTOR_KUBERNETES_PERSISTENCE_VOLUME_NAME: "sg-executor-pvc"
+  EXECUTOR_KUBERNETES_POD_AFFINITY: '[{"labelSelector": {"matchExpressions": [{"key": "app", "operator": "In", "values": ["executor"]}]}, "topologyKey": "kubernetes.io/hostname"}]'
+  # If Sourcegraph is not deployed in the `default` namespace, update this value
+  EXECUTOR_FRONTEND_URL: "http://sourcegraph-frontend.default.svc.cluster.local:30080"
+  EXECUTOR_MAXIMUM_NUM_JOBS: "8"
+  # Used configure which queues Executors will process.
+  # Can be "batches" or "codeintel"
+  # Either set this or EXECUTOR_QUEUE_NAMES.
+  # EXECUTOR_QUEUE_NAME: "codeintel"
+  # Used configure which queues Executors will process.
+  # Can be "batches" or "codeintel" or "batches,codeintel"
+  # Either set this or EXECUTOR_QUEUE_NAME.
+  EXECUTOR_QUEUE_NAMES: "batches,codeintel"
+  EXECUTOR_KUBERNETES_RESOURCE_REQUEST_MEMORY: "5Gi"
+#  KUBERNETES_RUN_AS_USER: "-1"
+#  KUBERNETES_RUN_AS_GROUP: "-1"
+#  KUBERNETES_FS_GROUP: "1000"

configure/executors/executor/executor.Deployment.yaml renamed to configure/executors/dind/executor.Deployment.yaml

@@ -0,0 +1,67 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: executor
+  annotations:
+    description: Runs sourcegraph executor replicas for batch changes and codeintel auto indexing.
+    kubectl.kubernetes.io/default-container: executor
+  labels:
+    deploy: sourcegraph
+    sourcegraph-resource-requires: no-cluster-admin
+    app.kubernetes.io/component: executor
+spec:
+  selector:
+    matchLabels:
+      app: executor
+  minReadySeconds: 10
+  replicas: 1
+  revisionHistoryLimit: 10
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  template:
+    metadata:
+      labels:
+        app: executor
+    spec:
+      serviceAccountName: executor
+      containers:
+        - name: executor
+          image: index.docker.io/sourcegraph/executor-kubernetes:5.1_230340_2023-06-23_5.0-93d39d620e83@sha256:172770133661d4d148327d2cac87c051a6409871ee6f6e28ce3495b60f883ad5
+          imagePullPolicy: Always
+          livenessProbe:
+            exec:
+              command:
+                - /usr/bin/pgrep
+                - -f
+                - /usr/local/bin/executor
+            initialDelaySeconds: 15
+            timeoutSeconds: 5
+          readinessProbe:
+            exec:
+              command:
+                - /usr/bin/pgrep
+                - -f
+                - /usr/local/bin/executor
+            periodSeconds: 5
+          terminationMessagePolicy: FallbackToLogsOnError
+          env:
+            - name: EXECUTOR_FRONTEND_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: executor-secret
+                  key: password
+          # Refer to https://docs.sourcegraph.com/admin/executors/deploy_executors_binary#step-2-setup-environment-variables on how to populate these variables
+          envFrom:
+            - configMapRef:
+                name: executor-config
+          volumeMounts:
+            - mountPath: /data
+              name: sg-executor-volume
+      volumes:
+        - name: sg-executor-volume
+          persistentVolumeClaim:
+            claimName: sg-executor-pvc

configure/executors/executor/executor.Service.yaml renamed to configure/executors/dind/executor.Service.yaml

@@ -0,0 +1,15 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: sg-executor-pvc
+  labels:
+    deploy: sourcegraph
+    sourcegraph-resource-requires: no-cluster-admin
+    app.kubernetes.io/component: executor
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 100Gi

configure/executors/k8s/executor.ConfigMap.yaml

@@ -0,0 +1,21 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+    prometheus.io/port: "6060"
+    sourcegraph.prometheus/scrape: "true"
+  labels:
+    app: executor
+    deploy: sourcegraph
+    sourcegraph-resource-requires: no-cluster-admin
+    app.kubernetes.io/component: executor
+  name: executor
+spec:
+  ports:
+    - name: debug
+      port: 6060
+      targetPort: debug
+  selector:
+    app: executor
+  type: ClusterIP

configure/executors/k8s/executor.Deployment.yaml

@@ -0,0 +1,29 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: executor
+  labels:
+    category: rbac
+    deploy: sourcegraph
+    sourcegraph-resource-requires: cluster-admin
+    app.kubernetes.io/component: executor
+rules:
+  # Executors create Job pods to run processes. Once Jobs are completed, they are cleaned up.
+  - apiGroups:
+      - batch
+    resources:
+      - jobs
+    verbs:
+      - create
+      - delete
+  # Executors need to look up and steam logs from the Job Pods.
+  - apiGroups:
+      - ""
+    resources:
+      - pods
+      - pods/log
+    verbs:
+      - get
+      - list
+      - watch

configure/executors/k8s/executor.PersistentVolumeClaim.yaml

@@ -0,0 +1,18 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: sg-executor-role-binding
+  labels:
+    category: rbac
+    deploy: sourcegraph
+    sourcegraph-resource-requires: cluster-admin
+    app.kubernetes.io/component: executor
+subjects:
+  - kind: ServiceAccount
+    name: executor
+    namespace: default
+roleRef:
+  apiGroup: "rbac.authorization.k8s.io"
+  kind: Role
+  name: executor