feat!: migrate to ARC (terraform-google-modules#173)

Author: gtsorbo

README.md

@@ -1,81 +1,41 @@
 # terraform-google-github-actions-runners
-
 [![awesome-runners](https://img.shields.io/badge/listed%20on-awesome--runners-blue.svg)](https://github.com/jonico/awesome-runners)
 
-Using these Terraform modules you can quickly deploy Self Hosted Github Runners for jobs in your GitHub Actions workflows
-
-## [Self Hosted Runners on GKE](modules/gh-runner-gke/README.md)
-
-The `gh-runner-gke` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using GKE.
-
-This includes
-
-- Enabling necessary APIs
-- VPC
-- GKE Cluster
-- Kubernetes Secret
-
-Below are some examples:
+## Description
+Using these Terraform modules you can quickly deploy self-hosted GitHub Runners for jobs in your GitHub Actions workflows
 
-### [Self Hosted runners on GKE that support Docker Workflows](examples/gh-runner-gke-dind/README.md)
+## Modules
 
-This example shows how to deploy Self Hosted Runners on GKE that supports Docker Workflows.
+### [Self Hosted Runners on GKE](modules/gh-runner-gke/)
+The `gh-runner-gke` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using GKE. This module follows the guidance for using [Actions Runner Controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller#installing-actions-runner-controller) provided by GitHub
 
-### [Simple Self Hosted Runners on GKE](examples/gh-runner-gke-simple/README.md)
-
-This example shows how to deploy a simple GKE Self Hosted Runner.
+Examples:
+- [Simple Self Hosted Runners on GKE](examples/gh-runner-gke-simple/)
+- [Self Hosted runners on GKE that support Docker Workflows](examples/gh-runner-gke-dind/)
+- [Self Hosted runners on GKE that support Docker Workflows in rootless configuration](examples/gh-runner-gke-dind-rootless/)
 
 More examples of [Self Hosted Runners on GKE/Anthos](https://github.com/github-developer/self-hosted-runners-anthos).
 
-## [Self Hosted Runners on Managed Instance Groups using VMs](modules/gh-runner-mig-vm/README.md)
 
+### [Self Hosted Runners on Managed Instance Groups using VMs](modules/gh-runner-mig-vm/)
 The `gh-runner-mig-vm` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using Managed Instance Groups.
 
-This includes
-
-- Enabling necessary APIs
-- VPC
-- NAT & Cloud Router
-- Service Account for MIG
-- MIG Instance Template
-- MIG Instance Manager
-- FW Rules
-- Secret Manager Secret
-
-Deployment of Managed Instance Groups requires a [Google VM image](https://cloud.google.com/compute/docs/images) with a startup script that downloads and configures the Runner or a pre-baked image with the runner installed.
-
-Below are some examples:
-
-### [Simple Self Hosted Runner on MIG VMs](examples/gh-runner-mig-native-simple/README.md)
-
-This example shows how to deploy a MIG Self Hosted Runner with startup scripts.
-
-### [Self Hosted Runner on MIG VMs from Packer Image](examples/gh-runner-mig-native-packer/README.md)
-
-This example shows how to deploy a MIG Self Hosted Runner with an image pre-baked using Packer.
-
-## [Self Hosted Runners on Managed Instance Groups using Container VMs](modules/gh-runner-mig-container-vm/README.md)
+Examples:
+- [Simple Self Hosted Runner on MIG VMs](examples/gh-runner-mig-native-simple/)
+- [Self Hosted Runner on MIG VMs from Packer Image](examples/gh-runner-mig-native-packer/)
 
+### [Self Hosted Runners on Managed Instance Groups using Container VMs](modules/gh-runner-mig-container-vm/)
 The `gh-runner-mig-container-vm` module provisions the resources required to deploy Self Hosted Runners on GCP infrastructure using Managed Instance Groups.
 
-This includes
-
-- Enabling necessary APIs
-- VPC
-- NAT & Cloud Router
-- MIG Container Instance Template
-- MIG Instance Manager
-- FW Rules
+Examples:
+- [Simple Self Hosted Runner on MIG Container VMs](examples/gh-runner-mig-container-vm-simple/)
+- [Self Hosted runners on MIG Container VMs that support Docker Workflows](examples/gh-runner-mig-container-vm-dind/)
 
-Below are some examples:
+### [GitHub OIDC](modules/gh-oidc/)
+This module handles the opinionated creation of infrastructure necessary to configure [Workload Identity pools](https://cloud.google.com/iam/docs/workload-identity-federation#pools) and [providers](https://cloud.google.com/iam/docs/workload-identity-federation#providers) for authenticating to GCP using GitHub Actions OIDC tokens.
 
-### [Self Hosted runners on MIG Container VMs that support Docker Workflows](examples/gh-runner-mig-container-vm-dind/README.md)
-
-This example shows how to deploy a Self Hosted Runner that supports Docker Workflows on MIG Container VMs.
-
-### [Simple Self Hosted Runner on MIG Container VMs](examples/gh-runner-mig-container-vm-simple/README.md)
-
-This example shows how to deploy a Self Hosted Runner on MIG Container VMs.
+Examples:
+- [Simple Workload Identity configuration for GitHub OIDC](examples/oidc-simple/)
 
 
 ## Requirements
@@ -94,5 +54,11 @@ The following dependencies must be available:
 Refer to the [contribution guidelines](./CONTRIBUTING.md) for
 information on contributing to this module.
 
+[iam-module]: https://registry.terraform.io/modules/terraform-google-modules/iam/google
+[project-factory-module]: https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
 [terraform-provider-gcp]: https://www.terraform.io/docs/providers/google/index.html
 [terraform]: https://www.terraform.io/downloads.html
+
+## Security Disclosures
+
+Please see our [security disclosure process](./SECURITY.md).

build/int.cloudbuild.yaml

@@ -45,22 +45,54 @@ steps:
   args: ['/bin/bash', '-c', 'cft test run TestGHRunnerMIGContainerVM --stage destroy --verbose']
   waitFor:
     - verify-gh-runner-mig-container-vm
-# test/fixtures/gh-runner-gke
-- id: apply-gh-runner-gke
+# TestAll/examples/gh-runner-gke-simple
+- id: apply-gh-runner-gke-simple
   name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
-  args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage apply --verbose']
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage apply --verbose']
   waitFor:
       - init-all
-- id: verify-gh-runner-gke
+- id: verify-gh-runner-gke-simple
   name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
-  args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage verify --verbose']
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage verify --verbose']
   waitFor:
-      - apply-gh-runner-gke
-- id: destroy-gh-runner-gke
+      - apply-gh-runner-gke-simple
+- id: destroy-gh-runner-gke-simple
   name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
-  args: ['/bin/bash', '-c', 'cft test run TestGHRunnerGKE --stage destroy --verbose']
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeSimple --stage destroy --verbose']
   waitFor:
-      - verify-gh-runner-gke
+      - verify-gh-runner-gke-simple
+# TestAll/examples/gh-runner-gke-dind
+- id: apply-gh-runner-gke-dind
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage apply --verbose']
+  waitFor:
+      - init-all
+- id: verify-gh-runner-gke-dind
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage verify --verbose']
+  waitFor:
+      - apply-gh-runner-gke-dind
+- id: destroy-gh-runner-gke-dind
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDind --stage destroy --verbose']
+  waitFor:
+      - verify-gh-runner-gke-dind
+# TestAll/examples/gh-runner-gke-dind-rootless
+- id: apply-gh-runner-gke-dind-rootless
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage apply --verbose']
+  waitFor:
+      - init-all
+- id: verify-gh-runner-gke-dind-rootless
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage verify --verbose']
+  waitFor:
+      - apply-gh-runner-gke-dind-rootless
+- id: destroy-gh-runner-gke-dind-rootless
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'cft test run TestGhRunnerGkeDindRootless --stage destroy --verbose']
+  waitFor:
+      - verify-gh-runner-gke-dind-rootless
 # examples/oidc-simple
 - id: apply-oidc
   name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'

docs/upgrading_to_v4.0.md

@@ -0,0 +1,30 @@
+# Upgrading to v4.0
+
+The v4.0 release of the `gh-runner-gke` module contains breaking changes.
+## Migration Instructions
+
+### `gh-runner-gke` module
+
+```diff
+ module "runner-gke" {
+  source  = "terraform-google-modules/github-actions-runners/google//modules/gh-runner-gke"
+-  version = "~> 3.0"
++  version = "~> 4.0"
+
+  project_id             = var.project_id
+  create_network         = true
++  cluster_suffix         = "repo"
+-  repo_name              = "repo"
+-  repo_owner             = "repo_owner"
+-  repo_url               = "repo_url"
+-  gh_token               = "gh_token"
++  gh_app_id              = "123456"
++  gh_app_installation_id = "12345678"
++  gh_app_private_key     = "sample"
++  gh_config_url          = "https://github.com/ORGANIZATION"
+}
+```
+
+- Previously, the name of the created GKE cluster appended `repo_name` as a suffix. Now, the value of the suffix is set via `cluster_suffix`, as runner clusters can be created for workflows more than one GitHub repo. To prevent cluster deletion when upgrading, set the value of `cluster_suffix` to the previous value of `repo_name`.
+- The module now prefers authentication via GitHub App installation. Follow the instructions for [authenticating to ARC via GitHub App](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app), and provide the necessary values to the module instead of the previous repo authentication values. Upgrading will destroy the previously created `runner_k8s_config` kubernetes secret, so previous GitHub Runner images may fail.
+- Previously, the GKE cluster service account was provided with the `roles/storage.objectViewer` role to enable pulling images from Container Registry. The module no longer requires access to Container Registry, and so IAM membership has been removed. If your cluster relies on this role, you will need to add it again manually.

examples/gh-runner-gke-dind-rootless/README.md

@@ -0,0 +1,41 @@
+# Self Hosted runners on GKE that support dind-rootless
+
+## Overview
+
+This example shows how to deploy runners that support Docker Workflows on GKE in a rootless configuration. This example is based on the instructions published in [Deploying GitHub Action Runners on GKE with dind-rootless](https://medium.com/google-cloud/github-action-runners-on-gke-with-dind-rootless-bd54e23516c9).
+
+More examples of [Self Hosted Runners on GKE/Anthos](https://github.com/github-developer/self-hosted-runners-anthos).
+
+## Deployment
+
+1. Follow the instructions in the [GitHub documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app) to create a GitHub App for authenticating ARC
+
+1. Gather the values for your GitHub App ID, GitHub App Installation ID, and GitHub App Private Key from the instructions linked above.
+
+1. Substitute your values into the example [`main.tf`](main.tf). Modify any other values as needed. For a full list of available variables, refer to the [module documentation](../../modules/gh-runner-gke/).
+
+1. Execute Terraform commands to create the required resources.
+```sh
+terraform init
+terraform apply
+```
+
+1. Your runners should become active at `https://github.com/organizations/ORGANIZATION/settings/actions/runners`.
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| project\_id | The project id to deploy Github Runner MIG | `string` | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| cluster\_name | Cluster name |
+| location | Cluster location |
+| project\_id | The project in which resources are created |
+| service\_account | The default service account used for running nodes. |
+
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/gh-runner-gke-dind-rootless/main.tf

@@ -0,0 +1,35 @@
+/**
+ * Copyright 2020 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+module "runner-gke" {
+  source  = "terraform-google-modules/github-actions-runners/google//modules/gh-runner-gke"
+  version = "~> 3.0"
+
+  project_id             = var.project_id
+  create_network         = true
+  network_name           = "runner-network-dind-r"
+  subnet_name            = "runner-subnet-dind-r"
+  cluster_suffix         = "dind-rootless"
+  gh_app_id              = "123456"
+  gh_app_installation_id = "12345678"
+  gh_app_private_key     = "sample"
+  gh_config_url          = "https://github.com/ORGANIZATION"
+
+  # pass values.yaml for dind-rootless runners configuratin
+  arc_runners_values = [
+    file("${path.module}/values.yaml")
+  ]
+}

test/fixtures/gh-runner-gke/outputs.tf renamed to examples/gh-runner-gke-dind-rootless/outputs.tf

@@ -16,28 +16,20 @@
 
 output "service_account" {
   description = "The default service account used for running nodes."
-  value       = module.example_gke_runner.service_account
+  value       = module.runner-gke.service_account
 }
 
 output "cluster_name" {
   description = "Cluster name"
-  value       = module.example_gke_runner.cluster_name
-}
-output "network_name" {
-  description = "Name of VPC"
-  value       = module.example_gke_runner.network_name
-}
-
-output "subnet_name" {
-  description = "Name of VPC"
-  value       = module.example_gke_runner.subnet_name
+  value       = module.runner-gke.cluster_name
 }
 
 output "location" {
   description = "Cluster location"
-  value       = module.example_gke_runner.location
+  value       = module.runner-gke.location
 }
 
 output "project_id" {
-  value = var.project_id_gke
+  value       = var.project_id
+  description = "The project in which resources are created"
 }

examples/gh-runner-gke-dind-rootless/providers.tf

@@ -0,0 +1,40 @@
+/**
+ * Copyright 2020 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*****************************************
+  Kubernetes provider configuration
+ *****************************************/
+
+data "google_client_config" "default" {
+}
+
+provider "kubernetes" {
+  host                   = "https://${module.runner-gke.kubernetes_endpoint}"
+  token                  = data.google_client_config.default.access_token
+  cluster_ca_certificate = base64decode(module.runner-gke.ca_certificate)
+}
+
+provider "helm" {
+  kubernetes {
+    host                   = "https://${module.runner-gke.kubernetes_endpoint}"
+    token                  = data.google_client_config.default.access_token
+    cluster_ca_certificate = base64decode(module.runner-gke.ca_certificate)
+    exec {
+      api_version = "client.authentication.k8s.io/v1beta1"
+      command     = "gke-gcloud-auth-plugin"
+    }
+  }
+}