Skip to content

Commit fd5329c

Browse files
amandakarinadaniel-citbharathkkb
authored
feat: Add step 5-app-infra (terraform-google-modules#382)
* Adds environment base module to create gce instance * Adds development environment for business unit 1 example * Adds empty README for step 5 * Adds non-prodcution environment for bu1 example * Adds production environment for bu1 example * Adds common.tfvars example and symbolic links * Adds fixture for 5-app-infra * Adds outputs for test, adds initial validation of gce instance * Updates README * fix linting on readme * Adds step on build, adds number of instances as variable * Adds tmp_plan folder on gitignore, pins version of modules * Changes impersionate configuration to native * Adds name of instances in outputs * Fixes symbolic links * Fixes fixture for test * Adds example tfvars * Adds initial version of README * Fixes type of regioni * Fixes type * Fixes integration test * Adds repository for step 5, adds instructions on README * Apply suggestions from code review Co-authored-by: Bharath KKB <[email protected]> * Undo app infra repo * Adds instructions for app infra pipeline * Removes static_ips variable, adds environment tfvars * Fixes service_account variable for test * Undo change on bootstrap test * Adds instructions for deploy * Renames variable for service account used to create infra * Apply suggestions from code review to step 5 README Co-authored-by: Bharath KKB <[email protected]> * fix lint issues Co-authored-by: Daniel da Silva Andrade <[email protected]> Co-authored-by: Bharath KKB <[email protected]>
1 parent 03bd05a commit fd5329c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+1322
-0
lines changed

‎.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ override.tf.json
5757
**/.kitchen
5858
**/.kitchen.local.yml
5959
**/Gemfile.lock
60+
# Plan files
61+
**/tmp_plan
6062

6163
test/fixtures/shared/terraform.tfvars
6264

‎.kitchen.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,3 +113,13 @@ suites:
113113
backend: local
114114
controls:
115115
- gcloud-projects
116+
- name: app-infra
117+
driver:
118+
root_module_directory: test/fixtures/app-infra/
119+
verifier:
120+
color: false
121+
systems:
122+
- name: app-infra
123+
backend: gcp
124+
controls:
125+
- gcp-app-infra

‎5-app-infra/README.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# 5-app-infra
2+
3+
The purpose of this step is to deploy a simple [Compute Engine](https://cloud.google.com/compute/) instance in one of the business unit projects using the infra pipeline setup in 4-projects.
4+
The infra pipeline is created in step 4-projects within the shared env and has a [Cloudbuild](https://cloud.google.com/build/docs) pipeline configured to manage infrastructure within projects. To enable deployment via this pipeline, the projects deployed should [enable](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/4-projects/business_unit_1/development/example_base_shared_vpc_project.tf#L31-L32) `enable_cloudbuild_deploy` flag and provide the Cloud Build service account value via`cloudbuild_sa`.
5+
6+
This enables the Cloud Build service account to impersonate the project service account and use it to deploy infrastructure. The roles required for project SA can also be [managed](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/4-projects/business_unit_1/development/example_base_shared_vpc_project.tf#L30) via `sa_roles`. (Note: This requires per project SA impersonation, if you would like to have a single SA managing an environment and all associated projects, that is also possible by [granting](https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/4-projects/modules/single_project/main.tf#L62-L68) `roles/iam.serviceAccountTokenCreator` to an SA with the right roles in `4-projects/env`.
7+
8+
There is also a [Source Repository](https://cloud.google.com/source-repositories) configured with build triggers similar to [foundation pipeline](https://github.com/terraform-google-modules/terraform-example-foundation#0-bootstrap) setup in `0-bootstrap`.
9+
This Compute Engine instance will be created using the base network created during step 3-networks to access private services.
10+
11+
## Prerequisites
12+
13+
1. 0-bootstrap executed successfully.
14+
1. 1-org executed successfully.
15+
1. 2-environments executed successfully.
16+
1. 3-networks executed successfully.
17+
1. 4-projects executed successfully.
18+
19+
## Usage
20+
21+
### Setup to run via Cloud Build
22+
23+
1. Clone repo `gcloud source repos clone bu1-example-app --project=prj-bu1-c-infra-pipeline-<random>`. (this is from the terraform output from the previous section, run `terraform output cloudbuild_project_id` in the `4-projects/business_unit_1/shared` folder)
24+
1. Navigate into the repo `cd bu1-example-app`.
25+
1. Change freshly cloned repo and change to non master branch `git checkout -b plan`.
26+
1. Copy contents of foundation to new repo `cp -RT ../terraform-example-foundation/5-app-infra/ .` (modify accordingly based on your current directory).
27+
1. Copy cloud build configuration files for terraform `cp ../terraform-example-foundation/build/cloudbuild-tf-* . ` (modify accordingly based on your current directory).
28+
1. Copy terraform wrapper script `cp ../terraform-example-foundation/build/tf-wrapper.sh . ` to the root of your new repository (modify accordingly based on your current directory).
29+
1. Ensure wrapper script can be executed `chmod 755 ./tf-wrapper.sh`.
30+
1. Rename `common.auto.example.tfvars` to `common.auto.tfvars` and update the file with values from your environment.
31+
1. Rename `bu1-development.auto.example.tfvars` to `bu1-development.auto.tfvars` and update the file with values from your environment.
32+
1. Rename `bu1-non-production.auto.example.tfvars` to `bu1-non-production.auto.tfvars` and update the file with values from your environment.
33+
1. Rename `bu1-production.auto.example.tfvars` to `bu1-production.auto.tfvars` and update the file with values from your environment.
34+
1. Commit changes with `git add .` and `git commit -m 'Your message'`.
35+
1. Push your plan branch to trigger a plan for all environments `git push --set-upstream origin plan` (the branch `plan` is not a special one. Any branch which name is different from `development`, `non-production` or `production` will trigger a terraform plan).
36+
1. Review the plan output in your cloud build project https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
37+
1. Merge changes to development with `git checkout -b development` and `git push origin development`.
38+
1. Review the apply output in your cloud build project https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
39+
1. Merge changes to non-production with `git checkout -b non-production` and `git push origin non-production`.
40+
1. Review the apply output in your cloud build project https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
41+
1. Merge changes to production branch with `git checkout -b production` and `git push origin production`.
42+
1. Review the apply output in your cloud build project https://console.cloud.google.com/cloud-build/builds?project=YOUR_CLOUD_BUILD_PROJECT_ID
43+
44+
### Run terraform locally
45+
46+
1. Change into 5-app-infra folder.
47+
1. Run `cp ../build/tf-wrapper.sh .`
48+
1. Run `chmod 755 ./tf-wrapper.sh`.
49+
1. Rename `common.auto.example.tfvars` to `common.auto.tfvars` and update the file with values from your environment and bootstrap.
50+
1. Update backend.tf with your bucket from infra pipeline example. You can run
51+
```for i in `find -name 'backend.tf'`; do sed -i 's/UPDATE_ME/<YOUR-BUCKET-NAME>/' $i; done```.
52+
53+
We will now deploy each of our environments(development/production/non-production) using this script.
54+
When using Cloud Build or Jenkins as your CI/CD tool each environment corresponds to a branch is the repository for 5-app-infra step and only the corresponding environment is applied.
55+
56+
To use the `validate` option of the `tf-wrapper.sh` script, the latest version of `terraform-validator` must be [installed](https://github.com/forseti-security/policy-library/blob/master/docs/user_guide.md#how-to-use-terraform-validator) in your system and in you `PATH`.
57+
58+
1. Run `./tf-wrapper.sh init production`.
59+
1. Run `./tf-wrapper.sh plan production` and review output.
60+
1. Run `./tf-wrapper.sh validate production $(pwd)/../policy-library <YOUR_INFRA_PIPELINE_PROJECT>` and check for violations.
61+
1. Run `./tf-wrapper.sh apply production`.
62+
1. Run `./tf-wrapper.sh init non-production`.
63+
1. Run `./tf-wrapper.sh plan non-production` and review output.
64+
1. Run `./tf-wrapper.sh plan non-production` and review output.
65+
1. Run `./tf-wrapper.sh validate non-production $(pwd)/../policy-library <YOUR_INFRA_PIPELINE_PROJECT>` and check for violations.
66+
1. Run `./tf-wrapper.sh apply non-production`.
67+
1. Run `./tf-wrapper.sh init development`.
68+
1. Run `./tf-wrapper.sh plan development` and review output.
69+
1. Run `./tf-wrapper.sh validate development $(pwd)/../policy-library <YOUR_INFRA_PIPELINE_PROJECT>` and check for violations.
70+
1. Run `./tf-wrapper.sh apply development`.
71+
72+
If you received any errors or made any changes to the Terraform config or `terraform.tfvars` you must re-run `./tf-wrapper.sh plan <env>` before run `./tf-wrapper.sh apply <env>`.

‎5-app-infra/bu1-development.auto.example.tfvars

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
project_service_account = "project-service-account@prj-bu1-d-sample-base-<random>.iam.gserviceaccount.com"

‎5-app-infra/bu1-non-production.auto.example.tfvars

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
project_service_account = "project-service-account@prj-bu1-n-sample-base-<random>.iam.gserviceaccount.com"

‎5-app-infra/bu1-production.auto.example.tfvars

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
project_service_account = "project-service-account@prj-bu1-p-sample-base-<random>.iam.gserviceaccount.com"

‎5-app-infra/business_unit_1/development/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
2+
## Inputs
3+
4+
| Name | Description | Type | Default | Required |
5+
|------|-------------|------|---------|:--------:|
6+
| folder\_prefix | Name prefix to use for folders created. | `string` | `"fldr"` | no |
7+
| instance\_region | The region where compute instance will be created. A subnetwork must exists in the instance region. | `string` | n/a | yes |
8+
| org\_id | The organization id for the associated services | `string` | n/a | yes |
9+
| parent\_folder | Optional - if using a folder for testing. | `string` | `""` | no |
10+
| project\_service\_account | Service account email of the account created on step 4-project for the project where the GCE will be created | `string` | n/a | yes |
11+
12+
## Outputs
13+
14+
| Name | Description |
15+
|------|-------------|
16+
| available\_zones | List of available zones in region |
17+
| instances\_details | List of details for compute instances |
18+
| instances\_names | List of names for compute instances |
19+
| instances\_self\_links | List of self-links for compute instances |
20+
| instances\_zones | List of zone for compute instances |
21+
| project\_id | Project where compute instance was created |
22+
| region | Region where compute instance was created |
23+
24+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

‎5-app-infra/business_unit_1/development/backend.tf

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
terraform {
18+
backend "gcs" {
19+
bucket = "UPDATE_ME"
20+
prefix = "terraform/app-infra/business_unit_1/development"
21+
}
22+
}

‎5-app-infra/business_unit_1/development/bu1-development.auto.tfvars

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../bu1-development.auto.tfvars

‎5-app-infra/business_unit_1/development/common.auto.tfvars

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../common.auto.tfvars

‎5-app-infra/business_unit_1/development/main.tf

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
18+
19+
data "google_active_folder" "env" {
20+
display_name = "${var.folder_prefix}-development"
21+
parent = var.parent_folder != "" ? "folders/${var.parent_folder}" : "organizations/${var.org_id}"
22+
}
23+
24+
module "base_shared_gce_instance" {
25+
source = "../../modules/env_base"
26+
environment = "development"
27+
vpc_type = "base"
28+
num_instances = 1
29+
folder_id = data.google_active_folder.env.name
30+
business_code = "bu1"
31+
project_suffix = "sample-base"
32+
region = var.instance_region
33+
}

‎5-app-infra/business_unit_1/development/outputs.tf

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
output "instances_self_links" {
18+
description = "List of self-links for compute instances"
19+
value = module.base_shared_gce_instance.instances_self_links
20+
}
21+
22+
output "instances_names" {
23+
description = "List of names for compute instances"
24+
value = [for u in module.base_shared_gce_instance.instances_details : u.name]
25+
}
26+
27+
output "instances_zones" {
28+
description = "List of zone for compute instances"
29+
value = [for u in module.base_shared_gce_instance.instances_details : u.zone]
30+
}
31+
32+
output "instances_details" {
33+
description = "List of details for compute instances"
34+
value = module.base_shared_gce_instance.instances_details
35+
}
36+
37+
output "available_zones" {
38+
description = "List of available zones in region"
39+
value = module.base_shared_gce_instance.available_zones
40+
}
41+
42+
output "project_id" {
43+
description = "Project where compute instance was created"
44+
value = module.base_shared_gce_instance.project_id
45+
}
46+
47+
output "region" {
48+
description = "Region where compute instance was created"
49+
value = module.base_shared_gce_instance.region
50+
}

‎5-app-infra/business_unit_1/development/providers.tf

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
locals {
18+
tf_sa = var.project_service_account
19+
}
20+
21+
/******************************************
22+
Provider credential configuration
23+
*****************************************/
24+
provider "google" {
25+
impersonate_service_account = local.tf_sa
26+
}
27+
28+
provider "google-beta" {
29+
impersonate_service_account = local.tf_sa
30+
}

‎5-app-infra/business_unit_1/development/variables.tf

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
/**
2+
* Copyright 2021 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
variable "project_service_account" {
18+
description = "Service account email of the account created on step 4-project for the project where the GCE will be created"
19+
type = string
20+
}
21+
22+
variable "org_id" {
23+
description = "The organization id for the associated services"
24+
type = string
25+
}
26+
27+
variable "instance_region" {
28+
description = "The region where compute instance will be created. A subnetwork must exists in the instance region."
29+
type = string
30+
}
31+
32+
variable "folder_prefix" {
33+
description = "Name prefix to use for folders created."
34+
type = string
35+
default = "fldr"
36+
}
37+
38+
variable "parent_folder" {
39+
description = "Optional - if using a folder for testing."
40+
type = string
41+
default = ""
42+
}

‎5-app-infra/business_unit_1/non-production/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
2+
## Inputs
3+
4+
| Name | Description | Type | Default | Required |
5+
|------|-------------|------|---------|:--------:|
6+
| folder\_prefix | Name prefix to use for folders created. | `string` | `"fldr"` | no |
7+
| instance\_region | The region where compute instance will be created. A subnetwork must exists in the instance region. | `string` | n/a | yes |
8+
| org\_id | The organization id for the associated services | `string` | n/a | yes |
9+
| parent\_folder | Optional - if using a folder for testing. | `string` | `""` | no |
10+
| project\_service\_account | Service account email of the account created on step 4-project for the project where the GCE will be created | `string` | n/a | yes |
11+
12+
## Outputs
13+
14+
| Name | Description |
15+
|------|-------------|
16+
| available\_zones | List of available zones in region |
17+
| instances\_details | List of details for compute instances |
18+
| instances\_names | List of names for compute instances |
19+
| instances\_self\_links | List of self-links for compute instances |
20+
| instances\_zones | List of zone for compute instances |
21+
| project\_id | Project where compute instance was created |
22+
| region | Region where compute instance was created |
23+
24+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

0 commit comments

Comments
 (0)