Merge pull request hashicorp#73 from hashicorp/automation-script-mods

added handling of run_status planned

Author: rberlind

operations/automation-script/README.md

@@ -8,14 +8,18 @@ The script does the following steps:
 1. Packages main.tf into the myconfig.tar.gz file.
 1. Creates the workspace.
 1. Creates a new configuration version.
-1. Uploads the myconfig.tar.gz file as a new configuration. (This step used to trigger an initial run which caused an error because we had not yet set the name variable in the workspace. But we now have configversion.json configured to use auto-queue-runs set to false. So, this run is no longer triggered.) 
+1. Uploads the myconfig.tar.gz file as a new configuration. (This step used to trigger an initial run which caused an error because we had not yet set the name variable in the workspace. But we now have configversion.json configured to use auto-queue-runs set to false. So, this run is no longer triggered.)
 1. Adds one Terraform variable called "name" and one Environment variable called "CONFIRM_DESTROY" to the workspace, getting their values from the variables.csv file. You can edit this file to add as many variables as you want.
+1. Determines the number of Sentinel policies.
 1. Starts a new run.
 1. Enters a loop to check the run results periodically.
+    - If $run_status is "planned", $is_confirmable is "True", and $override is "no", the script stops. In this case, no Sentinel policies existed or none of them were applicable to this workspace. The script will stop.  The user should can apply the run in the Terraform Enterprise UI.
+    - If $run_status is "planned", $is_confirmable is "True", and $override is "yes", the script will do an apply. As in the previous case, no Sentinel policies existed or none of them were applicable to this workspace.
     - If $run_status is "policy_checked", it does an Apply. In this case, all Sentinel policies passed.
     - If $run_status is "policy_override" and $override is "yes", it overrides the failed policy checks and does an Apply. In this case, one or more Sentinel policies failed, but they were marked "advisory" or "soft-mandatory" and the script was configured to override the failure.
     - If $run_status is "policy_override" and $override is "no", it prints out a message indicating that some policies failed and are not being overridden.
     - If $run_status is "errored", either the plan failed or a Sentinel policy marked "hard-mandatory" failed. The script terminates.
+    - Other values of $run_status cause the loop to repeat after a brief sleep.
 
 Note that some json template files are included from which other json files are generated so that they can be passed to the curl commands.
 
@@ -36,6 +40,12 @@ Do the following before using this script:
 1. `cd operations/automation-script`
 1. Make sure [python](https://www.python.org/downloads/) is installed on your machine and in your path since the script uses python to parse JSON documents returned by the Terraform Enterprise REST API.
 
+## Using with Private Terraform Enteprise Server using private CA
+If you use this script with a Private Terraform Enterprise (PTFE) server that uses a private CA instead of a public CA, you will need to ensure that the curl commands run by the script will trust the private CA.  There are several ways to do this.  The first is easiest for enabling the automation script to run, but it only affects curl. The second and third are useful for using the Terraform and TFE CLIs against your PTFE server. The third is a permanent solution.
+1. `export  CURL_CA_BUNDLE=<path_to_ca_bundle>`
+1. Export the Golang SSL_CERT_FILE and/or SSL_CERT_DIR environment variables. For instance, you could set the first of these to the same CA bundle used in option 1.
+1. Copy your certificate bundle to /etc/pki/ca-trust/source/anchors and then run `update-ca-trust extract`.
+
 ## Instructions
 Follow these instructions to run the script with the included main.tf and variables.csv files:
 
@@ -45,7 +55,7 @@ Follow these instructions to run the script with the included main.tf and variab
 1. `export ATLAS_TOKEN=<owners_token>` where \<owners_token\> is the token generated in the previous step.
 1. If you want, you can also change the name of the workspace that will be created and the sleep_duration variable which controls how often the script checks the status of the triggered run (in seconds).
 1. Edit variables.csv to specify the name you would like to set the name variable to by replacing "Roger" with some other name.
-1. Run `./loadAndRunWorkspace.sh` or `./loadAndRunWorkspace.sh <override>` where \<override\> is "yes" or "no". If you do not specify a value for \<override\>, the script will set it to "no".
+1. Run `./loadAndRunWorkspace.sh` or `./loadAndRunWorkspace.sh <override>` where \<override\> is "yes" or "no". If you do not specify a value for \<override\>, the script will set it to "no". The override variable is used in two ways: a) to automatically do an apply when no Sentinel policies exist or none of them are applicable to the workspace, and b) to override any soft-mandatory Sentinel policies that failed.
 
 ### Examples
 `./loadAndRunWorkspace` (no override will be done)
@@ -55,7 +65,7 @@ Follow these instructions to run the script with the included main.tf and variab
 `./loadAndRunWorkspace no` (no override will be done)
 
 ### Running with other Terraform code
-If you would like to load other Terraform code into a workspace with the script, replace main.tf in the config directory with your own Terraform code.  All files in the config directory will be uploaded to your TFE server.  Also edit variables.csv to remove the first row with the name variable and add rows for any Terraform and Environment variables that are required by your Terraform code.   
+If you would like to load other Terraform code into a workspace with the script, replace main.tf in the config directory with your own Terraform code.  All files in the config directory will be uploaded to your TFE server.  Also edit variables.csv to remove the first row with the name variable and add rows for any Terraform and Environment variables that are required by your Terraform code.  If your code has a terraform.tfvars file, please rename it to terraform.auto.tfvars since TFE overwrites any instance of terraform.tfvars with the variables set in the workspace. Adding variables already in a `*.auto.tfvars` file is not strictly necessary, but is recommended so that users looking at the workspace can see the values set on the variables.
 
 ## Cleaning Up
 If you want to run the script again, delete the workspace from the Settings tab of the workspace in the TFE UI. You do not need to delete or touch any of the files in the directory containing the script and other files.

operations/automation-script/loadAndRunWorkspace.sh

@@ -11,7 +11,7 @@ organization="<your_organization>"
 workspace="workspace-from-api"
 
 # You can change sleep duration if desired
-sleep_duration=15
+sleep_duration=5
 
 # Override soft-mandatory policy checks that fail.
 # Set to "yes" or "no" in second argument passed to script.
@@ -59,6 +59,11 @@ do
   upload_variable_result=$(curl --header "Authorization: Bearer $ATLAS_TOKEN" --header "Content-Type: application/vnd.api+json" --data @variable.json "https://${address}/api/v2/vars?filter%5Borganization%5D%5Bname%5D=${organization}&filter%5Bworkspace%5D%5Bname%5D=${workspace}")
 done < variables.csv
 
+# List Sentinel Policies
+sentinel_list_result=$(curl --header "Authorization: Bearer $ATLAS_TOKEN" --header "Content-Type: application/vnd.api+json" "https://${address}/api/v2/organizations/${organization}/policies")
+sentinel_policy_count=$(echo $sentinel_list_result | python -c "import sys, json; print(json.load(sys.stdin)['meta']['pagination']['total-count'])")
+echo "Number of Sentinel policies: " $sentinel_policy_count
+
 # Do a run
 sed "s/workspace_id/$workspace_id/" < run.template.json  > run.json
 run_result=$(curl --header "Authorization: Bearer $ATLAS_TOKEN" --header "Content-Type: application/vnd.api+json" --data @run.json https://${address}/api/v2/runs)
@@ -77,13 +82,31 @@ while [ $continue -ne 0 ]; do
   # Check the status of run
   check_result=$(curl --header "Authorization: Bearer $ATLAS_TOKEN" --header "Content-Type: application/vnd.api+json" https://${address}/api/v2/runs/${run_id})
 
-  # Parse out the run status
+  # Parse out the run status and is-confirmable
   run_status=$(echo $check_result | python -c "import sys, json; print(json.load(sys.stdin)['data']['attributes']['status'])")
   echo "Run Status: " $run_status
+  is_confirmable=$(echo $check_result | python -c "import sys, json; print(json.load(sys.stdin)['data']['attributes']['actions']['is-confirmable'])")
+  echo "Run can be applied: " $is_confirmable
 
   # Apply in some cases
+
+  # planned means plan finished and no Sentinel policies
+  # exist or are applicable to the workspace
+  if [[ "$run_status" == "planned" ]] && [[ "$is_confirmable" == "True" ]] && [[ "$override" == "no" ]] ; then
+    continue=0
+    echo "There are " $sentinel_policy_count "policies, but none of them are applicable to this workspace."
+    echo "Check the run in Terraform Enterprise UI and apply there if desired."
+  # planned means plan finished and no Sentinel policies
+  # exist or are applicable to the workspace
+  elif [[ "$run_status" == "planned" ]] && [[ "$is_confirmable" == "True" ]] && [[ "$override" == "yes" ]] ; then
+      continue=0
+      echo "There are " $sentinel_policy_count "policies, but none of them are applicable to this workspace."
+      echo "Since override was set to \"yes\", we are applying."
+      # Do the apply
+      echo "Doing Apply"
+      apply_result=$(curl --header "Authorization: Bearer $ATLAS_TOKEN" --header "Content-Type: application/vnd.api+json" --data @apply.json https://${address}/api/v2/runs/${run_id}/actions/apply)
   # policy_checked means all Sentinel policies passed
-  if [[ "$run_status" == "policy_checked" ]] ; then
+  elif [[ "$run_status" == "policy_checked" ]] ; then
     continue=0
     # Do the apply
     echo "Policies passed. Doing Apply"
@@ -118,6 +141,6 @@ while [ $continue -ne 0 ]; do
     continue=0
   else
     # Sleep a bit and then check status again in next loop
-    sleep $sleep_duration
+    echo "We will sleep a bit and try again soon."
   fi
 done