Skip to content

Commit 034e295

Browse files
authored
Merge pull request circleci#7623 from circleci/DOCTEAM-590_db_sch-pipe-updates
Sch pipe updates
2 parents 8a50a55 + bdf397d commit 034e295

5 files changed

+72
-65
lines changed

jekyll/_cci2/inject-environment-variables-with-api.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ The example below triggers a pipeline with parameter values for `workingdir` and
2727
NOTE: In order to pass a parameter when triggering a pipeline via the API, the parameter must be <<reusing-config#using-the-parameters-declaration,declared in the configuration file>>.
2828

2929
```shell
30-
curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
30+
curl -u ${CIRCLE_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
3131
"parameters": {
3232
"workingdir": "./myspecialdir",
3333
"image-tag": "4.8.2"

jekyll/_cci2/migrate-scheduled-workflows-to-scheduled-pipelines.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ NOTE: **The deprecation of the scheduled workflows feature has been postponed**.
1818

1919
Migrating to scheduled pipelines from scheduled workflows will eliminate the following limitations of scheduled workflows:
2020

21-
- Cannot control the actor, so scheduled workflows can't use restricted contexts
21+
- Cannot control the actor (yourself, or the scheduling system), so scheduled workflows cannot use restricted contexts
2222
- Cannot control the interaction with auto-cancelling of pipelines
2323
- Cannot use scheduled workflows together with dynamic config without complex workarounds
2424
- Cannot change or cancel scheduled workflows on a branch without triggering a pipeline

jekyll/_cci2/pipeline-variables.md

Lines changed: 43 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,29 @@
11
---
22
layout: classic-docs
33
title: Pipeline values and parameters
4-
description: "Detailed information about pipeline parameters and values"
4+
description: "Information about pipeline parameters and values and how to use them."
55
categories: [getting-started]
6-
order: 1
76
contentTags:
87
platform:
98
- Cloud
109
- Server v4.x
1110
- Server v3.x
1211
---
1312

13+
## Introduction
14+
{: #introduction}
15+
1416
Pipeline values and parameters can be used to create reusable pipeline configurations.
1517

1618
* **Pipeline values** represent pipeline metadata that can be used throughout the configuration.
17-
* **Pipeline parameters** are typed pipeline variables that are declared in the `parameters` key at the top level of a configuration. Users can pass `parameters` into their pipelines when triggering a new run of a pipeline through the API.
19+
* **Pipeline parameters** are typed pipeline variables that are declared in the `parameters` key at the top level of a configuration. Users can pass `parameters` into their pipelines when triggering a new run of a pipeline through the API or web app.
1820

1921
## Pipeline values
2022
{: #pipeline-values }
2123

2224
Pipeline values are available to all pipeline configurations and can be used without previous declaration.
2325

24-
For a full list of values and built-in environment variables, see the [Project Values and Variables guide]({{site.baseurl}}/variables/#pipeline-values).
26+
For a full list of values and built-in environment variables, see the [Project values and variables](/docs/variables/#pipeline-values) guide.
2527

2628
{% include snippets/pipeline-values.md %}
2729

@@ -45,28 +47,28 @@ jobs:
4547
- run: echo $CIRCLE_COMPARE_URL
4648
```
4749
48-
When using the above method to set the values in the `environment` key, note that if the pipeline variable is empty it will be set to `<nil>`. If you need an empty string instead, [set the variable in a shell command]({{ site.baseurl }}/set-environment-variable/#set-an-environment-variable-in-a-shell-command).
50+
When using the above method to set the values in the `environment` key, note that if the pipeline variable is empty it will be set to `<nil>`. If you need an empty string instead, [set the variable in a shell command](/docs/set-environment-variable/#set-an-environment-variable-in-a-shell-command).
4951
{: class="alert alert-info" }
5052

5153
## Pipeline parameters in configuration
5254
{: #pipeline-parameters-in-configuration }
5355

54-
Pipeline parameters are declared using the `parameters` key at the top level of a `.circleci/config.yml` file.
56+
Pipeline parameters are declared using the `parameters` key at the top level of a `.circleci/config.yml` file. Pipeline parameters can be referenced by value and used as a configuration variable under the scope `pipeline.parameters`.
5557

5658
Pipeline parameters support the following types:
59+
5760
* string
5861
* boolean
5962
* integer
6063
* enum
6164

62-
See [Parameter Syntax]({{ site.baseurl }}/reusing-config/#parameter-syntax) for usage details.
63-
64-
Pipeline parameters can be referenced by value and used as a config variable under the scope `pipeline.parameters`.
65+
See [Parameter syntax](/docs/reusing-config/#parameter-syntax) for usage details.
6566

66-
The example below shows a configuration with two pipeline parameters (`image-tag` and `workingdir`) defined at the top of the config, and then subsequently referenced in the `build` job:
67+
The example below shows a configuration with two pipeline parameters (`image-tag` and `workingdir`) defined at the top of the configuration, and then subsequently referenced in the `build` job:
6768

6869
```yaml
6970
version: 2.1
71+
7072
parameters:
7173
image-tag:
7274
type: string
@@ -95,12 +97,13 @@ jobs:
9597

9698
A pipeline can be triggered with specific `parameter` values using the API v2 endpoint to [trigger a pipeline](https://circleci.com/docs/api/v2/#trigger-a-new-pipeline). This can be done by passing a `parameters` key in the JSON packet of the `POST` body.
9799

98-
**Note:** Please note that the `parameters` key passed in this `POST` request is **NOT** secret.
100+
The `parameters` key passed in this `POST` request is **NOT** secret.
101+
{: class="alert alert-warning" }
99102

100-
The example below triggers a pipeline with the parameters described in the above config example (NOTE: To pass a parameter when triggering a pipeline via the API the parameter must be declared in the configuration file.).
103+
The example below triggers a pipeline with the parameters described in the above configuration example. If you run a command to trigger a pipeline, and the parameter has not been declared in the configuration file, you will receive an error response message, such as `Project not found`.
101104

102105
```shell
103-
curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
106+
curl -u ${CIRCLE_TOKEN}: -X POST --header "Content-Type: application/json" -d '{
104107
"parameters": {
105108
"workingdir": "./myspecialdir",
106109
"image-tag": "4.8.2"
@@ -111,29 +114,18 @@ curl -u ${CIRCLECI_TOKEN}: -X POST --header "Content-Type: application/json" -d
111114
### Passing parameters when triggering pipelines using the CircleCI web app
112115
{: #passing-parameters-when-triggering-pipelines-using-the-circleci-web-app }
113116

114-
In addition to using the CLI and API, you can also trigger a pipeline with parameters from the CircleCI web app. To do this:
115-
116-
1. Navigate to the dashboard view in the web app.
117-
2. Use the project filter to select the desired project.
118-
3. Use the branch filter to select the branch on which you want to run the new pipeline.
119-
4. Click the **Trigger Pipeline** button (towards the top right corner of the page).
120-
5. Use the **Add Parameters** dropdown to specify the type, name, and value of your desired parameters.
121-
6. Click **Trigger Pipeline**.
117+
In addition to using the API, you can also trigger a pipeline with parameters from the CircleCI web app. If you pass a parameter when triggering a pipeline from the web app, and the parameter has not been declared in the configuration file, the pipeline will fail with the error `Unexpected argument(s)`.
122118

123-
**NOTE:** If you pass a parameter when triggering a pipeline from the web app, and the parameter has not been declared in the configuration file, the pipeline will fail with the error `Unexpected argument(s)`)
124-
125-
126-
## The scope of pipeline parameters
127-
{: #the-scope-of-pipeline-parameters }
119+
1. Use the project filter to select the desired project.
120+
2. Use the branch filter to select the branch on which you want to run the new pipeline.
121+
3. Click the **Trigger Pipeline** button (towards the top right corner of the page).
122+
4. Use the **Add Parameters** dropdown to specify the type, name, and value of your desired parameters.
123+
5. Click **Trigger Pipeline**.
128124

129-
Pipeline parameters can only be resolved in the `.circleci/config.yml` file in which they are declared. Pipeline parameters are not available in orbs, including orbs declared locally in your config.yml file. This was done because access to the pipeline scope in orbs would break encapsulation and create a hard dependency between the orb and the calling config, potentially jeopardizing determinism and creating a surface area of vulnerability.
125+
Parameters can also be called when setting up a scheduled pipeline in the web app. The parameters are part of the trigger form in **Project Settings > Triggers**. Any parameter set up as a part of a scheduled pipeline will also need to be declared in the configuration file, otherwise the the pipeline will fail with the error `Unexpected argument(s)`.
130126

131-
132-
## Config processing stages and parameter scopes
133-
{: #config-processing-stages-and-parameter-scopes }
134-
135-
### Processing stages
136-
{: #processing-stages }
127+
## Configuration processing stages
128+
{: #configuration-processing-stages }
137129

138130
Configuration processing happens in the following stages:
139131

@@ -143,10 +135,15 @@ Configuration processing happens in the following stages:
143135

144136
The remaining configuration is processed, element parameters are resolved, type-checked, and substituted.
145137

146-
## Element parameter scope
138+
## The scope of pipeline parameters
139+
{: #the-scope-of-pipeline-parameters }
140+
141+
Pipeline parameters can only be resolved in the `.circleci/config.yml` file in which they are declared. Pipeline parameters are not available in orbs, including orbs declared locally in your `.circleci/config.yml` file. This is because access to pipeline scope in orbs would break encapsulation and create a hard dependency between the orb and the calling configuration. This would potentially create a surface area of vulnerability, increasing security risks.
142+
143+
### Element parameter scope
147144
{: #element-parameter-scope }
148145

149-
Element parameters use lexical scoping, so parameters are in scope within the element they are defined in, e.g. a job, a command, or an executor. If an element with parameters calls another element with parameters, like in the example below, the inner element does not inherit the scope of the calling element.
146+
Element parameters use lexical scoping, so parameters are in scope within the element they are defined in, for example, a job, a command, or an executor. If an element with parameters calls another element with parameters, like in the example below, the inner element does not inherit the scope of the calling element.
150147

151148
```yaml
152149
version: 2.1
@@ -160,25 +157,25 @@ commands:
160157
- run: echo << parameters.message >>
161158
162159
jobs:
163-
cat-file:
160+
daily-message:
161+
machine: true
164162
parameters:
165-
file:
163+
message:
166164
type: string
167165
steps:
168166
- print:
169-
message: Printing << parameters.file >>
170-
- run: cat << parameters.file >>
167+
message: Printing << parameters.message >>
171168
172169
workflows:
173170
my-workflow:
174171
jobs:
175-
- cat-file:
176-
file: test.txt
172+
- daily-message:
173+
message: echo << parameters.message >>
177174
```
178175

179-
Even though the `print` command is called from the cat-file job, the file parameter would not be in scope inside the print. This ensures that all parameters are always bound to a valid value, and the set of available parameters is always known.
176+
Even though the `print` command is called from the `cat-file` job, the file parameter would not be in scope inside the print job. This ensures that all parameters are always bound to a valid value, and the set of available parameters is always known. Running this would throw a pipeline error of `Arguments referenced without declared parameters: message`.
180177

181-
## Pipeline value scope
178+
### Pipeline value scope
182179
{: #pipeline-value-scope }
183180

184181
Pipeline values, the pipeline-wide values that are provided by CircleCI (e.g. `<< pipeline.number >>`) are always in scope.
@@ -194,11 +191,9 @@ Pipeline parameters which are defined in configuration are always in scope, with
194191
## Conditional workflows
195192
{: #conditional-workflows }
196193

197-
Use the [`when` clause]({{site.baseurl}}/configuration-reference/#using-when-in-workflows) (or the inverse clause `unless`) under a workflow declaration, along with a [logic statement]({{site.baseurl}}/configuration-reference/#logic-statements), to decide whether or not to run that workflow. Logic statements in a `when` or `unless` clause should evaluate to a truthy or falsy value.
198-
199-
The most common use of this construct is to use a pipeline parameter as the value, allowing an API trigger to pass that parameter to determine which workflows to run.
194+
Use the [`when` clause](/docs/configuration-reference/#using-when-in-workflows) (or the inverse clause `unless`) under a workflow declaration, along with a [logic statement](/docs/configuration-reference/#logic-statements), to decide whether or not to run that workflow. Logic statements in a `when` or `unless` clause should evaluate to a truthy or falsy value.
200195

201-
Below is an example configuration using the pipeline parameter `run_integration_tests` to drive whether the workflow `integration_tests` will run.
196+
The most common use of this construct is to use a pipeline parameter as the value, allowing a trigger to pass that parameter to determine which workflows to run. Below is an example configuration using the pipeline parameter `run_integration_tests` to set whether the workflow `integration_tests` will run.
202197

203198
```yaml
204199
version: 2.1
@@ -209,7 +204,6 @@ parameters:
209204
default: false
210205
211206
workflows:
212-
version: 2
213207
integration_tests:
214208
when: << pipeline.parameters.run_integration_tests >>
215209
jobs:
@@ -222,7 +216,7 @@ jobs:
222216
- ... # job steps
223217
```
224218

225-
The example shown above prevents the workflow `integration_tests` from being triggered unless it is explicitly invoked when the pipeline is triggered with the following in the `POST` body:
219+
In this example, the workflow `integration_tests` is not triggered unless it is explicitly invoked when the pipeline is triggered with the following in the `POST` body:
226220

227221
```json
228222
{

jekyll/_cci2/scheduled-pipelines.md

Lines changed: 26 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -8,27 +8,35 @@ contentTags:
88
- Cloud
99
---
1010

11-
## Introduction
12-
{: #introduction }
13-
1411
**Scheduled pipelines are currently available for GitHub and Bitbucket VCS users.** Scheduled pipelines allow you to trigger pipelines periodically based on a schedule. Scheduled pipelines retain all the features of pipelines:
1512

16-
- Control the actor associated with the pipeline, which can enable the use of [restricted contexts](/docs/contexts/#project-restrictions)
17-
- Use [dynamic config](/docs/dynamic-config) via setup workflows
18-
- Modify the schedule without having to edit `.circleci/config.yml`
19-
- Take advantage of [auto-cancelling](/docs/skip-build/#auto-cancelling)
20-
- Specify [pipeline parameters](/docs/pipeline-variables/#pipeline-parameters-in-configuration) associated with a schedule
21-
- Manage common schedules, for example, across workflows
13+
- Control the actor (yourself, or the scheduling system) associated with the pipeline, which can enable the use of [restricted contexts](/docs/contexts/#project-restrictions).
14+
- Use [dynamic config](/docs/dynamic-config) via setup workflows.
15+
- Modify the schedule without having to edit `.circleci/config.yml`.
16+
- Take advantage of [auto-cancelling](/docs/skip-build/#auto-cancelling).
17+
- Specify [pipeline parameters](/docs/pipeline-variables/#pipeline-parameters-in-configuration) associated with a schedule.
18+
- Manage common schedules, for example, across workflows.
2219

2320
Scheduled pipelines are configured through the API, or through the project settings in the CircleCI web app.
2421

2522
A scheduled pipeline can only be configured for one branch. If you need to schedule for two branches, you would need to set up two schedules.
2623
{: class="alert alert-info"}
2724

25+
## Introduction
26+
{: #introduction }
27+
28+
Scheduled pipelines allow you to trigger pipelines periodically based on a schedule, from either the CircleCI web app or API. Schedules can range from daily, weekly, monthly, or on a very specific timetable. To set up basic scheduled pipelines, you do not need any extra configuration in your `.circleci/config.yml` file, however, more advanced usage of the feature will require extra `.circleci/config.yml` configuration (for example, workflow filtering, or using parameters).
29+
30+
Pipeline parameters are typed pipeline variables in the form of a string, integer, or boolean. Adding a parameter to a scheduled pipeline can be done in the web app in the triggers form while setting up a schedule. Any parameters set up in this manner must be added to your configuration file using the `parameters` key.
31+
32+
Scheduled pipelines are set to run by an "actor", either the CircleCI scheduling system, or a specific user (for example, yourself). The scheduling actor is important to consider if making use of restricted contexts in workflows. If the user (actor) running the workflow does not have access to the context, the workflow will fail with the `Unauthorized` status.
33+
34+
You can find a basic how-to guide on the [Set a nightly scheduled pipeline](/docs/set-a-nightly-scheduled-pipeline) page, and more advanced examples on the [Schedule pipelines with multiple workflows](/docs/schedule-pipelines-with-multiple-workflows) pages.
35+
2836
## Get started with scheduled pipelines
2937
{: #get-started-with-scheduled-pipelines }
3038

31-
To get started with scheduled pipelines, you have the option of using the API, or using the CircleCI web app. Both methods are described below. If you have existing workflows you need to migrate to scheduled pipelines, use the [Scheduled pipelines migration](/docs/migrate-scheduled-workflows-to-scheduled-pipelines) guide.
39+
To get started with scheduled pipelines, you have the option of using the API, or using the CircleCI web app. Both methods are described below.
3240

3341
### Use project settings in the web app
3442
{: #use-project-settings }
@@ -38,7 +46,7 @@ To get started with scheduled pipelines, you have the option of using the API, o
3846
3. To create a new schedule, click **Add Trigger**.
3947
4. Define the new schedule by filling out the form, then click **Save Trigger**.
4048

41-
The form also provides the option of adding [pipeline parameters](/docs/pipeline-variables/), which are typed pipeline variables declared at the top level of a configuration.
49+
The form also provides the option of adding [pipeline parameters](/docs/pipeline-variables/), which are typed pipeline variables that you declare at the top level of a configuration.
4250

4351
If you would like to manage common schedules for multiple workflows, you will need to manually set this in your `.circleci/config.yml` file. See the [Schedule pipelines with multiple workflows](/docs/schedule-pipelines-with-multiple-workflows) page for examples.
4452

@@ -72,6 +80,11 @@ curl --location --request POST "https://circleci.com/api/v2/project/<project-slu
7280

7381
For additional information, refer to the **Schedule** section under the [API v2 docs](https://circleci.com/docs/api/v2).
7482

83+
## Migrate scheduled workflows to scheduled pipelines
84+
{: #migrate-scheduled-workflows-to-scheduled-pipelines }
85+
86+
If you have existing scheduled workflows you need to migrate to scheduled pipelines, use the [Scheduled pipelines migration](/docs/migrate-scheduled-workflows-to-scheduled-pipelines) guide.
87+
7588
## Scheduled pipelines video tutorial
7689
{: #scheduled-pipelines-video-tutorial }
7790

@@ -107,14 +120,14 @@ curl --location --request GET "https://circleci.com/api/v2/project/<project-slug
107120
--header "circle-token: <PERSONAL_API_KEY>"
108121
```
109122

110-
For GitHub and Bitbucket users: `project-slug` takes the form of `vcs-type/org-name/repo-name`, e.g. `gh/CircleCI-Public/api-preview-docs`.
123+
For GitHub and Bitbucket users: `project-slug` takes the form of `vcs-type/org-name/repo-name`, for example, `gh/CircleCI-Public/api-preview-docs`.
111124

112125
---
113126

114127
**Q:** Why is my scheduled pipeline not running?
115128

116129
**A:** There could be a few possible reasons:
117-
* Is the actor who is set for the scheduled pipelines still part of the organization?
130+
* Is the assigned actor who is set for the scheduled pipelines still part of the organization (you can find this setting is under **Attribution** in the **Triggers** section of the web app)?
118131
* Is the branch set for the schedule deleted?
119132
* Is your GitHub organization using SAML protection? SAML tokens expire often, which can cause requests to GitHub to fail.
120133

0 commit comments

Comments
 (0)