Skip to content

Commit ff8806e

Browse files
petetomasikpetetomasik
committed
Merge branch 'main' into SUP-5894-fix-notify-group-step
2 parents 5a62977 + c2feab3 commit ff8806e

8 files changed

Lines changed: 1407 additions & 35 deletions

File tree

‎README.md‎

Lines changed: 221 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,79 @@ This is a sub-section that provides configuration for running commands or trigge
7272
- [Group](https://buildkite.com/docs/pipelines/configure/step-types/group-step)
7373
- [Conditionals](https://buildkite.com/docs/pipelines/conditionals)
7474

75-
:warning: This plugin may accept configurations that are not valid pipeline steps, this is a known issue to keep its code simple and flexible.
75+
#### Step Validation
76+
77+
The plugin validates all step configurations before uploading the pipeline. Invalid steps are automatically skipped with a warning logged to the build output.
78+
79+
**A valid step must have:**
80+
- A `command` or `commands` field (for command steps), OR
81+
- A `trigger` field (for trigger steps), OR
82+
- A `group` field with either:
83+
- An action (`command`, `commands`, or `trigger`) directly on the group, OR
84+
- Valid nested `steps`
85+
86+
**Invalid configurations that will be skipped:**
87+
88+
```yaml
89+
# ❌ Empty step - no action defined
90+
- path: "app/"
91+
config:
92+
label: "Deploy app" # Only has a label, no command/trigger
93+
94+
# ❌ Empty group - no action and no nested steps
95+
- path: "services/"
96+
config:
97+
group: "Deploy"
98+
# Missing: steps array or action
99+
```
100+
101+
**Valid configurations:**
102+
103+
```yaml
104+
# ✅ Valid - has command
105+
- path: "app/"
106+
config:
107+
label: "Deploy app"
108+
command: "echo deploying"
109+
110+
# ✅ Valid - group with nested steps
111+
- path: "services/"
112+
config:
113+
group: "Deploy"
114+
steps:
115+
- command: "deploy.sh"
116+
```
117+
118+
#### Plugins in Step Configurations
119+
120+
The plugin preserves `plugins:` blocks when specified in command step configurations. This allows you to use Buildkite plugins within your monorepo-watched steps.
121+
122+
**Example**
123+
124+
```yaml
125+
steps:
126+
- label: "Triggering pipelines"
127+
plugins:
128+
- monorepo-diff#v1.8.0:
129+
watch:
130+
- path: services/api/
131+
config:
132+
command: "npm test"
133+
plugins:
134+
- artifacts#v1.9.4:
135+
upload: "coverage/**/*"
136+
- docker-compose#v5.12.1:
137+
run: api
138+
- path: services/web/
139+
config:
140+
command: "yarn build"
141+
plugins:
142+
- docker#v5.13.0:
143+
image: "node:20"
144+
workdir: /app
145+
```
146+
147+
When changes are detected in the watched paths, the plugin generates steps that include the specified plugins. The `plugins:` blocks are preserved exactly as configured.
76148

77149
```yaml
78150
steps:
@@ -89,9 +161,9 @@ steps:
89161
- path: docker/
90162
config:
91163
group: docker/**
92-
steps:
164+
steps: # Required: groups must have either 'steps' or an action
93165
- plugins:
94-
- docker#latest:
166+
- docker#v5.13.0:
95167
build: service
96168
push: service
97169
- command: docker/run-e2e-tests.sh
@@ -260,10 +332,89 @@ steps:
260332
build:
261333
message: "Deploying foo service"
262334
env:
263-
- HELLO=123
264-
- AWS_REGION
335+
HELLO: 123
336+
AWS_REGION: ~ # Null literal reads from $AWS_REGION
337+
```
338+
339+
### Environment Variables
340+
341+
Environment variables can be specified in two formats. Both formats are fully supported.
342+
343+
#### Map Format (Recommended)
344+
345+
The map format provides clean, readable syntax:
346+
347+
```yaml
348+
steps:
349+
- label: "Triggering pipelines"
350+
plugins:
351+
- monorepo-diff#v1.8.0:
352+
env:
353+
NODE_ENV: production
354+
API_URL: https://api.example.com
355+
PORT: 8080
356+
DEBUG: false
357+
AWS_REGION: ~ # Null literal reads from $AWS_REGION
358+
EMPTY_STRING: "" # Empty string sets to literal ""
359+
watch:
360+
- path: "services/"
361+
config:
362+
command: "npm test"
363+
env:
364+
TEST_ENV: integration
365+
MAX_WORKERS: 4
366+
```
367+
368+
Map format features:
369+
- Clean YAML syntax using key-value pairs
370+
- Supports non-string values (numbers, booleans) which are converted to strings automatically
371+
- Null values read from OS environment: use `KEY: ~` (recommended YAML null literal)
372+
- The explicit `~` ensures nothing is accidentally added during pipeline processing
373+
- Note: Unlike array format, you cannot use just `KEY` alone - you must use a null value
374+
- Empty string (`""`) is treated as a literal empty string value
375+
- Whitespace in values is preserved
376+
- Recommended for new configurations
377+
378+
#### Array Format (Fully Supported)
379+
380+
The array format uses key=value syntax:
381+
382+
```yaml
383+
steps:
384+
- label: "Triggering pipelines"
385+
plugins:
386+
- monorepo-diff#v1.8.0:
387+
env:
388+
- NODE_ENV=production
389+
- API_URL=https://api.example.com
390+
- AWS_REGION # Key-only reads from $AWS_REGION
391+
watch:
392+
- path: "services/"
393+
config:
394+
command: "npm test"
395+
env:
396+
- TEST_ENV=integration
265397
```
266398

399+
Array format features:
400+
- Key-only entries (e.g., `AWS_REGION`) read from OS environment variables
401+
- Supports values with equals signs: `BUILD_ARGS=--arg1=val1`
402+
- Whitespace trimmed from keys and values automatically
403+
- Fully supported alongside map format
404+
405+
#### Format Comparison
406+
407+
| Feature | Map Format | Array Format |
408+
|---------|-----------|--------------|
409+
| Syntax | `KEY: value` | `KEY=value` |
410+
| OS env reading | Null literal (`KEY: ~`) | Key-only entries (`KEY`) |
411+
| Empty string | `KEY: ""` sets to `""` | `KEY=` sets to `""` |
412+
| Type support | Numbers, booleans | Strings only |
413+
| Whitespace | Preserved in values | Trimmed |
414+
| Readability | High | Medium |
415+
416+
**Note:** The format is determined by YAML structure - you cannot mix array and map syntax at the same level. However, you can use different formats at different levels (e.g., map format at plugin level, array format at step level).
417+
267418
### `log_level` (optional)
268419

269420
Add `log_level` property to set the log level. Supported log levels are `debug` and `info`. Defaults to `info`.
@@ -309,6 +460,34 @@ steps:
309460

310461
The plugin automatically retries binary downloads up to 3 times with a 5-second delay between attempts. This handles transient network issues when downloading from GitHub.
311462

463+
### `verify_checksum` (optional)
464+
465+
Default: `false`
466+
467+
Enable SHA256 checksum verification for downloaded binaries to enhance security. When enabled, the plugin verifies checksums against those published in the GitHub release, providing protection against compromised artifacts, network attacks, and binary tampering.
468+
469+
Checksum verification is performed for:
470+
- Newly downloaded binaries (fails and deletes binary on mismatch)
471+
- Cached binaries before reuse (automatically re-downloads on mismatch)
472+
- Pre-installed binaries when `download: false` (best-effort, non-blocking)
473+
474+
To enable checksum verification:
475+
476+
```yaml
477+
steps:
478+
- label: "Triggering pipelines"
479+
plugins:
480+
- monorepo-diff#v1.8.0:
481+
verify_checksum: true # Recommended for enhanced security
482+
diff: "git diff --name-only HEAD~1"
483+
watch:
484+
- path: "foo-service/"
485+
config:
486+
trigger: "deploy-foo-service"
487+
```
488+
489+
If checksums are unavailable for a release or the SHA256 command is not found on the system, the plugin will warn but continue execution (graceful degradation).
490+
312491
### `hooks` (optional)
313492

314493
Currently supports a list of `commands` you wish to execute after the `watched` pipelines have been triggered
@@ -413,7 +592,7 @@ steps:
413592
diff: "git diff --name-only $(head -n 1 last_successful_build)"
414593
interpolation: false
415594
env:
416-
- env1=env-1 # this will be appended to all env configuration
595+
env1: env-1 # this will be appended to all env configuration
417596
hooks:
418597
- command: "echo $(git rev-parse HEAD) > last_successful_build"
419598
watch:
@@ -444,7 +623,7 @@ steps:
444623
artifacts:
445624
- "logs/*"
446625
env:
447-
- FOO=bar
626+
FOO: bar
448627
449628
wait: true
450629
```
@@ -489,6 +668,41 @@ steps:
489668
command: "echo deploy-bar"
490669
```
491670

671+
## Troubleshooting
672+
673+
### "Skipping invalid step" warnings
674+
675+
If you see warnings like `Skipping invalid step: empty step configuration`, check that your step configuration includes:
676+
677+
1. For command steps: `command` or `commands` field
678+
2. For trigger steps: `trigger` field
679+
3. For group steps: `group` field with either `steps` array or an action
680+
681+
**Common issues:**
682+
683+
- Forgetting to add `command:` or `trigger:` inside the `config` block
684+
- Creating empty groups without nested steps
685+
- Using only metadata fields like `label`, `key`, or `env` without an action
686+
687+
**Example of fixing an invalid configuration:**
688+
689+
```yaml
690+
# ❌ Invalid - missing action
691+
- path: "app/"
692+
config:
693+
label: "Deploy app"
694+
env:
695+
- ENV=production
696+
697+
# ✅ Fixed - added command
698+
- path: "app/"
699+
config:
700+
label: "Deploy app"
701+
command: "deploy.sh"
702+
env:
703+
- ENV=production
704+
```
705+
492706
## Compatibility
493707

494708
| Elastic Stack | Agent Stack K8s | Hosted (Mac) | Hosted (Linux) | Notes |

0 commit comments

Comments
 (0)