Skip to content

Commit a1828fb

Browse files
simusimu
authored
Merge pull request #194 from projectsyn/feat/reference-default-values
Implement default values for references
2 parents 55482eb + 73ef9e7 commit a1828fb

33 files changed

Lines changed: 1039 additions & 56 deletions

‎README-extensions.md‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# reclass-rs Extensions
2+
3+
Initially, reclass-rs only implemented features present in [kapicorp-reclass], including extensions introduced by kapicorp-reclass.
4+
This document covers extensions that are currently unique to reclass-rs.
5+
6+
## Non-compatible `compose_node_name` option
7+
8+
Reclass-rs supports the `compose_node_name` option.
9+
By default, in contrast to kapicorp-reclass, reclass-rs preserves literal dots in composed node names.
10+
For example, given the following inventory, the node's internal path is parsed as `['path', 'to', 'the.node']`:
11+
12+
```
13+
.
14+
├── classes
15+
│   ├── cls1.yml
16+
│   └── cls2.yml
17+
└── nodes
18+
└── path
19+
└── to
20+
└── the.node.yml
21+
```
22+
23+
However, optionally, reclass-rs can be configured to handle `compose_node_name` the same way that kapicorp-reclass does, by naively splitting node names on each dot.
24+
To enable this compatibility mode, set `compat_flags: ['ComposeNodeNameLiteralDots']` in your inventory's `reclass-config.yml`.
25+
In compatibility mode, the node's internal path for the previous inventory is `['path', 'to', 'the', 'node']`.
26+
27+
## Verbose warnings
28+
29+
Reclass-rs supports boolean config option `verbose_warnings`.
30+
When verbose warnings are enabled, reclass-rs produces the following informational messages on standard error:
31+
32+
* warnings when dropping unrendered values which could potentially contain missing references.
33+
* informational messages when replacing a missing reference with a default value.
34+
35+
## Default values for references
36+
37+
> [!IMPORTANT]
38+
> This feature is currently experimental and may change at any time.
39+
40+
Reclass-rs supports specifying default (fallback) values for references.
41+
The format for specifying a default value is
42+
43+
```
44+
${some:reference:path::the_default_value}
45+
```
46+
47+
We chose `::` as the separator between the reference path and the default value because `::` currently can't appear in references in valid kapicorp-reclass inventories (due to [kapicorp/kapitan#1171](https://github.com/kapicorp/kapitan/issues/1171)).
48+
49+
Reclass-rs first resolves nested references in the reference path and default value and then splits the reference contents into the path and default value.
50+
References with default values can be used everywhere that references are supported, including class includes.
51+
52+
> [!NOTE]
53+
> Currently, default values are only applied once all nested references have been successfully resolved.
54+
> A missing nested reference without its own default value will result in an error even if the top-level reference specifies a default value.
55+
56+
For further processing, the default value is parsed as YAML.
57+
Incomplete YAML flow values always raise an error (also for existing references that specify an incomplete YAML default value).
58+
59+
> [!NOTE]
60+
> While it's generally possible to specify arbitrarily complex YAML default values inline, we recommend specifying complex default values through a nested reference.
61+
> When providing complex default values inline, it may be necessary to carefully escape characters of the YAML value in order to ensure the reference parser keeps the YAML value intact.
62+
63+
### Example
64+
65+
```yaml
66+
# nodes/test.yml
67+
classes:
68+
- class1
69+
- class2
70+
71+
parameters:
72+
_base_directory: /tmp
73+
```
74+
75+
```yaml
76+
# classes/class1.yml
77+
parameters:
78+
helm_values:
79+
a: a
80+
b: b
81+
```
82+
83+
```yaml
84+
# classes/class2.yml
85+
parameters:
86+
_compile:
87+
jsonnet:
88+
- input_paths:
89+
- ${_base_directory}/foo.jsonnet
90+
type: jsonnet
91+
output_path: foo
92+
helm:
93+
- input_paths:
94+
- ${_base_directory}/charts/foo
95+
type: helm
96+
helm_values: ${helm_values}
97+
output_path: foo
98+
kustomize:
99+
- input_paths:
100+
- ${_base_directory}/kustomization.jsonnet
101+
type: jsonnet
102+
output_path: ${_base_directory}/kustomizations/foo
103+
- input_paths:
104+
- ${_base_directory}/kustomizations/foo/
105+
type: kustomize
106+
output_path: foo
107+
108+
# Fall back to jsonnet rendering if method isn't configured
109+
compile: ${_compile:${method::jsonnet}}
110+
```
111+
112+
The rendered parameters for node `test` shown above:
113+
114+
```yaml
115+
parameters:
116+
_reclass_:
117+
environment: base
118+
name:
119+
full: test
120+
parts:
121+
- test
122+
path: test
123+
short: test
124+
_base_directory: /tmp
125+
_compile:
126+
helm:
127+
- helm_values:
128+
a: a
129+
b: b
130+
input_paths:
131+
- /tmp/charts/foo
132+
output_path: foo
133+
type: helm
134+
jsonnet:
135+
- input_paths:
136+
- /tmp/foo.jsonnet
137+
output_path: foo
138+
type: jsonnet
139+
kustomize:
140+
- input_paths:
141+
- /tmp/kustomization.jsonnet
142+
output_path: /tmp/kustomizations/foo
143+
type: jsonnet
144+
- input_paths:
145+
- /tmp/kustomizations/foo/
146+
output_path: foo
147+
type: kustomize
148+
compile:
149+
- input_paths:
150+
- /tmp/foo.jsonnet
151+
output_path: foo
152+
type: jsonnet
153+
helm_values:
154+
a: a
155+
b: b
156+
```
157+
158+
> [!TIP]
159+
> This example can also be found as node `n9` and classes `component.defaults` and `component.component` in this repository's `tests/inventory-reference-default-values`.
160+
161+
[kapicorp-reclass]: https://github.com/kapicorp/reclass

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,14 @@ The implementation currently supports the following features of Kapicorp Reclass
3030
* reclass-rs uses the `fancy-regex` crate for regex patterns in `class_mappings`.
3131
The `fancy-regex` crate should support most regex patterns supported by Python.
3232

33+
Additionally the implementation introduces the following new features:
34+
35+
* Option `verbose_warnings` which generates additional informational messages on standard error
36+
* Default values for references
37+
38+
> [!TIP]
39+
> Documentation for Reclass extensions introduced by reclass-rs can be found at [README-extensions.md](./README-extensions.md).
40+
3341
The following Kapicorp Reclass features aren't supported:
3442

3543
* Inventory Queries

‎src/refs/mod.rs‎

Lines changed: 64 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,13 @@ impl ResolveState {
109109
)
110110
}
111111

112+
fn render_default_value_error(&self, refpath: &str, e: &anyhow::Error) -> anyhow::Error {
113+
let current_key = self.current_key();
114+
anyhow!(
115+
"Error parsing default value for reference '{refpath}' in parameter '{current_key}': {e}"
116+
)
117+
}
118+
112119
pub(crate) fn render_flattening_error(&self, msg: &str) -> anyhow::Error {
113120
let current_key = self.current_key();
114121
anyhow!("In {current_key}: {msg}")
@@ -207,7 +214,10 @@ impl Token {
207214
}
208215
// Construct flattened ref path by resolving any potential nested references in the
209216
// Ref's Vec<Token>.
210-
let path = interpolate_token_slice(parts, params, state, opts)?;
217+
let (path, default) = extract_path_and_default(
218+
interpolate_token_slice(parts, params, state, opts)?,
219+
"::",
220+
);
211221

212222
if state.seen_paths.contains(&path) {
213223
// we've already seen this reference, so we know there's a loop, and can abort
@@ -223,9 +233,18 @@ impl Token {
223233
let k0 = refpath_iter.next().unwrap();
224234
// v is the value which we update to point to the next value as we recursively
225235
// descend into the params Mapping
226-
let mut v = params
236+
let v = params
227237
.get(&k0.into())
228-
.ok_or_else(|| state.render_missing_key_error(&path, k0))?;
238+
.ok_or_else(|| state.render_missing_key_error(&path, k0));
239+
if let Some(verr) = v.as_ref().err()
240+
&& let Some(dv) = default
241+
{
242+
if opts.verbose_warnings {
243+
eprintln!("[INFO] Returning default value due to error: {verr}");
244+
}
245+
return dv.map_err(|e| state.render_default_value_error(&path, &e));
246+
}
247+
let mut v = v?;
229248

230249
// newv is used to hold temporary Values generated by interpolating v
231250
let mut newv;
@@ -249,9 +268,20 @@ impl Token {
249268
// trivial case: v is a Mapping, we can just lookup the next value based
250269
// on `key`.
251270
Value::Mapping(_) => {
252-
v = newv
271+
let nv = newv
253272
.get(&key.into())
254-
.ok_or_else(|| state.render_missing_key_error(&path, key))?;
273+
.ok_or_else(|| state.render_missing_key_error(&path, key));
274+
if let Some(verr) = nv.as_ref().err()
275+
&& let Some(dv) = default
276+
{
277+
if opts.verbose_warnings {
278+
eprintln!(
279+
"[INFO] Returning default value due to error: {verr}"
280+
);
281+
}
282+
return dv.map_err(|e| state.render_default_value_error(&path, &e));
283+
}
284+
v = nv?;
255285
}
256286
// Sequence lookups aren't supported by Python Reclass. We may implement
257287
// them in the future.
@@ -266,6 +296,9 @@ impl Token {
266296
"We should have rendered Value::String and Value::ValueList into some other variant"
267297
),
268298
// A lookup into any other Value variant is an error
299+
// NOTE(sg): We do not fall back to the default value (if any) here, since
300+
// references that try to recurse into a scalar value should always be
301+
// surfaced.
269302
_ => {
270303
return Err(state.render_lookup_error(
271304
&path,
@@ -384,6 +417,32 @@ fn interpolate_string_or_valuelist(
384417
}
385418
}
386419

420+
fn extract_path_and_default(
421+
path_with_default: String,
422+
sep: &str,
423+
) -> (String, Option<Result<Value>>) {
424+
if let Some((path, default)) = path_with_default.split_once(sep) {
425+
let default_value: Result<Value> = serde_yaml::from_str(default)
426+
.map(|v: serde_yaml::Value| {
427+
let v = Value::from(v);
428+
match v {
429+
// Convert parsed string default value into Value::Literal to avoid incorrect
430+
// additional reference resolution. We know that we've already resolved the
431+
// top-level nested references at the time we call `extract_path_and_default`.
432+
// NOTE(sg): We don't want to recursively replace `Value::String` with
433+
// `Value::Literal` so that default values which are references that expand to
434+
// complex values still get resolved correctly.
435+
Value::String(s) => Value::Literal(s),
436+
_ => v,
437+
}
438+
})
439+
.map_err(|e| anyhow!(format!("{e}")));
440+
(path.to_string(), Some(default_value))
441+
} else {
442+
(path_with_default, None)
443+
}
444+
}
445+
387446
#[derive(Debug)]
388447
/// Wraps errors generated when trying to parse a string which may contain Reclass references
389448
pub struct ParseError<'a> {

0 commit comments

Comments
 (0)