Skip to content

Commit 3e0d4dc

Browse files
committed
docs(metrics): mark GuardianCheck deprecated and document Intrinsics telemetry gap
Guardian Intrinsics are not Requirement subclasses and emit no mellea.requirement.checks/failures metrics. Users migrating from GuardianCheck would otherwise lose those counters silently. Also fix "Determine is" → "Determine if" typo in factuality_detection docstring. Assisted-by: Claude Code
1 parent 633155f commit 3e0d4dc

2 files changed

Lines changed: 10 additions & 2 deletions

File tree

docs/docs/observability/metrics.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,9 +246,17 @@ All sampling metrics include:
246246

247247
| Attribute | Description | Example Values |
248248
| --------- | ----------- | -------------- |
249-
| `requirement` | Requirement class name | `LLMaJRequirement`, `PythonExecutionReq`, `ALoraRequirement`, `GuardianCheck` |
249+
| `requirement` | Requirement class name | `LLMaJRequirement`, `PythonExecutionReq`, `ALoraRequirement`, `GuardianCheck` *(deprecated v0.4)* |
250250
| `reason` | Human-readable failure reason (`mellea.requirement.failures` only) | `"Output did not satisfy constraint"`, `"unknown"` |
251251

252+
> **Guardian Intrinsics and metrics:** `guardian_check()`, `policy_guardrails()`,
253+
> `factuality_detection()`, and `factuality_correction()` are not `Requirement`
254+
> subclasses and do not emit `mellea.requirement.checks` or
255+
> `mellea.requirement.failures` metrics. If you migrate from `GuardianCheck` to
256+
> Guardian Intrinsics, Guardian-related requirement counters will stop appearing
257+
> in your metrics. Wrap Guardian Intrinsic calls in a custom `Requirement` subclass
258+
> if you need to preserve this telemetry.
259+
252260
### Tool counter
253261

254262
| Metric Name | Type | Unit | Description |

mellea/stdlib/components/intrinsic/guardian.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ def guardian_check(
160160

161161

162162
def factuality_detection(context: ChatContext, backend: AdapterMixin) -> float:
163-
"""Determine is the last response is factually incorrect.
163+
"""Determine if the last response is factually incorrect.
164164
165165
Intrinsic function that evaluates the factuality of the
166166
assistant's response to a user's question. The context should end with

0 commit comments

Comments
 (0)