Validator
The Validator is the safety gate between plan generation and plan execution. Every plan — whether generated by the Planner, authored by a developer, or retrieved from the Plan Library — must pass validation before execution. No exceptions. This specification defines the validation pipeline, its checks, its output format, and its role as defense-in-depth behind the policy-aware Planner.
1. Design Goals
Section titled “1. Design Goals”- No invalid plan executes. The Validator is the last line of defense. If it passes a plan, the Execution Engine must be able to run it without structural failures.
- Structured, actionable failures. Validation failures are not generic errors. Each failure identifies the specific check that failed, the specific step or binding that caused it, and a human-readable explanation.
- Defense-in-depth. The Planner is policy-aware and should produce executable plans. The Validator is the safety net. Policy-denied plans from the Planner should be rare exceptions (race conditions, registry changes). The Validator catches them anyway.
- Deterministic. The same plan + the same registry/policy state = the same validation result. Always. No LLM involvement. No randomness.
- Fast. Validation should add minimal latency to the planning pipeline. Target: < 200ms for a 10-step plan.
- Cacheable for promoted plans. Promoted plans that haven’t changed can cache their validation result. Re-validation is only required when the plan, its capabilities, or its applicable policies change.
2. Validator Interface
Section titled “2. Validator Interface”2.1 Input
Section titled “2.1 Input”interface ValidationRequest { plan: PlanIR; // Full Plan IR (from Enrichment or direct authoring) user_context: UserContext; // Who is requesting execution requested_mode: ExecutionMode; // LIVE, DRY_RUN, or SIMULATED environment: string; // 'production', 'staging', 'development'}2.2 Output
Section titled “2.2 Output”interface ValidationResult { valid: boolean; plan_id: string; validation_id: string; // Unique ID for this validation run timestamp: string; duration_ms: number;
// If valid: these are populated validated_plan?: PlanIR; // The plan, unchanged (confirmation it passed) validation_hash?: string; // Hash of plan + registry state + policy state (for cache)
// If invalid: these are populated errors?: ValidationError[]; // Hard failures -- plan cannot execute warnings?: ValidationWarning[]; // Soft issues -- plan can execute but something is notable
// Always populated checks_performed: CheckResult[]; // Full audit of every check and its outcome}
interface ValidationError { check: ValidationCheck; // Which check failed severity: 'error'; step_index?: number; // Which step (if step-specific) field?: string; // Which field (if field-specific) message: string; // Human-readable explanation code: string; // Machine-readable error code suggestion?: string; // How to fix it (if known)}
interface ValidationWarning { check: ValidationCheck; severity: 'warning'; step_index?: number; field?: string; message: string; code: string;}
interface CheckResult { check: ValidationCheck; passed: boolean; duration_ms: number; details?: string; // Additional info (e.g., "3 capabilities checked, all available")}
type ValidationCheck = | 'SCHEMA_COMPATIBILITY' | 'CAPABILITY_AVAILABILITY' | 'CAPABILITY_VERSION' | 'POLICY_COMPLIANCE' | 'EFFECT_MODE_COMPATIBILITY' | 'BINDING_VALIDITY' | 'DEPENDENCY_GRAPH' | 'CONSTRAINT_SATISFACTION' | 'PLAN_STRUCTURE' | 'SUB_PLAN_DEPTH';3. Validation Pipeline
Section titled “3. Validation Pipeline”The Validator runs checks in a defined order. All checks run even if an earlier check fails — this ensures the developer gets a complete picture of all issues, not just the first one.
3.1 Check Order
Section titled “3.1 Check Order”Check 1: PLAN_STRUCTURE Verify the Plan IR is structurally valid JSON conforming to the Plan IR schema.
Check 2: CAPABILITY_AVAILABILITY Verify every capability referenced in the plan exists in the Capability Registry.
Check 3: CAPABILITY_VERSION Verify the pinned capability versions are still available and not quarantined.
Check 4: SCHEMA_COMPATIBILITY Verify data flow: each step's input bindings resolve to values whose types match the capability's input_schema.
Check 5: BINDING_VALIDITY Verify all binding path expressions are valid and reference existing steps, outputs, or intent inputs.
Check 6: DEPENDENCY_GRAPH Verify the depends_on graph is a valid DAG (no cycles). Verify all referenced step indices exist.
Check 7: EFFECT_MODE_COMPATIBILITY Verify the plan's aggregate effect class is permitted in the requested execution mode.
Check 8: POLICY_COMPLIANCE Run full pre-execution policy evaluation on the plan.
Check 9: CONSTRAINT_SATISFACTION Verify all capability-level constraints are satisfied (required_roles, rate limits, preconditions).
Check 10: SUB_PLAN_DEPTH If the plan invokes sub-plans, verify the total depth does not exceed the limit (default: 5).3.2 Check Details
Section titled “3.2 Check Details”Check 1: PLAN_STRUCTURE
Section titled “Check 1: PLAN_STRUCTURE”Validates the Plan IR document itself:
- Has all required top-level fields (
plan_id,name,version,steps,input_schema,output_schema,metadata). stepsis a non-empty array.- Each step has required fields (
step_index,name,capabilityorsub_plan,input_bindings). - No duplicate
step_indexvalues. step_indexvalues are sequential starting from 0.- Plan
versionis valid semver.
Error codes:
| Code | Meaning |
|---|---|
PLAN_MISSING_FIELD | Required field is absent |
PLAN_EMPTY_STEPS | Steps array is empty |
PLAN_DUPLICATE_INDEX | Two steps have the same step_index |
PLAN_INVALID_VERSION | Version string is not valid semver |
Check 2: CAPABILITY_AVAILABILITY
Section titled “Check 2: CAPABILITY_AVAILABILITY”For each step that references a capability:
- The capability name exists in the Capability Registry.
- The capability is not disabled or removed.
Error codes:
| Code | Meaning |
|---|---|
CAPABILITY_NOT_FOUND | Capability name is not registered |
CAPABILITY_DISABLED | Capability exists but is currently disabled |
Check 3: CAPABILITY_VERSION
Section titled “Check 3: CAPABILITY_VERSION”For each step:
- The pinned capability version (
capability.version) exists in the registry. - The version is not quarantined.
- If no version is pinned: resolve to the latest available version (and record which version was resolved in the check result).
Error codes:
| Code | Meaning |
|---|---|
VERSION_NOT_FOUND | The specific version is not registered |
VERSION_QUARANTINED | The version has been quarantined due to failures |
Check 4: SCHEMA_COMPATIBILITY
Section titled “Check 4: SCHEMA_COMPATIBILITY”For each step, validate that the resolved input values match the capability’s input_schema:
- For each input binding, trace the source (literal value, intent input, prior step output, named reference, transform result).
- Resolve the source’s type (from the source capability’s
output_schema, or from the plan’sinput_schemafor intent inputs). - Check structural compatibility: is the source type assignable to the target type? Uses the Type System’s compatibility rules (see Type System Specification).
This is the most complex check. It’s essentially a type checker for the plan’s data flow.
Compatibility rules:
| Source Type | Target Type | Compatible? |
|---|---|---|
string | string | Yes |
string (format: email) | string | Yes (format is a refinement) |
string | string (format: email) | Warning (source may not be a valid email) |
integer | number | Yes (integer is a subtype of number) |
number | integer | Warning (may lose precision) |
| Object with extra fields | Object (target) | Yes (extra fields are ignored) |
| Object missing required field | Object (target) | Error |
Error codes:
| Code | Meaning |
|---|---|
SCHEMA_TYPE_MISMATCH | Source type is incompatible with target type |
SCHEMA_MISSING_REQUIRED | A required input field has no binding |
SCHEMA_FORMAT_MISMATCH | Types match but format constraints may not (warning) |
Check 5: BINDING_VALIDITY
Section titled “Check 5: BINDING_VALIDITY”For each input binding in each step:
- Path expressions (
$.steps[N].out.field) reference valid step indices and output fields. - Named references (
$.named.key.field) reference steps with matchingoutput_name. - Intent input references (
$.inputs.field) match fields in the plan’sinput_schema. - Literal values (
$literal) are valid JSON. - Transform expressions (
$transform) reference valid transform functions.
Error codes:
| Code | Meaning |
|---|---|
BINDING_INVALID_PATH | Path expression references a non-existent step or field |
BINDING_MISSING_SOURCE | Binding references a step output that doesn’t exist |
BINDING_CIRCULAR | Binding creates a circular reference |
BINDING_INVALID_TRANSFORM | Transform function is not recognized |
Check 6: DEPENDENCY_GRAPH
Section titled “Check 6: DEPENDENCY_GRAPH”Analyze the depends_on fields across all steps:
- Build a directed graph from
depends_onrelationships. - Verify the graph is acyclic (DAG). A cycle means the plan cannot execute.
- Verify all step indices in
depends_onarrays reference existing steps. - Verify that data flow is consistent with dependencies (if step B reads step A’s output, step B must depend on step A).
v1 note: Even though v1 executes steps sequentially (ignoring depends_on for scheduling), the Validator still checks the dependency graph for structural correctness. This ensures plans are valid for future parallel execution and catches data flow inconsistencies.
Error codes:
| Code | Meaning |
|---|---|
DEPENDENCY_CYCLE | The depends_on graph contains a cycle |
DEPENDENCY_MISSING_STEP | depends_on references a non-existent step |
DEPENDENCY_MISSING_DATA_DEP | Step reads from a step it doesn’t depend on (would race in parallel execution) |
Check 7: EFFECT_MODE_COMPATIBILITY
Section titled “Check 7: EFFECT_MODE_COMPATIBILITY”Check the plan’s aggregate effect class against the requested execution mode:
| Aggregate Effect | SIMULATED | DRY_RUN | LIVE |
|---|---|---|---|
| READ | Allowed | Allowed | Allowed |
| WRITE_INTERNAL | Allowed (mocked) | Allowed (blocked at outbox) | Allowed |
| WRITE_EXTERNAL | Allowed (mocked) | Warning (should use sandbox) | Allowed |
| DESTRUCTIVE | Allowed (mocked) | Error (not permitted in DRY_RUN) | Allowed (with policy gates) |
The aggregate effect class is the highest effect class across all steps in the plan.
Error codes:
| Code | Meaning |
|---|---|
EFFECT_MODE_DENIED | Plan’s aggregate effect is not permitted in the requested mode |
Warning codes:
| Code | Meaning |
|---|---|
EFFECT_MODE_CAUTION | Plan has external effects in DRY_RUN (should use sandbox/test keys) |
Check 8: POLICY_COMPLIANCE
Section titled “Check 8: POLICY_COMPLIANCE”Run the Policy Engine’s pre-execution evaluation on the full plan:
- Collect all applicable policies (scoped by environment, tenant, user).
- Evaluate each policy’s conditions against the plan metadata (aggregate effect, tags, lifecycle state, step count).
- Aggregate results: any DENY stops the plan. GATEs are noted (they’ll trigger at execution time, but the plan itself is valid).
This is the defense-in-depth check. The Planner already performed per-capability policy pre-checks. This check evaluates plan-level policies that the Planner doesn’t see (e.g., “candidate plans must run in DRY_RUN first,” “plans with more than 5 steps require approval”).
Error codes:
| Code | Meaning |
|---|---|
POLICY_DENIED | A policy explicitly denies this plan |
POLICY_REQUIRES_MODE | A policy requires a different execution mode |
Info codes:
| Code | Meaning |
|---|---|
POLICY_GATE_PENDING | A policy will trigger a gate during execution (not a failure, informational) |
Check 9: CONSTRAINT_SATISFACTION
Section titled “Check 9: CONSTRAINT_SATISFACTION”For each capability in the plan, check its declared capability-level constraints:
- required_roles: Does the current user have at least one of the required roles?
- rate_limit: Has the user exceeded the rate limit for this capability? (Requires checking execution history.)
- preconditions: Are static preconditions met? (Dynamic preconditions are checked at execution time.)
Error codes:
| Code | Meaning |
|---|---|
CONSTRAINT_ROLE_MISSING | User lacks required role for a capability |
CONSTRAINT_RATE_EXCEEDED | User has exceeded the rate limit |
CONSTRAINT_PRECONDITION_FAILED | A static precondition is not met |
Note: Role constraint failures at validation time are unexpected — the Planner should have already filtered out capabilities the user can’t access. If this fires, it indicates a race condition (role changed between planning and validation) or a bug. The error message should note this.
Check 10: SUB_PLAN_DEPTH
Section titled “Check 10: SUB_PLAN_DEPTH”If the plan invokes sub-plans (via sub_plan field on steps):
- Count the maximum nesting depth.
- Verify it does not exceed the configured limit (default: 5).
- Recursively validate each sub-plan (all checks apply to sub-plans too).
Error codes:
| Code | Meaning |
|---|---|
SUB_PLAN_DEPTH_EXCEEDED | Nesting depth exceeds the limit |
SUB_PLAN_NOT_FOUND | Referenced sub-plan is not in the Plan Library |
4. Validation Caching
Section titled “4. Validation Caching”4.1 When to Cache
Section titled “4.1 When to Cache”Promoted plans with a stable validation result can be cached. The cache is invalidated when:
| Change | Cache Action |
|---|---|
| Plan version changes | Invalidate |
| Any referenced capability version changes | Invalidate |
| Any applicable policy changes (add, modify, delete) | Invalidate |
| User roles change | Invalidate (role-dependent checks) |
| Registry state changes (capability added/removed) | Invalidate affected plans |
| Environment or execution mode changes | Invalidate (different mode = different check results) |
4.2 Cache Key
Section titled “4.2 Cache Key”cache_key = SHA-256( plan_id + plan_version + sorted(capability_name:version for each step) + sorted(applicable_policy_id:version) + user_roles_hash + environment + requested_mode)4.3 Cache Scope
Section titled “4.3 Cache Scope”- Per-user caching: Since role checks are user-specific, each user gets their own cache entry for a plan.
- Cache TTL: Configurable. Default: 5 minutes. Short enough to catch policy changes, long enough to avoid repeated validation for the same user submitting the same intent.
5. Validation in Different Contexts
Section titled “5. Validation in Different Contexts”5.1 Planner-Generated Plans
Section titled “5.1 Planner-Generated Plans”Planner → Plan Draft → Plan Enrichment → Full Plan IR → Validator → Execution EngineThe most common path. The Planner already performed policy pre-checks, so the Validator is defense-in-depth. Most validation failures here are schema compatibility issues (if the Planner selected an incompatible capability combination) or race conditions.
5.2 Developer-Authored Plans
Section titled “5.2 Developer-Authored Plans”Developer writes Full Plan IR directly → Validator → Plan Library (or Execution Engine)For hand-authored plans, validation is the primary quality gate. Developers may submit plans that have schema errors, missing bindings, or policy violations. The Validator’s structured errors guide the developer to fix them.
The CLI provides a zakardo plan validate command that runs the Validator locally during development.
5.3 Plan Library Retrieval
Section titled “5.3 Plan Library Retrieval”Plan Library → Full Plan IR (promoted) → Validator (cached?) → Execution EnginePromoted plans retrieved from the Plan Library are re-validated before execution. This catches:
- Policy changes since the plan was promoted
- Capability deprecation or quarantine since the plan was promoted
- User role changes (the promoting user may have had different roles than the current user)
If the validation result is cached and the cache key matches, validation is skipped (cache hit).
5.4 Training Runs
Section titled “5.4 Training Runs”Training System → Plan IR → Validator (in TRAINING mode) → Execution Engine (TRAINING mode)Training runs validate plans with requested_mode: 'TRAINING'. The same checks apply, but execution mode compatibility is evaluated against TRAINING (which behaves like SIMULATED for external effects).
6. Validation Reporting
Section titled “6. Validation Reporting”6.1 Developer-Facing Output
Section titled “6.1 Developer-Facing Output”When a developer validates a plan via CLI, the output is formatted for readability:
$ zakardo plan validate ./plans/send_welcome_email.json
✗ Validation FAILED (3 errors, 1 warning)
ERRORS:
[SCHEMA_MISSING_REQUIRED] Step 2 (send_templated_email) Input field "template_id" is required but has no binding. Suggestion: Add a binding for template_id from intent inputs or a prior step.
[BINDING_INVALID_PATH] Step 1 (get_user_by_id) Binding "$.steps[3].out.user_id" references step 3, but the plan only has 3 steps (0-2). Suggestion: Check the step index in the binding path.
[POLICY_DENIED] Plan-level Policy "candidate_plan_safety" requires CANDIDATE plans to run in DRY_RUN mode first. Current requested mode: LIVE. Suggestion: Run with --mode dry_run first, then promote the plan.
WARNINGS:
[SCHEMA_FORMAT_MISMATCH] Step 2 (send_templated_email) Input "to" expects format "email", but source (step 0, field "contact_info") is typed as plain "string". The value may not be a valid email at runtime.
Checks performed: 10/10 | Duration: 47ms6.2 Frontend-Facing Output
Section titled “6.2 Frontend-Facing Output”When the frontend receives a validation failure (from the Runtime API), it gets the structured ValidationResult. The frontend SDK renders this as:
- A summary: “This plan cannot be executed because of 3 issues.”
- Each error with its human-readable message and suggestion.
- A link to retry after fixing (for developer-authored plans).
6.3 Trace Integration
Section titled “6.3 Trace Integration”Every validation run is recorded in the Trace:
interface ValidationTraceEntry { validation_id: string; plan_id: string; plan_version: string; user_id: string; requested_mode: ExecutionMode; environment: string; result: 'pass' | 'fail'; error_count: number; warning_count: number; checks: CheckResult[]; duration_ms: number; cache_hit: boolean; timestamp: string;}This enables:
- Auditing: every validation decision is recorded.
- Monitoring: track validation failure rates, common error types, cache hit rates.
- Debugging: when a plan fails execution despite passing validation, the trace shows exactly what was checked.
7. Validation and Plan Preview
Section titled “7. Validation and Plan Preview”When the frontend renders a Plan Preview (before the user approves execution), the validation result is included:
- Valid plan: Plan Preview shows the plan with a green “Ready to execute” indicator.
- Valid plan with warnings: Plan Preview shows the plan with a yellow “Proceed with caution” indicator and lists warnings.
- Invalid plan: Plan Preview shows the plan with a red “Cannot execute” indicator and lists errors. The “Execute” button is disabled.
The Plan Preview also shows any pending gates (from Check 8, POLICY_GATE_PENDING). These are not errors — the plan is valid — but the user should know that execution will pause for approvals.
8. Extensibility
Section titled “8. Extensibility”8.1 Custom Validation Checks
Section titled “8.1 Custom Validation Checks”Developers can register custom validation checks that run as part of the pipeline:
{ validator: { custom_checks: [ { name: 'COMPLIANCE_CHECK', description: 'Verify plan complies with internal compliance requirements', check: async (plan, context) => { // Custom logic if (plan.metadata.tags.includes('financial') && !context.user_context.roles.includes('compliance_approved')) { return { passed: false, errors: [{ check: 'COMPLIANCE_CHECK', severity: 'error', message: 'Financial plans require the compliance_approved role', code: 'COMPLIANCE_ROLE_MISSING', }], }; } return { passed: true }; }, }, ], }}Custom checks run after all built-in checks. They have access to the full Plan IR and user context.
8.2 Validation Hooks
Section titled “8.2 Validation Hooks”Developers can register hooks that fire after validation completes:
onValidationPass(result)— called when a plan passes. Useful for logging, metrics, or triggering external workflows.onValidationFail(result)— called when a plan fails. Useful for alerting, escalation, or auto-repair triggers.
9. Summary of Decisions
Section titled “9. Summary of Decisions”| Decision | Resolution |
|---|---|
| Validator is mandatory | Every plan must pass validation. No bypass. Not even promoted plans (they use cached validation). |
| Defense-in-depth | The Planner handles policy at generation time. The Validator catches what the Planner missed. Policy denials at validation are exceptional, not normal. |
| Deterministic | Same plan + same state = same result. No LLM. No randomness. |
| All checks run | Even if an early check fails, all checks execute. Developer gets the full picture. |
| 10 built-in checks | Structure, capability availability, version, schema compatibility, binding validity, dependency graph, effect/mode compatibility, policy compliance, constraint satisfaction, sub-plan depth. |
| Structured errors | Each error has: check type, severity, step index, field, message, code, and optional suggestion. |
| Validation caching | Promoted plans cache their validation result. Cache invalidated on plan, capability, policy, or role changes. Default TTL: 5 minutes. |
| Custom checks | Developers can register custom validation checks in config/execution.ts. Run after built-in checks. |
| Warnings are non-blocking | Warnings (e.g., format mismatches) do not prevent execution. Errors do. |
| Plan Preview integration | Validation result is shown in Plan Preview — valid (green), warnings (yellow), invalid (red). Pending gates listed as informational. |
| Trace integration | Every validation run recorded in the Trace. Enables auditing and monitoring. |
| Target latency | < 200ms for a 10-step plan (excluding sub-plan recursion). |
10. Questions Deferred
Section titled “10. Questions Deferred”| Question | Deferred To |
|---|---|
| How does the Execution Engine handle plans that pass validation but fail at runtime? | Execution Engine Specification |
| How are validation metrics monitored and alerted on? | Trace System Specification |
What is the exact zakardo plan validate CLI command interface? | CLI and DX Specification |
| How does the frontend render validation results in Plan Preview? | Frontend Architecture Specification |
| How are custom checks distributed in Capability Packs? | Registry and Marketplace Specification |
| How does validation interact with plan promotion criteria? | Plan Library and Lifecycle Specification |