Plan Library
The Plan Library is where plans live, evolve, and earn trust. It stores every plan version, tracks lifecycle states, computes trust scores, and serves as the Planner’s first source when resolving intents. This specification defines the Plan Library’s data model, lifecycle state machine, trust scoring, search and matching, versioning, and integration with the Planner, Training System, and Trace System.
1. Design Goals
Section titled “1. Design Goals”- Proven plans are preferred. The Planner should find an existing, promoted plan before generating a new one. Generating new plans is expensive (LLM calls, validation, shadow testing). The Plan Library is how Zakardo learns.
- Plans earn trust through evidence. Promotion is not manual and arbitrary — it’s driven by execution history, training results, and configurable criteria. Trust is computed, not assumed.
- Every version is preserved. The library never discards a plan version. History is essential for rollback, regression testing, and audit.
- Lifecycle is explicit. Every plan has a clear lifecycle state (CANDIDATE, SHADOW, PROMOTED, DEPRECATED, QUARANTINED) with defined transitions and transition reasons.
- Search is fast. The Planner queries the library on every intent. Search must be fast enough to not noticeably delay the response.
2. Plan Library Data Model
Section titled “2. Plan Library Data Model”2.1 Plan Record
Section titled “2.1 Plan Record”interface PlanRecord { plan_id: string; // Unique identifier for this plan (all versions share this) version: string; // Semver version name: string; // Human-readable name description?: string; // What this plan does
// The plan itself plan_ir: PlanIR; // Full Plan IR
// Lifecycle lifecycle_state: PlanLifecycleState; lifecycle_history: LifecycleTransition[];
// Trust trust_score: number; // 0.0 to 1.0 trust_signals: TrustSignals;
// Matching metadata intent_patterns: string[]; // Natural language patterns this plan matches tags: string[]; // Searchable tags capabilities_used: string[]; // List of capability names (for search) effect_class: EffectClass; // Aggregate effect class
// Authorship authored_by: 'planner' | 'developer'; // Who created this plan source_intent?: string; // If planner-generated: the original intent source_draft?: PlanDraft; // If planner-generated: the original draft
// Timestamps created_at: string; promoted_at?: string; deprecated_at?: string; quarantined_at?: string;
// Execution summary (aggregated from traces) execution_stats: ExecutionStats;}2.2 Lifecycle State
Section titled “2.2 Lifecycle State”type PlanLifecycleState = 'CANDIDATE' | 'SHADOW' | 'PROMOTED' | 'DEPRECATED' | 'QUARANTINED';2.3 Trust Signals
Section titled “2.3 Trust Signals”interface TrustSignals { total_executions: number; successful_executions: number; failed_executions: number; success_rate: number; // 0.0 to 1.0 average_duration_ms: number; p95_duration_ms: number; retry_rate: number; // Fraction of executions that needed retries training_pass_rate: number; // 0.0 to 1.0 (from training datasets) shadow_similarity_score?: number; // 0.0 to 1.0 (from shadow comparisons) last_execution_at?: string; last_failure_at?: string; manual_endorsements: number; // Developer endorsements manual_flags: number; // Developer-reported issues}2.4 Execution Stats
Section titled “2.4 Execution Stats”interface ExecutionStats { total: number; by_mode: Record<ExecutionMode, number>; by_status: Record<ExecutionStatus, number>; by_environment: Record<string, number>; average_duration_ms: number; p50_duration_ms: number; p95_duration_ms: number; p99_duration_ms: number; last_7_days: { total: number; success_rate: number }; last_30_days: { total: number; success_rate: number };}3. Plan Lifecycle State Machine
Section titled “3. Plan Lifecycle State Machine”3.1 State Diagram
Section titled “3.1 State Diagram” ┌──────────────────────────────────────┐ │ │ ▼ │CANDIDATE ──► SHADOW ──► PROMOTED ──► DEPRECATED │ │ │ │ │ │ │ │ │ │ │ └────────────┴───────────┴────────────┴──► QUARANTINED │ │ QUARANTINED ─┘ (can be restored to CANDIDATE)3.2 States
Section titled “3.2 States”| State | Meaning | Executable? | Selected by Planner? |
|---|---|---|---|
CANDIDATE | New plan, not yet proven. Just generated or authored. | Yes (with policy: may require DRY_RUN first) | No (Planner only matches PROMOTED plans) |
SHADOW | Running alongside a PROMOTED plan for comparison. | Yes (SIMULATED mode only, shadow runs) | No |
PROMOTED | Fully trusted. The preferred plan for matching intents. | Yes (all modes) | Yes (primary match target) |
DEPRECATED | Superseded by a newer version. Still valid but no longer preferred. | Yes (explicit invocation only) | No (unless no PROMOTED version exists) |
QUARANTINED | Removed from service due to failures or manual action. | No | No |
3.3 Transitions
Section titled “3.3 Transitions”| From | To | Trigger | Requirements |
|---|---|---|---|
CANDIDATE | SHADOW | Manual or automatic | Passes validation. Shadow deployment configured. |
CANDIDATE | PROMOTED | Manual or automatic (skip shadow) | Passes validation. Developer explicitly promotes (trusted authorship). Or: meets direct-promotion criteria (e.g., developer-authored plans with test suite). |
SHADOW | PROMOTED | Automatic (criteria met) | Meets promotion criteria (see Section 4). |
PROMOTED | DEPRECATED | Automatic or manual | New version of same plan promoted. Or: developer manually deprecates. |
| Any | QUARANTINED | Automatic or manual | Repeated failures (configurable threshold). Policy violation detected. Manual quarantine by developer/operator. |
QUARANTINED | CANDIDATE | Manual only | Developer reviews issue, fixes the plan, and restores it to CANDIDATE for re-evaluation. |
3.4 Transition Record
Section titled “3.4 Transition Record”Every lifecycle transition is recorded:
interface LifecycleTransition { from_state: PlanLifecycleState; to_state: PlanLifecycleState; reason: string; // Human-readable reason trigger: 'automatic' | 'manual' | 'training' | 'policy'; triggered_by?: string; // user_id or system component timestamp: string; evidence?: { // For automatic transitions execution_count?: number; success_rate?: number; shadow_similarity?: number; failure_count?: number; training_pass_rate?: number; };}4. Promotion Criteria
Section titled “4. Promotion Criteria”4.1 Shadow-to-Promoted Criteria
Section titled “4.1 Shadow-to-Promoted Criteria”A SHADOW plan is promoted when it meets all configured criteria:
interface PromotionCriteria { min_shadow_executions: number; // Default: 20 min_success_rate: number; // Default: 0.95 (95%) min_shadow_similarity: number; // Default: 0.90 (90% similar to promoted plan) max_retry_rate: number; // Default: 0.10 (max 10% of executions needed retries) no_policy_violations: boolean; // Default: true min_training_pass_rate?: number; // If training dataset exists: default 0.95 min_shadow_days?: number; // Minimum days in shadow. Default: 3}These are configurable in config/execution.ts:
{ plan_library: { promotion_criteria: { min_shadow_executions: 20, min_success_rate: 0.95, min_shadow_similarity: 0.90, max_retry_rate: 0.10, no_policy_violations: true, min_shadow_days: 3, } }}4.2 Direct Promotion (Skip Shadow)
Section titled “4.2 Direct Promotion (Skip Shadow)”In some cases, shadow testing is not necessary:
- Developer-authored plans with a passing test suite. If a developer writes a plan and provides a training dataset that all passes, the plan can be directly promoted.
- Manual promotion. A developer/operator can force-promote a CANDIDATE plan. This is recorded as a manual transition.
- Policy override. A policy can declare that certain plan types (e.g., simple READ-only plans) don’t require shadow testing.
Direct promotion always requires passing validation.
4.3 Auto-Quarantine Criteria
Section titled “4.3 Auto-Quarantine Criteria”A plan is automatically quarantined when:
interface QuarantineCriteria { failure_threshold: number; // Consecutive failures. Default: 5 failure_rate_threshold: number; // Failure rate over recent window. Default: 0.50 failure_rate_window: number; // Number of recent executions to consider. Default: 20 policy_violation_count: number; // Policy violations. Default: 1 (any violation quarantines)}5. Trust Score
Section titled “5. Trust Score”5.1 Computation
Section titled “5.1 Computation”The trust score is a weighted composite of trust signals:
function computeTrustScore(signals: TrustSignals): number { const weights = { success_rate: 0.3, training_pass_rate: 0.25, shadow_similarity: 0.15, recency: 0.1, retry_rate: 0.1, endorsements: 0.05, flags: 0.05, };
let score = 0;
// Success rate (higher is better) score += weights.success_rate * signals.success_rate;
// Training pass rate (higher is better) score += weights.training_pass_rate * signals.training_pass_rate;
// Shadow similarity (higher is better, 0 if not applicable) score += weights.shadow_similarity * (signals.shadow_similarity_score ?? 0);
// Recency (decay: plans not executed recently lose trust) const daysSinceExecution = signals.last_execution_at ? daysBetween(signals.last_execution_at, now()) : Infinity; const recencyScore = daysSinceExecution < 7 ? 1.0 : daysSinceExecution < 30 ? 0.8 : daysSinceExecution < 90 ? 0.5 : 0.2; score += weights.recency * recencyScore;
// Retry rate (lower is better) score += weights.retry_rate * (1 - signals.retry_rate);
// Endorsements (capped at 1.0) score += weights.endorsements * Math.min(signals.manual_endorsements / 5, 1.0);
// Flags (negative signal) score -= weights.flags * Math.min(signals.manual_flags / 3, 1.0);
return Math.max(0, Math.min(1, score));}5.2 When Trust Score Updates
Section titled “5.2 When Trust Score Updates”The trust score is recalculated:
- After every plan execution (success or failure).
- After a training run.
- After a manual endorsement or flag.
- On a daily schedule (for recency decay).
5.3 Trust Score Uses
Section titled “5.3 Trust Score Uses”| Consumer | How Trust Score Is Used |
|---|---|
| Planner | When multiple PROMOTED plans match an intent, prefer the one with the highest trust score. |
| Plan Library UI | Display trust score as a visual indicator (green/yellow/red). |
| Auto-quarantine | Plans with trust score below a threshold can be auto-quarantined. |
| Training System | Low-trust plans are prioritized for re-training. |
6. Plan Search and Matching
Section titled “6. Plan Search and Matching”6.1 When the Planner Searches
Section titled “6.1 When the Planner Searches”The Planner searches the Plan Library at the start of intent resolution (Step 4a in the Primary Flow). The search looks for PROMOTED plans that can fulfill the intent.
6.2 Matching Strategy
Section titled “6.2 Matching Strategy”Plan matching is a multi-signal process:
Intent arrives → 1. Extract keywords and intent structure from the user's input → 2. Search Plan Library: a. Match against intent_patterns (semantic similarity or keyword match) b. Match against plan tags c. Match against capabilities_used (if the intent maps to known capabilities) → 3. Filter results: a. Only PROMOTED plans (or DEPRECATED if no PROMOTED exists) b. Plan's capabilities are all available in the current registry c. Plan's policy requirements are satisfiable by the current user → 4. Rank results: a. Primary: intent match quality b. Secondary: trust score c. Tertiary: recency (prefer recently-used plans) → 5. If match found with high confidence: return the plan (skip generation) If match found with low confidence: return the plan but note it as "tentative" If no match: proceed to plan generation6.3 Intent Patterns
Section titled “6.3 Intent Patterns”Each plan has intent_patterns — a set of natural language descriptions of the intents it handles:
{ "intent_patterns": [ "delete a customer account", "remove customer from the system", "deactivate customer account", "close customer account" ]}For Planner-generated plans, the original intent is automatically added as the first pattern. The Planner may also generate additional patterns. For developer-authored plans, patterns are explicitly declared.
6.4 Match Confidence
Section titled “6.4 Match Confidence”The Planner assigns a confidence score to each match:
| Confidence | Meaning | Planner Action |
|---|---|---|
| High (> 0.85) | Strong match | Use the plan directly. No generation needed. |
| Medium (0.60 - 0.85) | Likely match but uncertain | Present the plan to the user for confirmation. |
| Low (< 0.60) | Weak match | Ignore the library result. Generate a new plan. |
Confidence thresholds are configurable.
7. Plan Versioning
Section titled “7. Plan Versioning”7.1 Version Model
Section titled “7.1 Version Model”Plans use semantic versioning (semver):
- PATCH (1.0.0 → 1.0.1): Metadata changes, description updates, no structural change to the plan.
- MINOR (1.0.0 → 1.1.0): Adding an optional step, changing error handling policy, non-breaking changes.
- MAJOR (1.0.0 → 2.0.0): Changing capabilities, changing step order, changing input/output schema, breaking changes.
7.2 Version Chain
Section titled “7.2 Version Chain”All versions of a plan share the same plan_id. The Plan Library stores every version:
plan_id: "plan_send_welcome_email" v1.0.0 DEPRECATED v1.1.0 DEPRECATED v2.0.0 PROMOTED v2.1.0 SHADOWOnly one version can be PROMOTED at a time for a given plan_id. Multiple can be CANDIDATE or SHADOW simultaneously.
7.3 Automatic Versioning
Section titled “7.3 Automatic Versioning”When the Planner generates a plan that closely matches an existing plan in the library (same intent pattern, same capabilities, different bindings or order), the system can:
- Create a new version of the existing plan (if the structural changes are minor).
- Create an entirely new plan (if the changes are major).
This decision is based on similarity analysis between the new plan draft and existing plans.
8. Shadow Deployment
Section titled “8. Shadow Deployment”8.1 How Shadow Works
Section titled “8.1 How Shadow Works”When a SHADOW plan exists alongside a PROMOTED plan for the same intent:
- An intent arrives that matches both plans.
- The PROMOTED plan executes normally and serves the user (LIVE, DRY_RUN, or whatever mode was requested).
- The SHADOW plan executes in SIMULATED mode (no side effects, no checkpointing).
- Both executions produce traces.
- The shadow comparison engine compares the outputs.
8.2 Shadow Comparison
Section titled “8.2 Shadow Comparison”The comparison evaluates:
| Signal | What It Checks | Weight |
|---|---|---|
| Output similarity | Do the plans produce structurally similar outputs? (Using JSON structural diff.) | High |
| Step count | Does the shadow plan use more/fewer steps? | Low |
| Duration | Is the shadow plan faster or slower? | Low |
| Error agreement | When the promoted plan fails, does the shadow plan also fail (for the same reason)? | Medium |
| Gate agreement | Do both plans trigger the same gates? | Medium |
The similarity score is stored in the plan’s trust_signals.shadow_similarity_score.
8.3 Shadow Overhead
Section titled “8.3 Shadow Overhead”Shadow execution adds overhead:
- One additional SIMULATED execution per intent that matches a shadow plan.
- SIMULATED uses mock services, so it’s fast and has no external effects.
- The comparison runs asynchronously after both executions complete.
Shadow can be disabled or rate-limited per plan or globally.
9. Plan Storage
Section titled “9. Plan Storage”9.1 Database Schema
Section titled “9.1 Database Schema”-- zakardo_plans tableCREATE TABLE zakardo_plans ( plan_id TEXT NOT NULL, version TEXT NOT NULL, name TEXT NOT NULL, description TEXT, plan_ir JSONB NOT NULL, lifecycle_state TEXT NOT NULL, trust_score REAL NOT NULL DEFAULT 0, trust_signals JSONB NOT NULL DEFAULT '{}', intent_patterns JSONB NOT NULL DEFAULT '[]', tags JSONB NOT NULL DEFAULT '[]', capabilities_used JSONB NOT NULL DEFAULT '[]', effect_class TEXT NOT NULL, authored_by TEXT NOT NULL, source_intent TEXT, execution_stats JSONB NOT NULL DEFAULT '{}', lifecycle_history JSONB NOT NULL DEFAULT '[]', created_at TIMESTAMP NOT NULL, promoted_at TIMESTAMP, deprecated_at TIMESTAMP, quarantined_at TIMESTAMP,
PRIMARY KEY (plan_id, version), INDEX idx_lifecycle (lifecycle_state), INDEX idx_trust (trust_score), INDEX idx_effect (effect_class), INDEX idx_created (created_at));
-- Full-text search index on intent_patterns and tags (PostgreSQL example)-- CREATE INDEX idx_intent_search ON zakardo_plans USING GIN (intent_patterns);For local development, plans are stored as JSON files in .zakardo/plans/ (one directory per plan_id, one file per version).
10. Plan Library API
Section titled “10. Plan Library API”10.1 Internal API (Used by Planner, Training System)
Section titled “10.1 Internal API (Used by Planner, Training System)”interface PlanLibrary { // Search for plans matching an intent search(query: PlanSearchQuery): Promise<PlanSearchResult[]>;
// Get a specific plan version get(plan_id: string, version?: string): Promise<PlanRecord | null>;
// Get all versions of a plan getVersions(plan_id: string): Promise<PlanRecord[]>;
// Store a new plan (enters as CANDIDATE) store(plan: PlanIR, metadata: PlanMetadata): Promise<PlanRecord>;
// Transition lifecycle state transition( plan_id: string, version: string, to: PlanLifecycleState, reason: string, trigger: TransitionTrigger, ): Promise<PlanRecord>;
// Record execution result (updates stats and trust score) recordExecutionResult(plan_id: string, version: string, result: ExecutionResult): Promise<void>;
// Record training result recordTrainingResult(plan_id: string, version: string, result: TrainingResult): Promise<void>;
// Endorse or flag a plan endorse(plan_id: string, version: string, user_id: string): Promise<void>; flag(plan_id: string, version: string, user_id: string, reason: string): Promise<void>;}10.2 External API (Runtime API endpoints)
Section titled “10.2 External API (Runtime API endpoints)”GET /api/plans List plans (filterable by state, tags, capability)GET /api/plans/:plan_id Get plan (latest promoted version)GET /api/plans/:plan_id/versions List all versionsGET /api/plans/:plan_id/:version Get specific versionPOST /api/plans Submit a new plan (developer-authored)POST /api/plans/:plan_id/:version/promote Force-promotePOST /api/plans/:plan_id/:version/deprecate DeprecatePOST /api/plans/:plan_id/:version/quarantine QuarantinePOST /api/plans/:plan_id/:version/restore Restore from quarantinePOST /api/plans/:plan_id/:version/endorse EndorsePOST /api/plans/:plan_id/:version/flag Flag11. Developer-Authored Plans
Section titled “11. Developer-Authored Plans”11.1 Authoring Flow
Section titled “11.1 Authoring Flow”Developers can write plans directly as Full Plan IR (without going through the Planner):
Developer writes plan.json (Full Plan IR) → CLI: zakardo plan validate ./plan.json → If valid: zakardo plan submit ./plan.json → Plan enters Plan Library as CANDIDATE → Developer can force-promote or let it go through shadow11.2 Why Author Plans?
Section titled “11.2 Why Author Plans?”- Precision. For critical workflows, a developer may want exact control over which capabilities are called and how data flows.
- Performance. Developer-authored plans skip the Planner entirely (no LLM calls, no generation latency).
- Testing. Plans can be authored as part of a test suite with expected inputs and outputs.
- Migration. When migrating from traditional architectures, developers can encode existing workflows as plans.
11.3 Plan Packs
Section titled “11.3 Plan Packs”Developers can bundle plans into Plan Packs (alongside Capability Packs):
my-crm-plans/ plans/ send_welcome_email.json create_new_customer.json generate_monthly_report.json training/ send_welcome_email.dataset.json create_new_customer.dataset.json pack.json # Pack metadataPlan Packs can be published to the Registry (Zakardo Cloud) for distribution.
12. Validation Cache Integration
Section titled “12. Validation Cache Integration”The Validator caches validation results for promoted plans (see Validator Specification, Section 4). The Plan Library tracks which validation cache entries are current:
- When a plan is promoted, its validation result is cached.
- When a plan’s capabilities change versions, the cache is invalidated.
- When policies change, the cache is invalidated.
- The Plan Library stores the
validation_hashfrom the last successful validation for quick cache lookups.
13. Summary of Decisions
Section titled “13. Summary of Decisions”| Decision | Resolution |
|---|---|
| Library-first resolution | The Planner searches the Plan Library before generating a new plan. Promoted plans are preferred. |
| Five lifecycle states | CANDIDATE, SHADOW, PROMOTED, DEPRECATED, QUARANTINED. Clear transitions with reasons. |
| Evidence-based promotion | SHADOW → PROMOTED requires meeting configurable criteria (min executions, success rate, shadow similarity). |
| Direct promotion path | Developer-authored plans with test suites or manual promotion can skip shadow. |
| Trust score | Weighted composite of success rate, training pass rate, shadow similarity, recency, retry rate, endorsements, flags. |
| One PROMOTED per plan_id | Only one version of a plan can be PROMOTED at a time. Older versions auto-deprecate. |
| All versions preserved | No version is ever deleted from the library. History is immutable. |
| Auto-quarantine | Plans that exceed failure thresholds are automatically quarantined. Configurable. |
| Shadow runs in SIMULATED | Shadow plans execute with no side effects. Comparison is async. Shadow can be rate-limited. |
| Plan Packs | Plans can be bundled with capabilities and training datasets for distribution. |
| Transition audit | Every lifecycle transition is recorded with timestamp, reason, trigger, and evidence. |
14. Questions Deferred
Section titled “14. Questions Deferred”| Question | Deferred To |
|---|---|
| How does the Training System drive plan evaluation and promotion? | Training and Validation Specification |
| How does the Planner perform semantic matching against intent patterns? | Planner Specification (already covered at high level) |
| How are Plan Packs published and distributed? | Registry and Marketplace Specification |
| What is the exact CLI interface for plan management? | CLI and DX Specification |
| How does the frontend display plan lifecycle and trust scores? | Frontend Architecture Specification |
| How does Zakardo Cloud sync plans across tenants? | PRD — Zakardo Cloud |