Skip to content

Plan Library

The Plan Library is where plans live, evolve, and earn trust. It stores every plan version, tracks lifecycle states, computes trust scores, and serves as the Planner’s first source when resolving intents. This specification defines the Plan Library’s data model, lifecycle state machine, trust scoring, search and matching, versioning, and integration with the Planner, Training System, and Trace System.


  1. Proven plans are preferred. The Planner should find an existing, promoted plan before generating a new one. Generating new plans is expensive (LLM calls, validation, shadow testing). The Plan Library is how Zakardo learns.
  2. Plans earn trust through evidence. Promotion is not manual and arbitrary — it’s driven by execution history, training results, and configurable criteria. Trust is computed, not assumed.
  3. Every version is preserved. The library never discards a plan version. History is essential for rollback, regression testing, and audit.
  4. Lifecycle is explicit. Every plan has a clear lifecycle state (CANDIDATE, SHADOW, PROMOTED, DEPRECATED, QUARANTINED) with defined transitions and transition reasons.
  5. Search is fast. The Planner queries the library on every intent. Search must be fast enough to not noticeably delay the response.

interface PlanRecord {
plan_id: string; // Unique identifier for this plan (all versions share this)
version: string; // Semver version
name: string; // Human-readable name
description?: string; // What this plan does
// The plan itself
plan_ir: PlanIR; // Full Plan IR
// Lifecycle
lifecycle_state: PlanLifecycleState;
lifecycle_history: LifecycleTransition[];
// Trust
trust_score: number; // 0.0 to 1.0
trust_signals: TrustSignals;
// Matching metadata
intent_patterns: string[]; // Natural language patterns this plan matches
tags: string[]; // Searchable tags
capabilities_used: string[]; // List of capability names (for search)
effect_class: EffectClass; // Aggregate effect class
// Authorship
authored_by: 'planner' | 'developer'; // Who created this plan
source_intent?: string; // If planner-generated: the original intent
source_draft?: PlanDraft; // If planner-generated: the original draft
// Timestamps
created_at: string;
promoted_at?: string;
deprecated_at?: string;
quarantined_at?: string;
// Execution summary (aggregated from traces)
execution_stats: ExecutionStats;
}
type PlanLifecycleState = 'CANDIDATE' | 'SHADOW' | 'PROMOTED' | 'DEPRECATED' | 'QUARANTINED';
interface TrustSignals {
total_executions: number;
successful_executions: number;
failed_executions: number;
success_rate: number; // 0.0 to 1.0
average_duration_ms: number;
p95_duration_ms: number;
retry_rate: number; // Fraction of executions that needed retries
training_pass_rate: number; // 0.0 to 1.0 (from training datasets)
shadow_similarity_score?: number; // 0.0 to 1.0 (from shadow comparisons)
last_execution_at?: string;
last_failure_at?: string;
manual_endorsements: number; // Developer endorsements
manual_flags: number; // Developer-reported issues
}
interface ExecutionStats {
total: number;
by_mode: Record<ExecutionMode, number>;
by_status: Record<ExecutionStatus, number>;
by_environment: Record<string, number>;
average_duration_ms: number;
p50_duration_ms: number;
p95_duration_ms: number;
p99_duration_ms: number;
last_7_days: { total: number; success_rate: number };
last_30_days: { total: number; success_rate: number };
}

┌──────────────────────────────────────┐
│ │
▼ │
CANDIDATE ──► SHADOW ──► PROMOTED ──► DEPRECATED │
│ │ │ │ │
│ │ │ │ │
└────────────┴───────────┴────────────┴──► QUARANTINED │
│
QUARANTINED ─┘
(can be restored
to CANDIDATE)
StateMeaningExecutable?Selected by Planner?
CANDIDATENew plan, not yet proven. Just generated or authored.Yes (with policy: may require DRY_RUN first)No (Planner only matches PROMOTED plans)
SHADOWRunning alongside a PROMOTED plan for comparison.Yes (SIMULATED mode only, shadow runs)No
PROMOTEDFully trusted. The preferred plan for matching intents.Yes (all modes)Yes (primary match target)
DEPRECATEDSuperseded by a newer version. Still valid but no longer preferred.Yes (explicit invocation only)No (unless no PROMOTED version exists)
QUARANTINEDRemoved from service due to failures or manual action.NoNo
FromToTriggerRequirements
CANDIDATESHADOWManual or automaticPasses validation. Shadow deployment configured.
CANDIDATEPROMOTEDManual or automatic (skip shadow)Passes validation. Developer explicitly promotes (trusted authorship). Or: meets direct-promotion criteria (e.g., developer-authored plans with test suite).
SHADOWPROMOTEDAutomatic (criteria met)Meets promotion criteria (see Section 4).
PROMOTEDDEPRECATEDAutomatic or manualNew version of same plan promoted. Or: developer manually deprecates.
AnyQUARANTINEDAutomatic or manualRepeated failures (configurable threshold). Policy violation detected. Manual quarantine by developer/operator.
QUARANTINEDCANDIDATEManual onlyDeveloper reviews issue, fixes the plan, and restores it to CANDIDATE for re-evaluation.

Every lifecycle transition is recorded:

interface LifecycleTransition {
from_state: PlanLifecycleState;
to_state: PlanLifecycleState;
reason: string; // Human-readable reason
trigger: 'automatic' | 'manual' | 'training' | 'policy';
triggered_by?: string; // user_id or system component
timestamp: string;
evidence?: {
// For automatic transitions
execution_count?: number;
success_rate?: number;
shadow_similarity?: number;
failure_count?: number;
training_pass_rate?: number;
};
}

A SHADOW plan is promoted when it meets all configured criteria:

interface PromotionCriteria {
min_shadow_executions: number; // Default: 20
min_success_rate: number; // Default: 0.95 (95%)
min_shadow_similarity: number; // Default: 0.90 (90% similar to promoted plan)
max_retry_rate: number; // Default: 0.10 (max 10% of executions needed retries)
no_policy_violations: boolean; // Default: true
min_training_pass_rate?: number; // If training dataset exists: default 0.95
min_shadow_days?: number; // Minimum days in shadow. Default: 3
}

These are configurable in config/execution.ts:

config/execution.ts
{
plan_library: {
promotion_criteria: {
min_shadow_executions: 20,
min_success_rate: 0.95,
min_shadow_similarity: 0.90,
max_retry_rate: 0.10,
no_policy_violations: true,
min_shadow_days: 3,
}
}
}

In some cases, shadow testing is not necessary:

  • Developer-authored plans with a passing test suite. If a developer writes a plan and provides a training dataset that all passes, the plan can be directly promoted.
  • Manual promotion. A developer/operator can force-promote a CANDIDATE plan. This is recorded as a manual transition.
  • Policy override. A policy can declare that certain plan types (e.g., simple READ-only plans) don’t require shadow testing.

Direct promotion always requires passing validation.

A plan is automatically quarantined when:

interface QuarantineCriteria {
failure_threshold: number; // Consecutive failures. Default: 5
failure_rate_threshold: number; // Failure rate over recent window. Default: 0.50
failure_rate_window: number; // Number of recent executions to consider. Default: 20
policy_violation_count: number; // Policy violations. Default: 1 (any violation quarantines)
}

The trust score is a weighted composite of trust signals:

function computeTrustScore(signals: TrustSignals): number {
const weights = {
success_rate: 0.3,
training_pass_rate: 0.25,
shadow_similarity: 0.15,
recency: 0.1,
retry_rate: 0.1,
endorsements: 0.05,
flags: 0.05,
};
let score = 0;
// Success rate (higher is better)
score += weights.success_rate * signals.success_rate;
// Training pass rate (higher is better)
score += weights.training_pass_rate * signals.training_pass_rate;
// Shadow similarity (higher is better, 0 if not applicable)
score += weights.shadow_similarity * (signals.shadow_similarity_score ?? 0);
// Recency (decay: plans not executed recently lose trust)
const daysSinceExecution = signals.last_execution_at
? daysBetween(signals.last_execution_at, now())
: Infinity;
const recencyScore =
daysSinceExecution < 7
? 1.0
: daysSinceExecution < 30
? 0.8
: daysSinceExecution < 90
? 0.5
: 0.2;
score += weights.recency * recencyScore;
// Retry rate (lower is better)
score += weights.retry_rate * (1 - signals.retry_rate);
// Endorsements (capped at 1.0)
score += weights.endorsements * Math.min(signals.manual_endorsements / 5, 1.0);
// Flags (negative signal)
score -= weights.flags * Math.min(signals.manual_flags / 3, 1.0);
return Math.max(0, Math.min(1, score));
}

The trust score is recalculated:

  • After every plan execution (success or failure).
  • After a training run.
  • After a manual endorsement or flag.
  • On a daily schedule (for recency decay).
ConsumerHow Trust Score Is Used
PlannerWhen multiple PROMOTED plans match an intent, prefer the one with the highest trust score.
Plan Library UIDisplay trust score as a visual indicator (green/yellow/red).
Auto-quarantinePlans with trust score below a threshold can be auto-quarantined.
Training SystemLow-trust plans are prioritized for re-training.

The Planner searches the Plan Library at the start of intent resolution (Step 4a in the Primary Flow). The search looks for PROMOTED plans that can fulfill the intent.

Plan matching is a multi-signal process:

Intent arrives
→ 1. Extract keywords and intent structure from the user's input
→ 2. Search Plan Library:
a. Match against intent_patterns (semantic similarity or keyword match)
b. Match against plan tags
c. Match against capabilities_used (if the intent maps to known capabilities)
→ 3. Filter results:
a. Only PROMOTED plans (or DEPRECATED if no PROMOTED exists)
b. Plan's capabilities are all available in the current registry
c. Plan's policy requirements are satisfiable by the current user
→ 4. Rank results:
a. Primary: intent match quality
b. Secondary: trust score
c. Tertiary: recency (prefer recently-used plans)
→ 5. If match found with high confidence: return the plan (skip generation)
If match found with low confidence: return the plan but note it as "tentative"
If no match: proceed to plan generation

Each plan has intent_patterns — a set of natural language descriptions of the intents it handles:

{
"intent_patterns": [
"delete a customer account",
"remove customer from the system",
"deactivate customer account",
"close customer account"
]
}

For Planner-generated plans, the original intent is automatically added as the first pattern. The Planner may also generate additional patterns. For developer-authored plans, patterns are explicitly declared.

The Planner assigns a confidence score to each match:

ConfidenceMeaningPlanner Action
High (> 0.85)Strong matchUse the plan directly. No generation needed.
Medium (0.60 - 0.85)Likely match but uncertainPresent the plan to the user for confirmation.
Low (< 0.60)Weak matchIgnore the library result. Generate a new plan.

Confidence thresholds are configurable.


Plans use semantic versioning (semver):

  • PATCH (1.0.0 → 1.0.1): Metadata changes, description updates, no structural change to the plan.
  • MINOR (1.0.0 → 1.1.0): Adding an optional step, changing error handling policy, non-breaking changes.
  • MAJOR (1.0.0 → 2.0.0): Changing capabilities, changing step order, changing input/output schema, breaking changes.

All versions of a plan share the same plan_id. The Plan Library stores every version:

plan_id: "plan_send_welcome_email"
v1.0.0 DEPRECATED
v1.1.0 DEPRECATED
v2.0.0 PROMOTED
v2.1.0 SHADOW

Only one version can be PROMOTED at a time for a given plan_id. Multiple can be CANDIDATE or SHADOW simultaneously.

When the Planner generates a plan that closely matches an existing plan in the library (same intent pattern, same capabilities, different bindings or order), the system can:

  1. Create a new version of the existing plan (if the structural changes are minor).
  2. Create an entirely new plan (if the changes are major).

This decision is based on similarity analysis between the new plan draft and existing plans.


When a SHADOW plan exists alongside a PROMOTED plan for the same intent:

  1. An intent arrives that matches both plans.
  2. The PROMOTED plan executes normally and serves the user (LIVE, DRY_RUN, or whatever mode was requested).
  3. The SHADOW plan executes in SIMULATED mode (no side effects, no checkpointing).
  4. Both executions produce traces.
  5. The shadow comparison engine compares the outputs.

The comparison evaluates:

SignalWhat It ChecksWeight
Output similarityDo the plans produce structurally similar outputs? (Using JSON structural diff.)High
Step countDoes the shadow plan use more/fewer steps?Low
DurationIs the shadow plan faster or slower?Low
Error agreementWhen the promoted plan fails, does the shadow plan also fail (for the same reason)?Medium
Gate agreementDo both plans trigger the same gates?Medium

The similarity score is stored in the plan’s trust_signals.shadow_similarity_score.

Shadow execution adds overhead:

  • One additional SIMULATED execution per intent that matches a shadow plan.
  • SIMULATED uses mock services, so it’s fast and has no external effects.
  • The comparison runs asynchronously after both executions complete.

Shadow can be disabled or rate-limited per plan or globally.


-- zakardo_plans table
CREATE TABLE zakardo_plans (
plan_id TEXT NOT NULL,
version TEXT NOT NULL,
name TEXT NOT NULL,
description TEXT,
plan_ir JSONB NOT NULL,
lifecycle_state TEXT NOT NULL,
trust_score REAL NOT NULL DEFAULT 0,
trust_signals JSONB NOT NULL DEFAULT '{}',
intent_patterns JSONB NOT NULL DEFAULT '[]',
tags JSONB NOT NULL DEFAULT '[]',
capabilities_used JSONB NOT NULL DEFAULT '[]',
effect_class TEXT NOT NULL,
authored_by TEXT NOT NULL,
source_intent TEXT,
execution_stats JSONB NOT NULL DEFAULT '{}',
lifecycle_history JSONB NOT NULL DEFAULT '[]',
created_at TIMESTAMP NOT NULL,
promoted_at TIMESTAMP,
deprecated_at TIMESTAMP,
quarantined_at TIMESTAMP,
PRIMARY KEY (plan_id, version),
INDEX idx_lifecycle (lifecycle_state),
INDEX idx_trust (trust_score),
INDEX idx_effect (effect_class),
INDEX idx_created (created_at)
);
-- Full-text search index on intent_patterns and tags (PostgreSQL example)
-- CREATE INDEX idx_intent_search ON zakardo_plans USING GIN (intent_patterns);

For local development, plans are stored as JSON files in .zakardo/plans/ (one directory per plan_id, one file per version).


10.1 Internal API (Used by Planner, Training System)

Section titled “10.1 Internal API (Used by Planner, Training System)”
interface PlanLibrary {
// Search for plans matching an intent
search(query: PlanSearchQuery): Promise<PlanSearchResult[]>;
// Get a specific plan version
get(plan_id: string, version?: string): Promise<PlanRecord | null>;
// Get all versions of a plan
getVersions(plan_id: string): Promise<PlanRecord[]>;
// Store a new plan (enters as CANDIDATE)
store(plan: PlanIR, metadata: PlanMetadata): Promise<PlanRecord>;
// Transition lifecycle state
transition(
plan_id: string,
version: string,
to: PlanLifecycleState,
reason: string,
trigger: TransitionTrigger,
): Promise<PlanRecord>;
// Record execution result (updates stats and trust score)
recordExecutionResult(plan_id: string, version: string, result: ExecutionResult): Promise<void>;
// Record training result
recordTrainingResult(plan_id: string, version: string, result: TrainingResult): Promise<void>;
// Endorse or flag a plan
endorse(plan_id: string, version: string, user_id: string): Promise<void>;
flag(plan_id: string, version: string, user_id: string, reason: string): Promise<void>;
}
GET /api/plans List plans (filterable by state, tags, capability)
GET /api/plans/:plan_id Get plan (latest promoted version)
GET /api/plans/:plan_id/versions List all versions
GET /api/plans/:plan_id/:version Get specific version
POST /api/plans Submit a new plan (developer-authored)
POST /api/plans/:plan_id/:version/promote Force-promote
POST /api/plans/:plan_id/:version/deprecate Deprecate
POST /api/plans/:plan_id/:version/quarantine Quarantine
POST /api/plans/:plan_id/:version/restore Restore from quarantine
POST /api/plans/:plan_id/:version/endorse Endorse
POST /api/plans/:plan_id/:version/flag Flag

Developers can write plans directly as Full Plan IR (without going through the Planner):

Developer writes plan.json (Full Plan IR)
→ CLI: zakardo plan validate ./plan.json
→ If valid: zakardo plan submit ./plan.json
→ Plan enters Plan Library as CANDIDATE
→ Developer can force-promote or let it go through shadow
  • Precision. For critical workflows, a developer may want exact control over which capabilities are called and how data flows.
  • Performance. Developer-authored plans skip the Planner entirely (no LLM calls, no generation latency).
  • Testing. Plans can be authored as part of a test suite with expected inputs and outputs.
  • Migration. When migrating from traditional architectures, developers can encode existing workflows as plans.

Developers can bundle plans into Plan Packs (alongside Capability Packs):

my-crm-plans/
plans/
send_welcome_email.json
create_new_customer.json
generate_monthly_report.json
training/
send_welcome_email.dataset.json
create_new_customer.dataset.json
pack.json # Pack metadata

Plan Packs can be published to the Registry (Zakardo Cloud) for distribution.


The Validator caches validation results for promoted plans (see Validator Specification, Section 4). The Plan Library tracks which validation cache entries are current:

  • When a plan is promoted, its validation result is cached.
  • When a plan’s capabilities change versions, the cache is invalidated.
  • When policies change, the cache is invalidated.
  • The Plan Library stores the validation_hash from the last successful validation for quick cache lookups.

DecisionResolution
Library-first resolutionThe Planner searches the Plan Library before generating a new plan. Promoted plans are preferred.
Five lifecycle statesCANDIDATE, SHADOW, PROMOTED, DEPRECATED, QUARANTINED. Clear transitions with reasons.
Evidence-based promotionSHADOW → PROMOTED requires meeting configurable criteria (min executions, success rate, shadow similarity).
Direct promotion pathDeveloper-authored plans with test suites or manual promotion can skip shadow.
Trust scoreWeighted composite of success rate, training pass rate, shadow similarity, recency, retry rate, endorsements, flags.
One PROMOTED per plan_idOnly one version of a plan can be PROMOTED at a time. Older versions auto-deprecate.
All versions preservedNo version is ever deleted from the library. History is immutable.
Auto-quarantinePlans that exceed failure thresholds are automatically quarantined. Configurable.
Shadow runs in SIMULATEDShadow plans execute with no side effects. Comparison is async. Shadow can be rate-limited.
Plan PacksPlans can be bundled with capabilities and training datasets for distribution.
Transition auditEvery lifecycle transition is recorded with timestamp, reason, trigger, and evidence.

QuestionDeferred To
How does the Training System drive plan evaluation and promotion?Training and Validation Specification
How does the Planner perform semantic matching against intent patterns?Planner Specification (already covered at high level)
How are Plan Packs published and distributed?Registry and Marketplace Specification
What is the exact CLI interface for plan management?CLI and DX Specification
How does the frontend display plan lifecycle and trust scores?Frontend Architecture Specification
How does Zakardo Cloud sync plans across tenants?PRD — Zakardo Cloud