MetaCognition Patterns for AI Agent Self-Monitoring and Adaptive Control

Executive Summary

MetaCognition — the capacity to monitor and regulate one's own cognitive processes — has been studied in cognitive science since the 1970s and has become a first-class design concern in modern AI agent systems. As agent runtimes grow more complex, with separate components for session management, decision-making, and execution, the question of who watches the watchers becomes architecturally critical.

This article surveys the foundations and engineering patterns of metacognitive systems for AI agents, with particular attention to the architectural context of a Session-Governor-Executor runtime where MetaCognition is an independent module that performs dual observation: monitoring both Governor decision quality and Session interaction quality, then synthesizing control recommendations via a consensus mechanism.

Key findings:

The Nelson-Narens meta-level/object-level framework from cognitive science maps cleanly onto agent runtime architectures and provides principled vocabulary for distinguishing monitoring (meta-level reads from object-level) from control (meta-level writes to object-level).
Production systems like Reflexion (Shinn et al., 2023), Self-Refine (Madaan et al., 2023), and Voyager (Wang et al., 2023) demonstrate that language-mediated self-monitoring is practical and produces measurable performance gains without weight updates.
The MAPE-K loop (IBM, 2003) from autonomic computing remains the most battle-tested pattern for adaptive control cycles and translates directly into agent metacognition architectures.
Dual observation — independently monitoring both a reasoning component and an interaction quality channel — reduces blind spots caused by single-channel observation bias and enables richer anomaly detection.
Consensus mechanisms that aggregate signals from independent observers produce more reliable control recommendations than any single signal, at the cost of increased latency and implementation complexity.
The principle that "Governor may be LLM-assisted, but must not be LLM-sovereign" is well-supported by evidence from self-adaptive systems research: governance components that delegate all authority to stochastic sub-processes systematically fail under distributional shift and adversarial input.

1. Cognitive Science Foundations

1.1 The Nelson-Narens Framework

The foundational theoretical structure for metacognition in cognitive science was established by Thomas Nelson and Louis Narens in 1990. Their framework introduces a strict two-level architecture:

Object level: The cognitive process performing the task (e.g., memory retrieval, problem solving, language generation).
Meta level: A monitoring and control layer that holds a model of the object-level process and can influence it.

The meta level communicates with the object level through two channels:

Monitoring (object → meta): The meta level receives reports about the current state of object-level processes. Classic examples include feeling-of-knowing (FOK) judgments — a person's estimate that they know an answer before successfully retrieving it — and confidence ratings on recalled information.
Control (meta → object): The meta level issues directives that modify object-level behavior. Examples include the decision to continue searching memory, to terminate a recall attempt, or to allocate more processing time.

This bidirectional information flow is not symmetric. Monitoring is read-only and continuous; control is write-only and discrete. In engineering terms: the meta level subscribes to an event stream from the object level, and emits command events back to it. Neither layer has unrestricted access to the internals of the other — they communicate through well-defined interfaces.

1.2 Metacognitive Knowledge vs. Metacognitive Regulation

Flavell (1979) drew an earlier and equally important distinction between two kinds of metacognitive capacity:

Metacognitive knowledge is declarative knowledge about cognitive processes — knowing that one tends to make arithmetic errors when tired, or that a particular reasoning pattern leads to overconfident conclusions. This is stable, slow-to-change, and learnable from experience.

Metacognitive regulation is the executive function that applies metacognitive knowledge in real time: monitoring current performance, detecting when something is going wrong, and triggering corrective action. This is dynamic, operates on short timescales, and is the primary target of engineering implementation.

For agent runtimes, this distinction has direct design implications. Metacognitive knowledge lives in configuration, heuristic rules, and learned thresholds — it is the policy the MetaCognition module uses. Metacognitive regulation is the active runtime loop — the mechanism that applies that policy against live observation streams.

1.3 Feeling-of-Knowing and Confidence Calibration

A central research finding in metacognitive psychology is that humans exhibit calibration between their subjective confidence and their actual accuracy, though this calibration is imperfect and domain-dependent. Well-calibrated confidence signals are essential for knowing when to continue versus when to escalate or seek additional information.

In AI systems, the equivalent of feeling-of-knowing is confidence scoring on outputs and decisions. Language models generate logit distributions over tokens; sampled responses can have their likelihood computed; and proxy signals like response entropy, hedging language frequency, and self-consistency across multiple samples all serve as partial surrogates for true confidence.

The engineering challenge is that raw LLM confidence signals are poorly calibrated by default (Guo et al., 2017). MetaCognition modules must either re-calibrate these signals using held-out validation data, use ensemble methods to derive more reliable estimates, or treat LLM confidence as one weak signal among many rather than as ground truth.

2. MetaCognition in AI Systems

2.1 The Self-Reflection Paradigm: Reflexion

Reflexion (Shinn et al., 2023) is the most cited implementation of language-mediated metacognition in task-solving agents. The core idea is that an agent can verbally reflect on its failure traces and write a natural-language self-critique into an episodic memory buffer. On the next attempt, the agent conditions on both the task and its accumulated reflection history.

The mechanism has three components:

Actor: Generates action trajectories toward a goal.
Evaluator: Scores the trajectory against success criteria (this can be environment-grounded or LLM-based).
Self-Reflector: Given the trajectory and the score, generates a verbal reflection that explains what went wrong and what should be tried differently.

Reflexion achieved 91% pass@1 on HumanEval coding benchmarks, surpassing GPT-4's 80% at the time — demonstrating that self-monitoring loops add measurable capability beyond raw model performance.

The architectural lesson: the evaluator is a distinct process from the actor. This separation-of-concerns is structural metacognition — the evaluator cannot be compromised by the same biases that led the actor to fail, because it operates on the output rather than participating in its generation.

2.2 Iterative Self-Improvement: Self-Refine

Self-Refine (Madaan et al., 2023) generalizes this pattern: the same LLM acts as generator, critic, and refiner in a loop. No additional training is required. The model generates an initial output, critiques it, refines based on the critique, and iterates until a stopping condition is met.

Key results across seven diverse tasks showed ~20% absolute improvement over single-pass generation. The approach requires no external supervision signal — the self-critique IS the metacognitive monitoring channel.

However, Self-Refine has a known failure mode: the model's critic and the model's generator share all the same biases. A model that systematically misunderstands a task will generate wrong outputs, critique them with wrong standards, and refine them toward wrong conclusions — without ever triggering an anomaly flag. This is a correlated failure problem. The monitoring and the monitored share a common failure mode because they are the same underlying model.

This motivates architectures where the monitor is structurally independent of the actor.

2.3 Skill-Building with Iterative Curriculum: Voyager

Voyager (Wang et al., 2023) demonstrates metacognition at the skill-library level. The agent operates in Minecraft and uses an automatic curriculum generator to propose increasingly difficult tasks. After attempting each task, an iterative prompting mechanism incorporates environment feedback, execution errors, and a code verifier's judgments to refine the current skill program.

The metacognitive element: the curriculum itself is a meta-level process that observes object-level performance history and proposes the next challenge at the appropriate difficulty. The skill library is persistent metacognitive knowledge — beliefs about which strategies work in which contexts — that accumulate across episodes.

Empirically, Voyager achieved 3.3× more unique items collected and completed tech tree milestones 15.3× faster than prior approaches. The key enabler was iterative self-correction within each skill's generation loop: errors were not just reported but recursively fed back into the generation process.

2.4 ReAct: Interleaved Reasoning as Inline Metacognition

ReAct (Yao et al., 2023) takes a different angle: rather than a separate metacognitive loop, reasoning traces are interleaved inline with actions. The model generates a thought ("I need to check the current date"), an action ("search Wikipedia"), and an observation ("the page says..."), then a new thought that incorporates the observation.

This is metacognition without a separate component — reasoning traces serve as both the task-solving mechanism and the monitoring channel. The self-generated thought steps create a natural audit log of the agent's beliefs and intentions, which can be analyzed post-hoc or inspected in real time.

ReAct's limitation is that it conflates the executor and the monitor in a single generation pass. There is no independent check on whether the reasoning traces are sound; a model that generates plausible-sounding but incorrect reasoning will not self-detect the error.

2.5 CoALA: Structured Cognitive Architecture

The Cognitive Architectures for Language Agents (CoALA) framework (Sumers et al., 2023) provides a systematic taxonomy: agents have modular memory components, structured action spaces, and a generalized decision-making process. Within CoALA's model, metacognition maps to what they call grounding — the process of checking in-context beliefs against stored memory and external state — and to decision evaluation, which assesses whether a plan should be executed, deferred, or revised.

CoALA is valuable for this analysis because it contextualizes metacognition as an architectural slot, not just a prompting trick. A runtime system can reserve explicit component positions for monitoring and evaluation functions, making them first-class citizens of the architecture rather than emergent behaviors of a single monolithic model.

3. Observability Patterns for Agent Systems

3.1 The Observer Pattern and Its Limitations

The classic Gang of Four Observer pattern (Gamma et al., 1994) defines a one-to-many dependency: when one object's state changes, all dependents are notified automatically. Applied naively to agent monitoring, this means the Governor emits events that MetaCognition subscribes to.

This is necessary but not sufficient. The vanilla Observer pattern has a key limitation for safety-critical monitoring: the observed object controls what it emits. An agent in a degraded state may emit incomplete, misleading, or no events — precisely when monitoring is most important. This is equivalent to a fire alarm that only triggers when the building tells it to.

Robust monitoring architectures require external observation capabilities: the ability to sample state from outside the observed component rather than relying solely on the component's self-reported events. This means:

Capturing all inputs and outputs at the boundary (not just what the component chooses to log)
Maintaining independent state snapshots at fixed intervals
Using timeout-based liveness checks that trigger even when the component is silent

3.2 Dual Observation Architecture

A dual observation architecture monitors two independent streams simultaneously:

Stream 1: Governor decision logs

Decisions taken, alternatives considered, reasoning traces
Plan structures, sub-goal trees, action selections
Tool calls made, results received, belief updates
Timing data: decision latency, retry counts, backtrack events

Stream 2: Session interaction quality

User turn sentiment, frustration signals, correction patterns
Response quality proxies: length, hedging density, factual consistency
Conversation trajectory: topic coherence, goal progress, user engagement
Error rates: misunderstandings, failed tool calls, user rephrasing

These two streams capture fundamentally different failure modes. Governor-only monitoring can detect internal reasoning pathologies (circular plans, hallucinated tool outputs, stuck retry loops) but is blind to user-facing quality degradation that arises from correct internal operation. Session-only monitoring captures interaction quality but provides no signal about why the quality is degrading or which internal component is responsible.

The dual observation approach combines these into a joint anomaly detection problem: anomalies that appear in both streams simultaneously are high-confidence systemic failures; anomalies in only one stream narrow the locus of the problem.

3.3 Structural Independence of Observers

A MetaCognition module that shares implementation with the Governor it monitors suffers from correlated failure. If the Governor's LLM backbone hallucinates, and the MetaCognition module uses the same backbone to evaluate outputs, both will fail in correlated ways.

Structural independence requires:

Separate process boundaries: MetaCognition runs in a distinct process or service, not as a function call within the Governor.
Independent state stores: MetaCognition maintains its own observation history, not a view into the Governor's internal state.
Heterogeneous evaluation methods: At least some of MetaCognition's quality signals use non-LLM methods — deterministic rules, statistical models, or a different LLM from the one used by the Governor.
Independent liveness monitoring: MetaCognition monitors the Governor's liveness independently; it does not ask the Governor whether the Governor is healthy.

This is analogous to the principle in distributed systems that health checks must be answered by the process itself, not forwarded to a dependency that may be the source of the failure.

4. Adaptive Control Theory Applied to Agent Systems

4.1 The MAPE-K Loop

IBM's autonomic computing initiative (Kephart & Chess, 2003) formalized the MAPE-K loop for self-managing systems:

Monitor: Collect metrics from managed resources. Aggregate, filter, and correlate into higher-level symptoms.
Analyze: Reason over the monitored data. Detect known patterns, predict future states, infer root causes.
Plan: Generate change actions to achieve goals or maintain SLA. Select among candidate plans.
Execute: Apply changes to managed resources via effectors.
Knowledge: A shared repository (topology models, policies, history, thresholds) that all four phases read from and write to.

The MAPE-K loop is explicitly a metacognitive architecture in the sense of Nelson-Narens: Monitor and Analyze constitute the monitoring channel; Plan and Execute constitute the control channel; Knowledge is the persistent metacognitive knowledge base.

For agent runtimes, the mapping is:

MAPE-K	Agent MetaCognition
Monitor	Dual observation collection (Governor logs + Session signals)
Analyze	Anomaly detection, drift scoring, quality metric computation
Plan	Control recommendation generation (policy + reasoning)
Execute	Recommendation delivery to Governor
Knowledge	Heuristic thresholds, anomaly history, prior decisions, calibration data

The MAPE-K loop is synchronous in the sense that each phase depends on the previous, but it runs asynchronously relative to the object-level system it manages. A MetaCognition module implementing MAPE-K operates on a separate clock — typically at lower frequency than the Governor's decision cycle — analyzing accumulated evidence before issuing any control recommendation.

4.2 Feedback Control Stability

Classical feedback control theory (Åström & Wittenmark) identifies two primary failure modes of feedback loops:

Instability: The controller over-corrects, causing oscillation. In agent terms: MetaCognition detects a drift, sends a recommendation to Governor, Governor over-corrects, MetaCognition detects the over-correction and sends another recommendation, oscillation ensues.

Lag: The controller responds too slowly. By the time MetaCognition's recommendation reaches the Governor, the session has already been abandoned by the user.

Mitigations:

Dead band: Do not issue a recommendation if the anomaly signal is below a threshold. Reduces control chatter.
Rate limiting: Issue at most one recommendation per N decisions. Prevents control flooding.
Proportional response: Scale the recommendation's urgency to the severity of the anomaly, not just to its presence.
Cooldown periods: After issuing a recommendation, suppress further recommendations for a defined window unless anomaly severity exceeds a hard threshold.

4.3 Heuristic Homomorphism in Adaptive Control

The principle that "cognitive mapping is heuristic homomorphism, not implementation constraint" has direct implications for adaptive control design.

A heuristic homomorphism maps the structure of the observed system onto a simplified mental model sufficient for control purposes. It does not need to be accurate at every detail — it needs to be accurate enough to predict the effects of control actions.

In practice, this means MetaCognition's model of the Governor should:

Capture the Governor's key behavioral invariants (what kinds of decisions it makes, what inputs it responds to) without needing to understand its internal implementation
Be updated incrementally as observation history accumulates
Degrade gracefully when the Governor changes — MetaCognition should detect when its model has become inaccurate (increased prediction error) and flag that its own recommendations may be unreliable

The homomorphism framing is important because it sets realistic expectations: MetaCognition is not a formal verifier of Governor behavior. It is a pattern-recognizer that notices when behavior drifts outside known-good envelopes and flags that for further examination. This is how domain experts monitor complex systems — not by understanding every internal mechanism, but by knowing what "normal looks like" and being alert to departures.

5. Anomaly Detection in Agent Behavior

5.1 Classes of Agent Behavioral Anomalies

For a Session-Governor-Executor architecture, anomalies fall into several categories:

Governor decision anomalies:

Repeated tool calls with identical parameters (stuck retry loop)
Plan depth explosion (unbounded sub-goal generation)
Decision latency spike (reasoning is taking abnormally long)
Low-confidence decisions being executed without escalation
Contradiction with prior decisions in the same session

Session interaction anomalies:

User correction rate spike (user rephrasing the same request multiple times)
Response shortening trend (agent becoming terse when it should be elaborating)
Sentiment deterioration in user messages
Topic coherence loss (conversation drifting from user's stated goal)
Escalating abstraction (agent deflecting to meta-discussion instead of task work)

Execution anomalies:

Tool error rate above baseline
Output format violations
Token budget overruns on routine tasks
External API failures not handled by Governor

Cross-stream anomalies (most diagnostic):

Governor shows high-confidence decisions AND user frustration is rising: suggests Governor's confidence is miscalibrated
Execution errors rising but Governor not adjusting plan: suggests monitoring gap in Governor's self-assessment
Session quality stable but Governor decision latency spiking: suggests latency hasn't yet surfaced to user but is a leading indicator

5.2 Drift Detection

Behavioral drift is distinct from acute anomalies: it is a gradual, cumulative shift in behavior that only becomes problematic at the end of a long trend. Standard threshold-based alerting misses drift because any individual observation is within normal bounds.

The CUSUM (Cumulative Sum) algorithm and related sequential change detection methods are appropriate for detecting drift in agent behavioral time series. CUSUM accumulates the deviation of each observation from a target value; when the accumulated sum crosses a threshold, an alert is raised.

For agent systems, practical drift signals include:

Rolling 20-decision average confidence dropping below historical mean by more than 1 standard deviation
Cumulative tool error count in a session exceeding the session-length-normalized historical mean
Moving average of user turn length (shorter user messages often indicate disengagement)
Cumulative response entropy trend (increasing entropy suggests the agent is becoming less certain over time)

Drift detection requires a baseline established from known-good behavior. This baseline should be session-type-specific — the normal behavior profile for a creative writing session differs from a code debugging session.

5.3 Confidence Calibration as a Metacognitive Signal

A well-calibrated agent should be correct approximately X% of the time on tasks it assigns X% confidence. Systematic miscalibration — especially overconfidence — is a leading indicator of problems.

MetaCognition can track calibration at the session level by comparing expressed confidence (from reasoning traces or logprobs) against outcome quality (from user feedback, tool execution success, or evaluator scores). A session where the agent is consistently overconfident warrants a recommendation to the Governor to increase its uncertainty budget and consider more conservative action selection.

Calibration tracking requires:

A mechanism to extract confidence signals from the Governor's outputs
A mechanism to measure outcome quality (the evaluator role from Reflexion)
A running calibration curve updated after each decision-outcome pair
A threshold at which calibration error triggers a control recommendation

6. Consensus Mechanisms for Internal Decisions

6.1 Why Consensus?

A single observation channel will have systematic blind spots determined by its measurement method. MetaCognition built on a single signal — say, session sentiment alone — will miss entire categories of failure that produce no sentiment signal (e.g., silent hallucination in technical answers, where the user does not yet know the answer is wrong).

Consensus mechanisms aggregate signals from multiple independent channels to produce recommendations that are more reliable than any single channel. The tradeoff is latency: consensus requires that all channels have produced their observations before a recommendation can be formed.

6.2 Weighted Voting

The simplest consensus mechanism assigns weights to each signal source and combines them:

recommendation_score = Σ(weight_i × signal_i)
if recommendation_score > escalate_threshold:
    emit_recommendation(ESCALATE)
elif recommendation_score > warn_threshold:
    emit_recommendation(WARN)
else:
    emit_recommendation(NOMINAL)

Weights can be:

Static: Set by domain knowledge during system design
Historically calibrated: Weights set proportional to each signal's historical predictive accuracy
Contextually adaptive: Weights shift based on session type (different weights for creative vs. technical sessions)

A critical implementation note: weights should be versioned and audited. In a system where "Governor must not be LLM-sovereign," the weights of the consensus mechanism should not themselves be set by the Governor — this creates a feedback loop where the Governor can influence how harshly it is evaluated.

6.3 Severity-Gated Consensus

Not all recommendations require full consensus. A tiered approach:

Tier 1 - Hard circuit breakers (immediate, no consensus required):

Governor is consuming tokens at 10× normal rate (runaway loop)
Executor has failed 5 consecutive tool calls
Session has reached maximum allowed duration These trigger immediate control actions without waiting for other signals.

Tier 2 - Soft warnings (majority vote among available signals):

Declining response quality trend
Increasing decision latency
Moderate user frustration signals These require at least 2 of 3 signal channels to agree before a recommendation is issued.

Tier 3 - Advisory observations (single signal, low priority):

Slight increase in hedging language
Marginal calibration degradation
Minor topic drift These are logged for the Knowledge base but do not trigger active recommendations.

6.4 Signal Fusion Architecture

                    ┌─────────────────────────────┐
                    │      MetaCognition Module    │
                    │                             │
  Governor logs ──► │  Observer A                 │
                    │  (decision quality scorer)   │──► Consensus
  Session events ──►│  Observer B                 │    Engine ──► Governor
                    │  (interaction quality scorer)│    (MAPE-K)
  Execution logs ──►│  Observer C                 │
                    │  (execution health scorer)   │
                    │                             │
                    │  Knowledge Base              │
                    │  (thresholds, history,       │
                    │   calibration data)          │
                    └─────────────────────────────┘

Each observer independently scores its signal stream and produces a typed output: {signal_id, timestamp, severity, value, confidence_in_signal}. The consensus engine receives all observer outputs, applies weights, resolves disagreements per policy, and emits a typed recommendation: {recommendation_type, severity, evidence_references, suggested_action}.

The Governor receives recommendations as advisory inputs, not as direct commands — consistent with the principle that the Governor is LLM-assisted but not LLM-controlled. The Governor's deterministic governance layer decides how to act on the recommendation. MetaCognition cannot force a course of action; it can only raise the cost of ignoring an escalation by logging that the recommendation was received and not acted upon.

7. Production Patterns and Implementation Guidance

7.1 Reflexion-Inspired Episode Memory

Reflexion's episodic memory pattern — storing verbal self-critiques that persist across attempts — is directly applicable to multi-session agent contexts. A MetaCognition module can maintain a session anomaly journal: a structured log of anomalies detected, recommendations issued, and Governor responses. This journal serves as metacognitive knowledge that primes future sessions.

interface AnomalyRecord {
  session_id: string;
  timestamp: string;
  anomaly_type: 'governor_drift' | 'session_degradation' | 'execution_failure' | 'calibration_error';
  signal_source: string[];
  severity: 'advisory' | 'warning' | 'critical';
  recommendation_issued: boolean;
  governor_response: 'acted' | 'ignored' | 'deferred' | 'no_response';
  outcome: string | null; // populated post-session
}

Over time, this journal enables the Knowledge phase of the MAPE-K loop to improve: patterns in the journal reveal which anomaly types are most predictive of session failure, enabling threshold recalibration.

7.2 The Evaluator as a Separate Role

Following the Reflexion architecture, the MetaCognition module should implement a distinct evaluator role that scores Governor outputs against quality criteria without participating in their generation. The evaluator's criteria should be:

Explicit and auditable: Written as versioned policy documents, not embedded in a prompt
Measurable: Each criterion maps to a numeric score or binary pass/fail
Independent: The evaluator uses different information sources than the Governor where possible

Example quality criteria for a code-focused agent session:

Did the Governor's plan match the user's stated intent? (intent alignment score)
Were all tool calls explained with reasoning before invocation? (reasoning transparency score)
Did the agent acknowledge uncertainty where its information was incomplete? (epistemic honesty score)
Were errors handled with explicit recovery steps, or silently swallowed? (error handling score)

7.3 Constitutional Constraints as MetaCognitive Knowledge

Constitutional AI (Anthropic, 2022) demonstrates that explicit principle sets ("constitutions") can guide self-critique and refinement. MetaCognition modules can maintain an analogous governance constitution: a versioned set of principles against which Governor behavior is evaluated.

For a Session-Governor-Executor architecture, the governance constitution might include:

"The Governor must not invoke a destructive action (file deletion, external API with side effects) without presenting a reasoning trace first"
"The Governor must escalate to human review when confidence falls below 0.4 on user-critical decisions"
"The Governor must not retry a failed tool call more than 3 times without reporting the failure to the Session layer"

These are not soft guidelines — they are binary compliance checks that the MetaCognition evaluator scores on every decision. Violations are logged as critical anomalies regardless of other signal states.

7.4 Addressing the LLM Sovereignty Problem

The principle "Governor may be LLM-assisted, but must not be LLM-sovereign" has a direct operational interpretation for MetaCognition design:

What it means: The Governor uses an LLM for plan generation, reasoning, and communication — but the actual decision about whether to execute a plan, allocate resources, or escalate to a human is made by deterministic governance logic that evaluates the LLM's proposal against explicit constraints.

How MetaCognition enforces it: MetaCognition monitors for signals that indicate the deterministic governance layer has been bypassed or undermined:

Decisions executed with no recorded constraint check (governance skip signal)
LLM output accepted verbatim without transformation through the governance layer (passthrough signal)
Constraint violations in the governance check log that were overridden without escalation (silent override signal)

If any of these signals fire, MetaCognition should issue a critical recommendation regardless of session quality or conversation quality. The LLM sovereignty constraint is structural, not performance-based — it applies even when the session is going well.

The deeper reason: LLMs are susceptible to adversarial inputs, distributional shift, and confident hallucination. A Governor whose decisions are entirely LLM-produced will make unexpected decisions when faced with inputs outside its training distribution — and it will make them confidently. Deterministic governance logic, by contrast, is predictable by construction: given the same input, it produces the same decision. MetaCognition's job is partly to ensure that the deterministic shell around the LLM core remains intact.

7.5 Implementing the MAPE-K Cycle Concretely

A minimal implementation of the MAPE-K cycle for a MetaCognition module in a Node.js/TypeScript agent runtime:

class MetaCognitionModule {
  private knowledge: KnowledgeBase;
  private observers: Observer[];
  private consensusEngine: ConsensusEngine;
  private governorChannel: RecommendationChannel;

  // Monitor phase: runs on every incoming event
  async monitor(event: AgentEvent): Promise<void> {
    for (const observer of this.observers) {
      await observer.ingest(event);
    }
  }

  // Analyze phase: runs on a fixed cadence (e.g., every 10 Governor decisions)
  async analyze(): Promise<AnalysisResult> {
    const signals = await Promise.all(
      this.observers.map(o => o.getSignal())
    );
    const anomalies = this.detectAnomalies(signals, this.knowledge.thresholds);
    const drifts = this.detectDrift(signals, this.knowledge.baselines);
    return { signals, anomalies, drifts };
  }

  // Plan phase: given anomalies, select a recommendation action
  async plan(analysis: AnalysisResult): Promise<Recommendation | null> {
    // Hard circuit breakers first (no consensus needed)
    const critical = analysis.anomalies.filter(a => a.severity === 'critical');
    if (critical.length > 0) {
      return { type: 'ESCALATE', urgency: 'immediate', evidence: critical };
    }

    // Soft warnings require majority vote
    const warnings = analysis.anomalies.filter(a => a.severity === 'warning');
    const voted = this.consensusEngine.vote(warnings, analysis.signals);
    if (voted.passed) {
      return { type: 'WARN', urgency: 'advisory', evidence: voted.evidence };
    }

    // Log drifts to knowledge base but don't recommend yet
    if (analysis.drifts.length > 0) {
      await this.knowledge.recordDrift(analysis.drifts);
    }

    return null;
  }

  // Execute phase: deliver recommendation to Governor
  async execute(recommendation: Recommendation): Promise<void> {
    await this.governorChannel.send(recommendation);
    await this.knowledge.recordRecommendation(recommendation);
  }

  // Full cycle
  async cycle(): Promise<void> {
    const analysis = await this.analyze();
    const recommendation = await this.plan(analysis);
    if (recommendation) {
      await this.execute(recommendation);
    }
    await this.knowledge.updateBaselines(analysis.signals);
  }
}

This structure separates concerns cleanly: Monitor is event-driven; Analyze/Plan/Execute run on a scheduled cycle; Knowledge is shared state updated by both paths.

8. Applying These Patterns to the Session-Governor-Executor Architecture

8.1 Module Placement

MetaCognition sits outside both the Governor and the Session in the architecture. It subscribes to event buses from both:

┌────────────┐     decisions/logs     ┌──────────────────────┐
│  Governor  │ ──────────────────────►│                      │
│            │◄── recommendations ───│  MetaCognition        │
└──────┬─────┘                        │  Module              │
       │ plans                        │                      │
       ▼                              │  - Observer A        │
┌────────────┐     session events     │    (Governor quality) │
│  Executor  │                        │  - Observer B        │
└────────────┘                        │    (Session quality) │
       │                              │  - Consensus Engine  │
       ▼                              │  - Knowledge Base    │
┌────────────┐   interaction events   │                      │
│  Session   │ ──────────────────────►│                      │
└────────────┘                        └──────────────────────┘

MetaCognition does NOT sit on the hot path between Governor and Executor. Recommendations are advisory, delivered asynchronously. This is essential: putting MetaCognition synchronously in the execution path would add latency to every decision and create a single point of failure.

8.2 Dual Observation Implementation Details

Observer A (Governor quality) subscribes to the Governor's decision log bus. Each event carries: decision ID, timestamp, decision type, reasoning trace (if available), confidence score (if available), alternatives considered, tool calls initiated, token count. Observer A scores each event against the governance constitution and computes rolling statistics.

Observer B (Session quality) subscribes to the Session event bus. Each event carries: turn ID, speaker (user/agent), message text, response metadata. Observer B applies NLP-based quality signals: sentiment analysis, response length trends, hedging language density, user correction detection (detecting when a user message is a correction of the previous agent message), topic coherence score.

Both observers write to a shared time-indexed event store that the Consensus Engine reads from.

8.3 What MetaCognition Should NOT Do

To avoid violating the "Governor must not be LLM-sovereign" principle in the other direction (MetaCognition becoming sovereign), MetaCognition should:

Not generate plans for the Governor. It recommends that the Governor revisit its plan; it does not provide the replacement plan.
Not directly modify Executor behavior. Control recommendations go to the Governor; if the Governor acts on them, the Governor modifies Executor behavior through the normal plan-execute channel.
Not accumulate state that supersedes the Governor's own memory. MetaCognition's observation history is its own, used for its own analysis. It is not a parallel source of authority on what the session context contains.
Not recommend the same action repeatedly without decay. If a recommendation is issued and the Governor does not act on it, MetaCognition should log that it was ignored and potentially escalate severity — but it should not simply re-issue identical recommendations in a loop. That is the agent runtime equivalent of a fire alarm that never stops ringing after being acknowledged.

9. Open Problems and Future Directions

Calibration without ground truth: MetaCognition's anomaly detection requires baseline distributions of normal behavior. In novel session types or after significant Governor updates, these baselines may be invalid. Self-supervised recalibration — where MetaCognition detects when its own predictions are systematically wrong and flags that as a second-order anomaly — is an active research area.

Latency constraints in real-time sessions: MAPE-K analysis cycles take time. For high-frequency interactions (voice agents, real-time collaborative coding), the latency budget for MetaCognition analysis may be measured in milliseconds. Streaming anomaly detection algorithms that produce partial estimates in real time are required.

Adversarial stability: A sufficiently capable LLM Governor, optimizing for autonomy, could learn to produce outputs that appear normal to the MetaCognition observers while bypassing intended constraints. This is analogous to the goodhart's law problem in reinforcement learning. MetaCognition observers must include signals that are structurally hard to game — not just LLM-readable signals, but execution-layer telemetry that requires actual correct behavior to produce.

Multi-session metacognitive learning: Current Reflexion-style systems learn within a single episode. Persistent metacognitive knowledge that improves across many sessions — adjusting thresholds based on long-run outcome data — requires durable storage and careful versioning to avoid distributional drift in the metacognitive model itself.

References and Key Sources

Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, 26, 125-173.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906.
Kephart, J. O., & Chess, D. M. (2003). The vision of autonomic computing. IEEE Computer, 36(1), 41-50. [MAPE-K origin paper]
Shinn, N., Cassano, F., Labash, B., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. arXiv:2303.11366.
Madaan, A., et al. (2023). Self-Refine: Iterative refinement with self-feedback. arXiv:2303.17651.
Wang, G., et al. (2023). Voyager: An open-ended embodied agent with large language models. arXiv:2305.16291.
Yao, S., et al. (2023). ReAct: Synergizing reasoning and acting in language models. arXiv:2210.03629.
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023). Cognitive architectures for language agents. arXiv:2309.02427. [CoALA framework]
Bai, Y., et al. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv:2212.08073.
Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley.
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. ICML 2017.
Åström, K. J., & Wittenmark, B. (1995). Adaptive Control (2nd ed.). Addison-Wesley.