Work / Case study

Agent Telemetry — Closed-loop memory & governance

Across 434 multi-turn sessions in Claude Code, OpenAI Codex, Antigravity, and Cursor, autonomous coding models repeatedly failed in the same five ways. This case study documents an involuntary telemetry, Thrash Index, and rule-synthesis engine built to turn developer friction into deterministic behavioral constraints.

Work with GaroDiscuss Agent Governance

The numbers across 4 ecosystems

Why I built it

Working full-time with autonomous coding agents reveals a consistent pattern: model capabilities are remarkable, but cross-session operational memory is zero. An agent that breaks UI touch targets or creates Next.js hydration loops at 2:00 PM will make that exact same mistake at 8:00 PM in a fresh session.

Manual prompt engineering and constant supervision cause severe cognitive fatigue. I needed a system that observed every model turn invisibly, quantified when the model was flailing in unproductive loops, and automatically extracted permanent constraints to protect the codebase.

The closed-loop lifecycle

• HarvestInvoluntarily extract raw session logs and SQLite stores from Claude Code, OpenAI Codex, Antigravity, and Cursor via macOS background daemons without developer memory burden.
• NormalizeStream heterogeneous vendor schemas into a single unified JSONL format tracking tokens, tool executions, chain-of-thought steps, and message turns.
• MeasureCalculate the Thrash Index (TI) to quantitatively score session efficiency, tool failure rates, and model flailing before human fatigue sets in.
• ClusterMap user friction signals (reverts, undo commands, broken layouts, CI failures) into 5 deterministic root failure categories.
• EnforceAuto-synthesize concise 1-line behavioral rules and promote them directly into AGENTS.md and CLAUDE.md to prevent repeat failures.

The Thrash Index (TI)

To detect when an agent is flailing rather than making progress, the pipeline calculates the Thrash Index: the ratio of failed tool executions, user interventions, and file rollbacks relative to total session turns.

Sessions scoring above 15% indicate severe context exhaustion or prompt derailment. Rather than continuing to burn tokens on speculative repairs, the system triggers automated session compaction and alerts the developer.

How it’s built

• Forensics EnginePython 3.12, SQLite3 read-only URI drivers, streaming JSONL parsers, and regex-driven failure clustering.
• Involuntary DaemonsmacOS launchd plists running memory-autopilot and telemetry harvesting every 30 minutes with zero cloud latency.
• Review InterfaceNon-blocking CLI review banner (work-brief) with shift-aware macOS text-to-speech audio for night-shift development.
• Task GovernanceStrict git worktree isolation (agent-task harness), automated unit verification, and clean-room privacy guarantees.
• Rule SynthesizerDeterministic rule extraction (synthesize_rules.py) converting high-thrash patterns into live system prompt constraints.

What it demonstrates for engineering teams

• Systems & FDE judgmentReverse-engineered undocumented local storage formats across 4 commercial agent ecosystems to build an observable telemetry plane.
• Closed-loop memory designSolved the core limitation of frontier coding models: individual capability without cross-session operational memory.
• Cognitive accessibilityBuilt around ADHD and executive function realities: ambient audio briefings, 5-second auto-skip prompts, zero manual logging.
• Clean-room data ethicsStrict isolation between sensitive developer session logs (kept local/gitignored) and clean synthetic test suites.

Boundaries & honest non-claims

• Local workstation harness, not a multi-tenant commercial SaaS product.
• macOS-native implementation using launchd and local system primitives; Linux/Windows parity is not promised as shipped.
• Developer session transcripts, tokens, and prompt logs remain 100% private in local storage.
• Governance rules constrain agent behavior; human developer retains final code review and push responsibility.