Orchestrating work on this repo

How a main session builds OpenTokenMini by delegating to subagents in short loops, using its own telemetry (otm-live MCP, 06-optional/mcp-server/live.py) to manage context, time and quota. Rules below come from measured sessions; see Evidence for numbers and method.

Agents (.claude/agents/)

otm-worker.md asks for ≤60k because on the measured machine a ctx-guard hook (~/.claude/hooks/ctx-guard.py, user config, not in this repo) warns subagents at 60k and denies tools at 80k, except writing a handoff. A worker that hits it stops mid-task.

Worker prompt contract

Every worker prompt today that landed cleanly had the same six parts, ~1.5-2.3k chars:

1. Role + read list: otm-worker.md, root AGENTS.md, the component's AGENTS.md, and the 2-4 files it will copy or touch. Name them; don't say "explore".
2. Isolation: git worktree add /tmp/otm-wt-<name> -b wt/<name>, work only there.
3. Ownership: "You own ONLY . Do not edit ( is editing them in parallel)." Naming the parallel neighbour is what keeps workers out.
4. Goal + 2-4 checkable criteria, at least one of them a real run on real data with a number to report.
5. Hard rules: never delete (todelete/), no push, session line in the commit message.
6. Return contract: "Return ≤N words: hash, branch, , unverified." Ask for "not done / out of scope" explicitly; that list is the next batch.

One item = one deliverable set that fits ≤60k. The one prompt today with six deliverables (code, HTTP endpoint, tests, MCP registration, docs, commit; 5.2k chars) is the one that hit the guard and needed a second worker.

Ownership and merging

• One worktree per worker, one owner per directory. The orchestrator creates worktrees up front (for n in ...; do git worktree add -q /tmp/otm-wt-$n -b wt/$n; done) or the worker does it as step 1.
• Shared index files (a component table in a parent AGENTS.md, repo-map.md Status, root AGENTS.md) belong to nobody during a batch. The orchestrator adds the row after merging (sed -i '/^|prev\//a | ...', commit), or one later "reconcile" worker owns all of them.
• Merge on each completion notification: git merge --no-edit wt/<name>, then run that stage's tests (full loop after a pipeline or shared-code merge). Don't wait for the batch.
• A 1-line integration bug found after a merge: fix it yourself and commit. Anything larger: a new worker.
• Don't edit a file a running worker owns, even a one-line fix; use a different route or wait.

Contracts between parallel workers

If worker B consumes what worker A produces in the same batch, write the interface into both prompts: file name, location, keys, and what "missing/skipped" looks like. Without it B guesses. Today the pipeline worker wrote readout.json; the parallel dashboard worker, told only "read those summaries as optional files", guessed deployments_summary.json, and a reconcile worker ($0.55) had to rewire it. The reconcile prompt that fixed it pasted the exact keys; that is the pattern.

When the interface is not decided yet, run A first and paste its result into B's prompt rather than parallelising.

Loop

1. overview (otm-live): quota left, live sessions, subagents with ctx/cost, overlaps. Stop delegating when 7d utilization reaches the human's target or 5h is near its limit.
2. Pick items from Status in repo-map.md, the human's list, or the previous batch's "not done" lines. Group into a batch of items with disjoint owned paths.
3. declare the paths; who_touches before starting a worker on paths near a live one.
4. record phase=start (item, agent, topology), spawn workers in background with the contract above.
5. On each notification: merge, test, add index rows, record phase=end (tokens, commit, tests_pass, accepted). Reviewer only for shared code, data contracts or public surfaces.
6. A worker that stopped at the guard: spawn a new one with the handoff path (/tmp/claude-1000/handoffs/<id>.md), the resolved facts it asked about, and only the pending steps. A reviewer FAIL: one fix round with the defects pasted; second FAIL, park it and tell the human.
7. Next batch from the "not done" lines. Compact the main session between batches (/compact <short handoff>); today's orchestrator stayed ≤73k with 2 compacts over 12 merges.

The main session reads summaries, git show --stat, test tails and otm-live, not implementation files.

What wasted tokens

• Workers past 100k ctx. Across 152 completed workers, the 10 that passed 100k had median cost $9.28 vs $0.55 under 60k and $1.10 at 60-100k; in the session without the guard they were 10 of 59 workers and $171 of $233.
• Long multi-deliverable prompts. Prompt length correlates with cost (Spearman 0.40, n=150 Opus workers): median $0.52 under 1.2k chars, $0.71 at 1.2-2k, $1.31 at 2-3k, $1.90 above 3k. Confounded with task size; the fix is splitting the item, not trimming words.
• Result size limits are overshot but cheap: "≤120 words" returned median 160 words today, 287 in a session that asked for "~250". Result length does not track worker cost (Spearman 0.03). It costs orchestrator context only; keep the limit and ask for numbers, not prose.
• A planner returning 3,110 words (one session) cost the orchestrator more context than ten workers' results.
• Expensive orchestrator model with long context: one session ran the orchestrator on Fable to 258k peak, $61; today's Opus orchestrator ran $2.95 for 12 landed items.

Delegation ledger

$OTM_DATA_DIR/delegations.jsonl, written only by the otm-live record tool (the code in live.py is the truth):

• record phase=start item=<id> [agent desc topology] before launching the worker: stores the start ts and quota_before in <data dir>/live/pending.json.
• record phase=end item=<id> [tokens commit tests_pass accepted] after it returns: appends one line and clears the pending entry. Fields given at end override those from start.

Line: {"ts", "agent", "desc", "item", "topology", "quota_before", "quota_after", "tokens", "commit", "tests_pass", "accepted"}. ts is the start time; quota_* map each window to utilization percent ({"5h": 41, "7d": 63}, null when quota is unreadable); missing fields are null. overview shows the last lines.

topology is a short label of how the item was split (single, worker+review, parallel:3, scout+worker). Comparing tokens, tests_pass and accepted per topology is how the pattern gets tuned; new numbers in this doc come from this file or from the method below.

Evidence

Four Claude Code sessions on one machine, 172 background subagents, 152 completed (last text block ≥60 words; the rest were interrupted by the user or still running when measured, 2026-10-25).

Today (e3073449): batches of 1, 3, 6, 2 and 10 workers; 11 worktree merges, 0 conflicts (CONFLICT and Automatic merge failed absent from tool results); rework: 1 continuation worker (guard stop), 1 reconcile worker (missing contract), 1 one-line fix by the orchestrator; 3 index rows added by the orchestrator. Workers ran 1-9 min wall time. The data does not show whether batches of 6 or 10 are better; both merged cleanly.

Method (scripts kept outside the repo; reproduce from these definitions):

• Source: ~/.claude/projects/<project>/<sid>.jsonl and <sid>/subagents/agent-*.jsonl + .meta.json.
• Assistant messages deduplicated by message.id. Peak ctx = max of input + cache_read + cache_creation tokens per message.
• Cost = usage × 02-measure/schemas/prices.json via schemas/cost.py price_for, cache writes split by 5m/1h TTL when present. List prices; opus-5-5 rates are fitted, so dollar figures are estimates; ratios between workers are the reliable part.
• Prompt = first user message of the subagent; result = words in its last non-empty text block.
• Merges, conflicts and rework from the orchestrator's Bash tool_use inputs, tool results and task-notification summaries.