dsh-trajectory-ablation

A debugger that finds the actual cause of an agent failure, instead of guessing.

A plugin for DeepSeek Harness that reconstructs exactly what your agent saw before a given step, diffs that against another step, and — most importantly — proves which piece of context actually caused a decision by removing it and replaying the same model call for real.

See trajectory-ablation.md for the full design rationale.

Why

When an agent does something wrong, most tools show you a transcript and let you guess, or ask the agent to explain itself — which is just the same model generating a plausible-sounding story about its own past decision, not a verified fact. This plugin replaces both with something checkable:

1. Reconstruct — the exact, complete list of everything the model saw before a step, grouped by source. No model calls, just reading the log.
2. Diff — what changed between two steps: added, removed, compressed, or reordered. No model calls.
3. Ablate — remove one piece of context at a time and replay the model call for real, k times per piece. If removing something changes the decision, it's a cause. If it doesn't, it's bystander context.

Install

Mount it in your DeepSeek Harness profile ($DSH_HOME/cordis.patch.yml):

- insert:
    - id: trajectory-ablation
      name: 'dsh-trajectory-ablation'

Then pnpm install in your profile directory and restart. See cordis.patch.yml.example for a fuller walkthrough.

Usage

Once mounted, three commands are available in any session:

/context-reconstruct <turn> <step>            — what did the model see before this step?
/context-diff <turnA> <stepA> <turnB> <stepB> — what changed between two steps?
/ablate <turn> <step> [k]                      — which block actually caused this decision?

Or use it programmatically — reconstructContext and diffContext need only a session's event log (no live model connection required); ablateStep needs a live ctx.llm:

import { reconstructContext, diffContext, ablateStep } from 'dsh-trajectory-ablation'

Try it without a live session

npm install
npm run demo

Runs the full pipeline against a scripted example (a stale README misleading an agent into editing the wrong file) with no API key required.

Development

npm install
npm run build       # compile
npm run typecheck   # type-check only
npm test            # run the test suite

License

MIT — see LICENSE.