@deepseek-ai/dsh-llm-fallback

English | 中文

Self-healing cross-provider model fallback. When a model request fails with a healable code and in-place retries are spent, the next fallback in the chain is selected and the retry is re-bound to it through the same agent/request waterfall the runtime model picker uses — the session continues on a healthy model instead of dying.

How it works

The plugin listens on two agent-scoped waterfalls:

• agent/request-error — the recovery decision. Mounted after @deepseek-ai/dsh-llm-retry, it only sees failures that policy delegated: a spent retry budget, an unretryable code in healOn, or an over-cap provider delay. It selects the first live, not-yet-tried fallback, appends a durable non-surface llm/fallback event, and returns { kind: 'retry' }.
• agent/request — the re-bind. While a heal is in force, the request config is re-written to the fallback route; a configured reasoningEffort travels with it, and an absent one clears any inherited effort (the same semantics the runtime model picker applies).

An always-mode primary is respected by delegation: it never settles on this plugin, so chains should watch routes with normal-mode policies. An explicit runtime model selection outranks healing — the first picked attempt self-defuses the heal.

Every decision is durable: the llm/fallback event records the failed route, the failure facts, and the selected fallback, so a remote renderer can surface the heal without loading this runtime.

Configuration

The composition entry supplies the base; the llm-fallback section of the user settings document replaces it wholesale on the next failure (no restart):

llm-fallback:
  chains:
    - primary: { provider: z-ai-glm-53-flash }
      fallbacks:
        - { provider: openrouter, model: nvidia/nemotron-3-ultra-550b-a55b:free }
        - { provider: openrouter, model: minimax/minimax-m3:free }
  healOn: [SERVER, TIMEOUT, TRANSPORT, RATE_LIMIT, EMPTY_RESPONSE, INVALID_REQUEST]
  sticky: true

Field	Meaning
chains	Ordered healing chains; an empty list keeps the plugin dormant.
chains[].primary	The watched route. model absent watches every model on the provider route.
chains[].fallbacks	Ordered targets; each needs provider and model, with an optional reasoningEffort (off/low/high/max).
healOn	Failure codes that trigger a heal; anything else delegates downstream.
sticky	true keeps the heal for the agent's remaining life; false re-binds the agent's own route when the healing turn closes.

A malformed section is refused where it is written — the settings write fails with the offending chain and field named.

Installation

While the DeepSeek Harness packages are pre-release (not yet on npm), vendor this repository into a DSH checkout:

git clone https://github.com/force-push/dsh-llm-fallback.git packages/llm/llm-fallback
pnpm install && pnpm run build:lib:host

After the harness packages publish, npm install dsh-llm-fallback replaces the vendored drop-in.

Mounting

Add the plugin after llm-retry in the host composition (or a profile patch):

- id: llm-fallback
  name: '@deepseek-ai/dsh-llm-fallback'

The fallback routes must be registered adapters — configured provider routes of @deepseek-ai/dsh-llm-pi-ai or any other adapter family. Entries naming a route that is not currently registered are skipped at heal time.

Semantics and limits

• Retries stay in place first. The chain is consulted only after the watched route's own retry policy is spent for that step.
• Some failures heal immediately. Codes an in-place retry cannot fix (INVALID_REQUEST-class refusals) heal on the first occurrence.
• Context capacity is not checked. A fallback model smaller than the conversation will fail on context; compose with compaction deliberately.
• Agent turns are the healing boundary — direct ctx.llm.stream() consumers are single-attempt, matching dsh-llm-retry.
• Healing masks misconfiguration. A bad effort map or endpoint surfaces as a healed-away failure; the durable events are the audit trail.