@deepseek-ai/dsh-llm-local

English | 中文

OpenAI-compatible local model adapter for the harness LLM seam: direct fetch + SSE (framed by eventsource-parser) against any server that speaks the OpenAI chat-completions dialect — Ollama (/v1), the llama.cpp server (/v1), Microsoft Foundry Local, LM Studio, vLLM, and OpenAI-compatible gateways, local or remote. No DeepSeek-specific wire fields are sent, so the same adapter serves every endpoint; reasoning-capable servers that emit reasoning_content still stream it as harness reasoning blocks.

The package owns the provider routes ollama, llamacpp, and foundry-local as shipped presets, plus every route you declare by hand. The plugin is mounted dormant in the product: it shows up on the Models page, but zero routes serve requests until a llm-local: settings section (written by the web Models page or by hand) supplies provider profiles.

About this repository — this is a standalone source snapshot of the @deepseek-ai/dsh-llm-local plugin, extracted from the DeepSeek Harness monorepo (packages/llm/llm-local). It is a Cordis plugin for the harness and is built and tested inside that monorepo; the package.json dependencies and tsconfig.json references therefore resolve against the harness workspace. The canonical upstream is https://github.com/deepseek-ai/deepseek-harness.

Installation

dsh-llm-local is a plugin for DeepSeek Harness. It is wired into the harness's base bundle and mounts dormant: the ollama, llamacpp, and foundry-local routes appear on the Models page but serve no requests until a llm-local: settings section supplies profiles.

As an end user — install the harness, then configure the plugin (no separate install of this repository is needed):

npx @deepseek-ai/dsh web

Open the web UI at http://127.0.0.1:3080, go to Settings → Models, and edit one of the local provider rows, or write a llm-local: section in $DSH_HOME/settings.yaml (see Configuration).

The npx command fetches the published harness build, so that build must include this package (and its base-bundle entry) for the plugin to be present. Until that release lands, build from source instead.

As a developer — build inside the harness monorepo. This package uses workspace:^ dependencies and monorepo tsconfig.json references, so it is not built standalone:

git clone https://github.com/deepseek-ai/deepseek-harness.git
cd deepseek-harness
# place this repository's src/ and tests/ under packages/llm/llm-local/
pnpm install
pnpm run build
pnpm dsh web

Then configure the plugin exactly as an end user would.

Quick start

1. Restart the dsh server (the web GUI process). The plugin is loaded at boot; a running server from before the plugin existed will silently ignore llm-local: settings — there is no error, the section just does nothing until the restart.
2. Configure the provider either in the web UI (Settings → Models → the provider row → Edit → Base URL + Fetch available models + Apply) or in $DSH_HOME/settings.yaml (see below).
3. In a new conversation, switch the model: open the model picker (/model) and choose the local provider + model id. The default provider of new sessions is still deepseek-official until you pick another one.

Where settings.yaml lives

The settings document is settings.yaml inside the harness home:

• Default home: ~/.dsh — on Windows that is C:\Users\<you>\.dsh, so the file is C:\Users\<you>\.dsh\settings.yaml.
• If DSH_HOME is set, the file is $DSH_HOME/settings.yaml.
• The path shown by the web UI's settings page, or set in a composition cordis.yml (settings-file.config.path), wins over both.

The file is hot-reloaded: an external edit is picked up within ~100 ms and the very next model request uses the new values — no restart needed for edits after the plugin is loaded once.

Configuration

The plugin registers the llm-local settings namespace. The providers dict keys ARE the provider routes: ollama, llamacpp, and foundry-local are shipped presets (endpoint and display name inherited), any other key is a hand-declared endpoint and must name a baseURL.

Minimal working example (Ollama)

llm-local:
  providers:
    ollama:
      models:
        - id: qwen3:8b
          contextWindow: 32768

That is everything: the preset endpoint http://127.0.0.1:11434/v1 is inherited, no API key is needed, and the route registers the moment the file is saved. Request with provider: ollama, model: qwen3:8b.

Full example with every field

llm-local:
  providers:
    # Shipped preset: endpoint defaults to http://127.0.0.1:11434/v1.
    ollama:
      models:                        # optional; discovery can fill these
        - id: qwen3:8b
          contextWindow: 32768
      # defaultContextWindow: 128000   # fallback capacity for unlisted models
      # maxTokens: 32768               # default per-request output cap
      # usageInStream: true            # stream_options.include_usage
      # vision: false                  # image input for vision models
      # streamIdleTimeoutMs: 300000    # per-read idle budget
    # Shipped preset for the llama.cpp server.
    llamacpp:
      baseURL: http://127.0.0.1:8080/v1
      models:
        - id: qwen2.5-coder-14b-instruct-q4_K_M
          contextWindow: 32768
    # Shipped preset for Microsoft Foundry Local.
    foundry-local:
      baseURL: http://127.0.0.1:53415/v1   # confirm the port in `foundry local` output
      models: []
    # Hand-declared route: any OpenAI-compatible endpoint.
    my-gateway:
      displayName: My Gateway
      baseURL: http://127.0.0.1:1234/v1
      apiKeyEnv: MY_GATEWAY_KEY   # optional; local servers usually need none
      vision: true                # optional; image input for vision models
      usageInStream: true         # optional; stream_options.include_usage
      maxTokens: 32768            # optional default per-request output cap
      defaultContextWindow: 128000  # optional fallback context capacity
      streamIdleTimeoutMs: 300000   # optional per-read idle budget
      retryPolicy:                  # optional; normal defaults when omitted
        mode: normal
        maxRetries: 2
      models:
        - id: gpt-oss-120b
          contextWindow: 131072
          maxTokens: 32768

Per-provider fields:

• baseURL — the OpenAI-compatible root (/chat/completions and /models are appended). Presets default to http://127.0.0.1:11434/v1 (Ollama), http://127.0.0.1:8080/v1 (llama.cpp), http://127.0.0.1:53415/v1 (Foundry Local — confirm the exact port in foundry local output). Required for hand-declared routes.
• apiKeyEnv — credential reference resolved per request through ctx.credentials, then the environment. Omitted means unauthenticated — the local default; Ollama, llama.cpp, and Foundry Local need no key. Once named, a miss fails loud with MISSING_CREDENTIAL.
• models — advisory catalog exposed to selectors and discovery; requests accept unlisted ids too. Each entry: id, optional name/description/contextWindow/maxTokens/vision. models: [] advertises none.
• vision (profile or per-model) — declares image input. Image blocks serialize as image_url data-URL parts through the durable attachment service; a text-only model rejects them with UNSUPPORTED_CONTENT.
• usageInStream — default true (stream_options.include_usage). Disable for an older llama.cpp build that rejects the field.
• maxTokens, defaultContextWindow, streamIdleTimeoutMs, retryPolicy — same semantics as the DeepSeek adapter.

Configuring from the Models page

The web Models settings page knows this family: each provider row (the three presets are always listed, dormant or not) opens an editor with API key (optional — leave blank for unauthenticated local access), Base URL (pre-filled from the preset), and a model catalog with a Fetch available models button that interrogates GET {baseURL}/models — for a preset route it works even before you type a URL. "Add provider" offers the three dormant presets. Hand-declared routes still need settings.yaml (there is no per-route protocol selector for local routes because they all speak the same dialect).

Dynamic configuration (settings + credentials)

Connection facts are not frozen at load. resolveAdapterOptions is the one explicit resolve step, and the adapter re-reads facts through a thunk once per operation: base URL, catalog, usage reporting, vision, output cap, and idle budget all take effect on the next request, while an in-flight stream keeps the facts it started with.

• ctx.settings — the plugin registers the llm-local namespace with this same Config schema and its composition entry as the base, so a llm-local: section in the user settings document overrides any field without a restart.
• ctx.credentials — the key resolves per stream call from the same snapshot that supplies the endpoint. Configuration carries only apiKeyEnv, never a literal key; a route without one is asked unauthenticated.

The one registration-captured fact is the retry policy: when a route's resolved value changes, the plugin re-registers the route set in place (same adapter instance, one synchronous section). Route-set changes (adding/removing a route) re-register the same way. Every route change logs a line (llm-local: serving ollama (http://127.0.0.1:11434/v1), ...), and a dormant mount logs llm-local: dormant; ... — the first thing to check when a section appears to do nothing.

Troubleshooting: my settings.yaml does nothing

1. Did the server restart since the plugin was added? A llm-local: section in the document is silently preserved but inert until the plugin registers its namespace at boot. Restart the dsh server, then look for the llm-local: log line.
2. Is the file in the right place? The section must be at the top level of $DSH_HOME/settings.yaml (default ~/.dsh/settings.yaml), not inside another key, and not in a cordis.yml.
3. Is the YAML valid and indented like the examples? providers is a dict, so the provider key is indented under it and the profile fields under the provider key.
4. Did the route register? The Models page row for the provider flips from a dormant "add" state to a configured row; the server log prints llm-local: serving <route> (<baseURL>).
5. Is the session using the provider? Configuring a provider only makes it available; a conversation still uses whatever provider/model it was created with. Open the model picker (/model) in a new conversation and choose the local route.
6. Check the server log for errors. A schema-invalid section logs settings: keeping last good "llm-local" after invalid stored section with the reason; a route that fails to resolve keeps the previous routes and logs llm-local: keeping the previously registered routes after a refused update.

App attribution

Every request carries the shared attribution header from dsh-llm's attributionHeaders() (the mandatory User-Agent baseline). Request identity rides the wire as x-deepseek-harness-user-id (the stable anonymous id) and, when the request has a session, x-deepseek-harness-session-id. No provider-specific headers are added — local endpoints need none.

Wire-format notes

• Streaming only; stream_options.include_usage sent unless usageInStream: false. usage may arrive attached to the finish chunk or as a trailing usage-only chunk — the translator defers both to [DONE], so usage always precedes finish.
• Assistant history never replays reasoning_content (OpenAI's dialect has no CoT passback; local servers ignore or reject it). Incoming reasoning_content deltas still become harness reasoning blocks.
• Cache accounting: cacheReadTokens ← prompt_tokens_details.cached_tokens (llama.cpp) or prompt_cache_hit_tokens (DeepSeek-style spellings); the count is subtracted from inputTokens to keep disjoint totals.
• The first thinking-mode chunk often carries reasoning_content: "" — handled (no spurious reasoning block).

Errors

Non-2xx responses throw LlmError with stable codes: AUTH (401/403), QUOTA (provider details identify exhausted quota), RATE_LIMIT (429), CONTEXT_WINDOW_EXCEEDED (a 400 whose details identify context overflow), INVALID_REQUEST (other 400s), SERVER (5xx), HTTP_<status> otherwise. The serializable failure retains the HTTP status plus a valid positive Retry-After delay and x-request-id when present. Pre-response transport failures (DNS, refused connection, TLS, proxy) throw TRANSPORT naming the configured endpoint; caller aborts throw ABORTED; protocol violations throw STREAM_CLOSED (no [DONE]) or MALFORMED_RESPONSE (bad JSON payload). Unknown wire finish_reasons become finish {kind: 'error'} chunks, and a completed stream whose stop finish opened no content blocks becomes EMPTY_RESPONSE (retried by default policy).

Known Limitations and Deferred Work

• A settings models list replaces the composition list wholesale — settings-layer merging is per-field, and arrays are one field.
• tool_choice is not mapped — not part of the core vocabulary (shared with the DeepSeek and pi-ai adapters).
• Requests use raw fetch, not @cordisjs/plugin-http — no shared proxy/interception configuration.
• No reasoning-effort selectors — local OpenAI-compatible servers have no standard reasoning_effort dialect; reasoning models stream CoT but the harness cannot pick a thinking level for them.
• Serialization flattens user and tool-result content to text unless the model declares vision; empty tool output crosses the wire as the literal (no output).