dsh-web-search-ext

Multi-backend web_search provider for DeepSeek Harness (DSH), registered into the web capability seam (ctx.web) under one stable provider id (web-search-ext).

The built-in web_search tool is backend-pluggable; the in-box default (deepseek-official) requires a DeepSeek API key. This plugin provides a key-free-capable alternative with automatic failover between backends.

Current backends

Backend	With key	Without key
Exa	REST POST https://api.exa.ai/search (higher limits, highlight snippets)	Anonymous hosted MCP POST https://mcp.exa.ai/mcp (JSON-RPC 2.0, documented public fallback, rate-limited → HTTP 429)
Firecrawl	POST https://api.firecrawl.dev/v2/search (Bearer)	Keyless requests when firecrawlKeyless: true (unofficial; may be rate-limited or removed)

On any backend failure (429, 401/402/403, 5xx, network, malformed body) the search fails over to the next backend. A 429 additionally starts a per-backend cooldown (default 60 s) so a saturated backend is skipped on subsequent searches. When all backends fail, the error lists each failure (including cooldown state).

Adding another backend (e.g. SearXNG)

The provider is deliberately small and uniform; a backend is:

1. one async function <name>Search(options, apiKey?, request, signal) that returns { sources, truncated: false } and throws WebError (WEB_RATE_LIMIT for 429, WEB_PROVIDER_ERROR otherwise, aborts as WEB_ABORTED);
2. one entry pushed onto plan in search() when its availability condition holds (key present, or keyless mode allowed);
3. its config fields in the Config schema and key resolution in resolveKey().

No other code changes: failover, cooldown, abort handling, and snippet bounding all apply automatically. A self-hosted SearXNG instance (JSON API) is the natural next candidate — no key, no rate limit of your own, fully local.

Install

dsh plugin --profile web add dsh-web-search-ext
# or from a local checkout:
dsh plugin --profile web add ./path/to/dsh-web-search-ext

Installing a plugin requires restarting the running dsh web process (the profile bundle list is resolved at boot). Config changes afterwards are hot — no restart.

The bundle patch selects this provider for the web_search tool by setting web.searchProvider: web-search-ext. The official deepseek-official provider stays registered but unused; an explicit selection also prevents WEB_PROVIDER_AMBIGUOUS.

Keys (optional but recommended)

Any of these, in order of precedence per backend:

1. Literal key in the settings section (exaApiKey / firecrawlApiKey)
2. Credentials service: EXA_API_KEY / FIRECRAWL_API_KEY (the Web "Models" page writes them; ~/.dsh/.credentials.yaml)
3. Launch environment variable of the same name

No keys at all still works: Exa uses its anonymous MCP endpoint and Firecrawl is tried keyless.

Configuration

Settings namespace web-search-ext in ~/.dsh/settings.yaml (hot-reloaded):

web-search-ext:
  preferred: exa               # exa | firecrawl — which backend to try first
  # exaApiKeyEnv: EXA_API_KEY      # defaults shown
  # firecrawlApiKeyEnv: FIRECRAWL_API_KEY
  # exaApiUrl: https://api.exa.ai/search
  # exaMcpUrl: https://mcp.exa.ai/mcp
  # firecrawlBaseUrl: https://api.firecrawl.dev/v2
  numResults: 8                # default result count when the tool doesn't cap it
  maxSnippetChars: 500         # snippet length bound
  rateLimitCooldownSec: 60     # skip a 429'd backend this long; 0 disables
  firecrawlKeyless: true       # allow keyless Firecrawl requests

Or select this provider without the bundle patch: DSH_WEB_SEARCH_PROVIDER=web-search-ext.

Uninstall

dsh plugin --profile web remove dsh-web-search-ext   # then restart dsh web

Security notes

• The only outbound requests are to the configured Exa and Firecrawl endpoints; nothing else is contacted.
• API keys travel only in the authorization header of their own backend's requests — never in bodies, never to the other backend, never in error messages.
• No install-time scripts: plain ESM JavaScript, no build step, no postinstall/prepare.
• Snippets are bounded (maxSnippetChars) and Firecrawl's page-markdown descriptions are stripped of image links before entering model context.

Verified against (2026-08-17)

• Exa anonymous MCP: live call to mcp.exa.ai/mcp returned structured Title:/URL:/Highlights: results without credentials.
• Exa REST without key: HTTP 402 (X402_PAYMENT_REQUIRED) — treated as "no key", not an error to surface.
• Firecrawl v2 without key: live call returned data.web[] (v1 shape data[] is also handled).
• Test suite: node test/failover.test.mjs — 10 mocked failover/mapping scenarios + live smoke calls.