dsh-image-generation-responses

简体中文

A DeepSeek Harness Cordis plugin that exposes image tools backed by the Responses API: generate_image (text-to-image and image-to-image editing through the image_generation tool) and analyze_image (image understanding through a plain vision completion). Generated bytes are validated and stored as durable DSH attachments, then rendered directly in the conversation by the included web client view.

Supported provider contract

This plugin intentionally supports a specific OpenAI-style contract rather than every API described as "OpenAI compatible":

• POST {baseURL}/responses
• Authorization: Bearer <credential>
• Responses API image_generation tool calls
• non-streaming JSON responses containing base64 image data

Azure-style api-version queries, api-key headers, arbitrary custom headers, remote image URLs, and the legacy /images/generations endpoint are not supported.

Requirements

• Node.js 20.3 or newer
• DeepSeek Harness packages compatible with 0.1.0-rc.6
• A provider/model combination that supports the Responses image-generation tool
• DSH services: tools, credentials, and attachments
• The standard DSH web client packages for inline rendering

Installation

Install the package in the DSH web profile that owns your cordis.patch.yml:

npm install dsh-image-generation-responses

Mount it in the profile patch:

- insert:
    - id: image-generation-responses
      name: dsh-image-generation-responses
      config:
        baseURL: https://api.openai.com/v1
        apiKeyEnv: OPENAI_API_KEY
        responseModel: gpt-5.6-sol
        imageModel: gpt-image-2
        size: 1024x1024
        quality: medium
        background: opaque
        format: png
        timeoutMs: 120000
        maxResponseBytes: 33554432

Provide the API key through DSH's credentials service or the environment. Never put a real key in cordis.patch.yml or commit it to Git.

After first installing this package, restart the running DSH process and refresh the web page. DSH discovers a package's Client half at process startup. Later edits to an already-discovered lib/client.js may use the normal client-plugin HMR flow when its watcher is available.

Configuration

Option	Default	Description
baseURL	https://api.openai.com/v1	Trusted deployment API base. /responses is appended; query and fragment are removed.
apiKeyEnv	OPENAI_API_KEY	Credential reference resolved for every call.
responseModel	gpt-5.6-sol	Top-level Responses model. Override for compatible providers.
imageModel	gpt-image-2	Model field on the image_generation tool.
size	1024x1024	1024x1024, 1024x1536, 1536x1024, or auto.
quality	medium	low, medium, high, or auto.
background	opaque	opaque, transparent, or auto.
format	png	png, jpeg, or webp. Transparent JPEG is rejected.
timeoutMs	120000	Cooperative request and tool timeout.
maxResponseBytes	33554432	Limit for the JSON response body and decoded image.
visionModel	= responseModel	Model answering analyze_image vision calls.

baseURL is administrator-controlled configuration. Never derive it from user or model input. Plain HTTP is accepted for trusted local development endpoints; production deployments should use HTTPS.

Tool

generate_image(prompt, images?, input_fidelity?, size?, quality?, background?, format?)

The tool returns canonical JSON metadata including the durable attachment reference, model names, generation options, the resolved action, and provider IDs when present. Its model-facing rendering contains a text summary naming the saved attachment id, plus an image ContentBlock only when the conversation model declares image input (resolved per call from the session's request header through llm.resolveModelInfo). Text-only models such as a non-vision route receive a text-only result — adapters like pi-ai reject a whole turn with UNSUPPORTED_CONTENT when tool-result content carries an image the model cannot read. The web UI is unaffected either way; it renders the result from the presentation meta.

Image-to-image (editing)

Passing images — attachment ids of images already present in the conversation — turns the call into editing instead of generating from scratch. prompt is then the edit instruction.

generate_image(prompt: "make it night", images: ["att_..."], input_fidelity: "high")

What changes on the wire: the tool entry carries action: "edit" (instead of "generate"), optionally input_fidelity, and input becomes a message array holding an input_text block followed by one input_image block per reference image. With no images, the request is byte-for-byte the text-to-image shape.

Constraints worth knowing:

• Up to 8 reference images per call.
• Ids must be visible in the calling session's own log. The referenced bytes are read back through attachments.readImage, which verifies the complete reference (media type, byte length, and intrinsic dimensions) against the stored object — so an id alone cannot be used to read an attachment, and editing is confined to images that session can already see.
• input_fidelity is edit-only; requesting it without images is rejected. Upstream supports it on gpt-image-1/gpt-image-1.5 and later, and not on gpt-image-1-mini.
• An unresolvable id fails before any credential is resolved or provider call is made.

Image understanding

analyze_image(question, images) → answer

analyze_image answers natural-language questions about images from the conversation — describe, read text, compare, inspect details — by sending them to a vision model over the same Responses endpoint. It completes the generate → review → edit loop: a text-only conversation model can pass a generate_image attachment id here, read the answer, and then issue a better edit.

• Same endpoint, credential, and reference resolution as generation; the model defaults to responseModel and is overridable with visionModel.
• The wire call is a plain completion (input_text + input_image blocks, no tools), parsed by the same strict envelope handling.
• The result is pure text, so it is safe on every conversation route — no capability gating is needed.

Storage and conversation rendering

The Host half strictly decodes base64 and calls attachments.saveImage(). DSH validates and stores the image in its attachment backend; it is not written to the workspace as a normal .png file.

The Client half registers a keyed generate_image view in tool.call.toolview. It resolves a session-authorized attachment URL through the conversation service and renders DSH's ImageGallery, including loading, retry, and original-image preview behavior.

Session image dock

The Client half also contributes a session image dock: a vertically centred strip of every durable image in the current session, beside the conversation column.

It has no menu-bar or sidebar button. The strip appears exactly when the conversation holds at least one image and is absent otherwise. Thumbnails are capped at 120px — half the 240px single box used in chat history — newest first and deduplicated by attachment id.

Clicking a thumbnail opens the original-image preview. The strip renders through a document.body portal and stacks above that preview, so it stays visible and clickable while a preview is open: clicking another thumbnail switches the preview in place rather than requiring a close first. This portal is required — the shell.overlay layer establishes its own stacking context at z-index: 20, which would otherwise trap the strip beneath the lightbox's z-index: 1000.

The dock collects only model-returned images — assistant output blocks and tool results (including the presentation-meta fallback used when content blocks were pruned). User uploads, steering messages, and context injections are deliberately excluded. It is an additive shell.overlay list entry addressed by a namespaced id, so no shipped overlay UI is replaced, and it subscribes to the current session's conversation snapshot only while mounted. It measures the sidebar column so it stays flush against the conversation column across collapse and drag.

Localization

Every user-facing string in the web client — the tool row, the dock, and the gallery/lightbox affordances — is localized through the shell's locale service (zh and en, following the active language and its switcher; English is the fallback when the locale seat is absent). Model-facing text (tool descriptions, result summaries, error messages) intentionally stays English.

Errors and limits

Failures use stable ImageGenerationError.code values such as MISSING_CREDENTIAL, HTTP_ERROR, TIMEOUT, BAD_BASE64, OVERSIZED, REFUSED, and MISSING_OUTPUT. Responses and decoded images are bounded. Remote image URLs and HTTP redirects are rejected.

Prompts and generated images are processed by the configured provider. Review that provider's data and content policies before use.

Development

npm install
npm test
npm run check
npm pack --dry-run

Tests use mocked transport and tiny fixtures. They require no credentials and make no paid generation calls.

lib/client.js is authored directly in DSH's distributable browser module-loader format. There is intentionally no generated client artifact or hidden transform step; changes must preserve the window.__ModuleLoader__.load({ id, factory }) contract and platform seed-module boundary.

React, React DOM, and @deepseek-ai/dsh-client-ui-attachment are optional npm peers because the supported DSH web shell supplies them as platform seed modules. Materializing this Client half outside that shell is unsupported.

The DSH host platform packages this plugin imports (@deepseek-ai/dsh-tools, @deepseek-ai/dsh-credentials, @deepseek-ai/dsh-llm) are non-optional npm peers — never runtime dependencies, so an install cannot nest a copy that shadows the host's own. They are mirrored in devDependencies so local development and CI can run the test suite.

See SECURITY.md for private vulnerability reporting, CONTRIBUTING.md for contribution guidelines, and RELEASING.md for the maintainer release checklist.

License

MIT © Poepon and contributors.