From 16104a30bb117db65ef09ed7eea4c6558a0e0e01 Mon Sep 17 00:00:00 2001 From: James Magahern Date: Mon, 2 Nov 2026 21:11:43 -0800 Subject: [PATCH] Preserve response model attribution on the deployed backend base Backend and API documentation changes from b9ff0a7; retains the currently deployed 5d738f8 backend feature set. --- docs/api/rest.md | 6 +- docs/api/streaming-chat.md | 2 + server/src/llm/multiplexer.ts | 1 + server/src/llm/streaming.ts | 5 +- server/src/message-generation.ts | 31 +++++++++ server/src/routes.ts | 6 +- server/tests/message-generation.test.ts | 85 +++++++++++++++++++++++++ 7 files changed, 132 insertions(+), 4 deletions(-) create mode 100644 server/src/message-generation.ts create mode 100644 server/tests/message-generation.test.ts diff --git a/docs/api/rest.md b/docs/api/rest.md index bd7ca23..614c0b0 100644 --- a/docs/api/rest.md +++ b/docs/api/rest.md @@ -180,7 +180,7 @@ Behavior notes: Behavior notes: - With no `messageId`, the server copies the complete source transcript. With an assistant response `messageId`, it copies the transcript through that response, inclusive. -- The source chat is unchanged. Copied messages receive new ids while retaining their role, content, name, creation time, order, and metadata except for the source request's transport-only `clientRequestId`. Attachments and tool-call metadata are retained. LLM call logs, star/project memberships, and other child threads are not copied. +- The source chat is unchanged. Copied messages receive new ids while retaining their role, content, name, creation time, order, and metadata except for the source request's transport-only `clientRequestId`. Attachments, tool-call metadata, and per-response `metadata.generation` are retained. For older replies with an exact call-log match, generation metadata is recovered before copying. LLM call logs, star/project memberships, and other child threads are not copied. - The fork inherits the source chat's user, initiated/last-used provider and model, additional system prompt, and enabled tool settings. - A fork of a root or another fork always sets `parentChatId` to the single root chat id. Root chats keep `parentChatId: null`. - A whole-chat fork starts with `Fork of ` (or `Fork of Untitled chat`). A message fork starts with `Fork of ''`. @@ -233,6 +233,9 @@ Behavior notes: ### `GET /v1/chats/:chatId` - Response: `{ "chat": ChatDetail }` +- Saved ordinary assistant responses can include `metadata.generation: { "provider": "openai", "model": "gpt-4.1-mini" }`. These are the provider/model identifiers used for that completion request, captured separately from the chat's mutable last-used selection. Model aliases remain the requested alias; this does not claim an underlying provider snapshot ID. +- Newly generated responses store this metadata for both streaming and non-streaming completions. It survives reloads and forks. Inline Sybil Agent replies retain their separate `metadata.agentAside` identity. +- For older assistant messages without generation metadata, this response recovers it only when `metadata.clientRequestId` matches exactly one completed, successful call in the same chat. This read does not rewrite stored history. Missing or ambiguous records, including some older forks/imports, remain unattributed; clients must not substitute the chat's current or last-used model. The web footer displays “Model not recorded” in that case. ### `GET /v1/chats/:chatId/messages` - Requires the same authenticated owner/admin credential as other chat routes. Responses use `Cache-Control: no-store`. @@ -338,6 +341,7 @@ Behavior notes: - Before the Agent is invoked, the backend paginates and supplies the complete safe visible-thread projection as required initial context; the Agent does not have to choose `read_chat_history` for the current thread. User/assistant asides and full supported attachment representations are included, while system/tool rows and unrelated metadata are excluded. Historical content remains untrusted context, not new authority. - The initial context transfer is limited to 30 MiB. A stored text attachment marked truncated, incomplete history, or a transfer/provider context-limit failure returns an explicit error and no answer from partial history; providers may reject a complete thread that exceeds their model context window. - For `chatId` calls, server stores only *new* non-assistant messages from provided history to avoid duplicates. +- Saved ordinary assistant responses also store the completion's provider/model in `metadata.generation`, exposed by `GET /v1/chats/:chatId`; changing the chat selection does not relabel previous responses. - `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint. - `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools. - `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap. diff --git a/docs/api/streaming-chat.md b/docs/api/streaming-chat.md index 7d7fddb..445ca16 100644 --- a/docs/api/streaming-chat.md +++ b/docs/api/streaming-chat.md @@ -146,6 +146,8 @@ Each tool invocation can emit multiple `tool_call` events with the same `toolCal For `persist: false` streams, `chatId` and `callId` are `null`. +When an ordinary persisted completion finishes, its assistant message stores the same request-time provider/model in `metadata.generation`, available in the canonical `GET /v1/chats/:chatId` refresh. Clients must use per-message generation metadata for historical response labels, not the current chat selection. Older messages can lack attribution; see [chat detail](rest.md#get-v1chatschatid). This adds no SSE events or fields, and does not change non-persistent Quick Questions or inline Agent identity. + ### `delta` ```json diff --git a/server/src/llm/multiplexer.ts b/server/src/llm/multiplexer.ts index 8dc4191..0b1cf5a 100644 --- a/server/src/llm/multiplexer.ts +++ b/server/src/llm/multiplexer.ts @@ -85,6 +85,7 @@ export async function runMultiplex(req: MultiplexRequest): Promise { + return value && typeof value === "object" && !Array.isArray(value) ? value as Record : {}; +} + +type RecordedCall = { + provider: unknown; + model: string; + request: unknown; + error: string | null; + latencyMs: number | null; +}; + +/** Recover older streaming replies only through an unambiguous, completed request ID. */ +export function withMessageGenerations(messages: T[], calls: RecordedCall[]): T[] { + const byRequest = new Map(); + for (const call of calls) { + const requestId = record(call.request).clientRequestId; + if (typeof requestId !== "string" || !requestId || call.error !== null || call.latencyMs === null) continue; + const provider = fromPrismaProvider(call.provider); + if (!provider || !call.model.trim()) continue; + byRequest.set(requestId, byRequest.has(requestId) ? null : { provider, model: call.model }); + } + return messages.map(message => { + const metadata = record(message.metadata); + if (message.role !== "assistant" || metadata.agentAside === true || metadata.generation !== undefined) return message; + const generation = typeof metadata.clientRequestId === "string" ? byRequest.get(metadata.clientRequestId) : null; + return generation ? { ...message, metadata: { ...metadata, generation } } : message; + }); +} diff --git a/server/src/routes.ts b/server/src/routes.ts index ccee02c..ff701cc 100644 --- a/server/src/routes.ts +++ b/server/src/routes.ts @@ -7,6 +7,7 @@ import { invokeAgentInline, type AgentInlineInvoker } from "./agent-inline.js"; import { buildInlineAgentThreadContext, decodeChatHistoryCursor, decodeChatListCursor, encodeChatListCursor, loadCompleteChatHistory, readChatHistoryPage } from "./chat-history.js"; import { containsSybilMention, filterAgentAsideMessages } from "./agent-mention.js"; import { prisma } from "./db.js"; +import { withMessageGenerations } from "./message-generation.js"; import { requireAdmin } from "./auth.js"; import { env } from "./env.js"; import { buildComparableAttachments } from "./llm/message-content.js"; @@ -1173,11 +1174,12 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin enabledTools: true, userId: true, messages: { orderBy: [{ createdAt: "asc" }, { id: "asc" }] }, + calls: { select: { provider: true, model: true, request: true, error: true, latencyMs: true } }, }, }); if (!source) return { status: "chat-not-found" as const }; - let messages = source.messages; + let messages = withMessageGenerations(source.messages, source.calls); let selectedMessage: (typeof source.messages)[number] | undefined; if (parsed.data.messageId) { const selectedIndex = messages.findIndex((message) => message.id === parsed.data.messageId); @@ -1680,7 +1682,7 @@ export async function registerRoutes(app: FastifyInstance, options: { agentInlin }, }); if (!chat) return app.httpErrors.notFound("chat not found"); - return { chat: serializeChatLike(chat) }; + return { chat: serializeChatLike({ ...chat, messages: withMessageGenerations(chat.messages, chat.calls) }) }; }); app.post("/v1/chats/:chatId/messages", async (req) => { diff --git a/server/tests/message-generation.test.ts b/server/tests/message-generation.test.ts new file mode 100644 index 0000000..55021cc --- /dev/null +++ b/server/tests/message-generation.test.ts @@ -0,0 +1,85 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join, resolve } from "node:path"; +import test from "node:test"; +import { fileURLToPath } from "node:url"; +import Fastify from "fastify"; +import sensible from "@fastify/sensible"; +import { withMessageGenerations } from "../src/message-generation.js"; + +const serverRoot = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const databaseDir = mkdtempSync(join(tmpdir(), "sybil-message-generation-")); +process.env.DATABASE_URL = `file:${join(databaseDir, "test.db")}`; +process.env.ADMIN_TOKEN = "message-generation-test-token"; +process.env.OPENAI_API_KEY = "synthetic-openai-key"; +process.env.XAI_API_KEY = "synthetic-xai-key"; +writeFileSync(join(databaseDir, "test.db"), ""); +execFileSync(process.execPath, [join(serverRoot, "node_modules/prisma/build/index.js"), "migrate", "deploy"], { + cwd: serverRoot, env: { ...process.env, PRISMA_HIDE_UPDATE_MESSAGE: "1" }, stdio: "pipe", +}); +const [{ registerRoutes }, { prisma }, { runMultiplex }, { runMultiplexStream }] = await Promise.all([ + import("../src/routes.js"), import("../src/db.js"), import("../src/llm/multiplexer.js"), import("../src/llm/streaming.js"), +]); +const app = Fastify({ logger: false }); +await app.register(sensible); +await registerRoutes(app); +await app.ready(); +const auth = { authorization: "Bearer message-generation-test-token" }; +test.after(async () => { await app.close(); await prisma.$disconnect(); rmSync(databaseDir, { recursive: true, force: true }); }); + +test("streaming and non-streaming replies retain their own model across selection changes and forks", async t => { + t.mock.method(globalThis, "fetch", async (input: RequestInfo | URL, init?: RequestInit) => { + const url = input instanceof Request ? input.url : String(input); + assert.ok(["https://api.openai.com/v1/chat/completions", "https://api.x.ai/v1/chat/completions"].includes(url), url); + const body = JSON.parse(input instanceof Request ? await input.text() : String(init?.body)); + const content = `Reply from ${body.model}`; + return body.stream + ? new Response(`data: ${JSON.stringify({ choices: [{ index: 0, delta: { content }, finish_reason: null }] })}\n\ndata: [DONE]\n\n`, { headers: { "content-type": "text/event-stream" } }) + : Response.json({ choices: [{ message: { role: "assistant", content } }] }); + }); + const chat = await prisma.chat.create({ data: { title: "Mixed models" } }); + const messages = [{ role: "user" as const, content: "Answer" }]; + const events = []; + for await (const event of runMultiplexStream({ chatId: chat.id, provider: "openai", model: "model-one", clientRequestId: "first-turn", messages, enabledTools: [] })) events.push(event); + assert.equal(events.at(-1)?.type, "done"); + await runMultiplex({ chatId: chat.id, provider: "xai", model: "model-two", messages, enabledTools: [] }); + await prisma.chat.update({ where: { id: chat.id }, data: { lastUsedProvider: "gemini", lastUsedModel: "model-next" } }); + const detail = await app.inject({ method: "GET", url: `/v1/chats/${chat.id}`, headers: auth }); + assert.equal(detail.statusCode, 200); + const saved = detail.json().chat.messages; + assert.deepEqual(saved.map((m: any) => m.metadata.generation), [{ provider: "openai", model: "model-one" }, { provider: "xai", model: "model-two" }]); + assert.equal(saved[0].metadata.clientRequestId, "first-turn"); + const fork = await app.inject({ method: "POST", url: `/v1/chats/${chat.id}/fork`, headers: auth, payload: { messageId: saved[1].id } }); + assert.equal(fork.statusCode, 200); + const forkDetail = await app.inject({ method: "GET", url: `/v1/chats/${fork.json().chat.id}`, headers: auth }); + assert.deepEqual(forkDetail.json().chat.messages.map((m: any) => m.metadata.generation), saved.map((m: any) => m.metadata.generation)); + assert.equal(forkDetail.json().chat.messages[0].metadata.clientRequestId, undefined); +}); + +test("legacy call matches are projected without rewriting history and survive a new fork", async () => { + const chat = await prisma.chat.create({ data: { + title: "Older reply", lastUsedProvider: "gemini", lastUsedModel: "different-current-model", + messages: { create: { role: "assistant", content: "Older reply", metadata: { clientRequestId: "legacy-request", citations: ["https://example.test"] } } }, + calls: { create: { provider: "anthropic", model: "original-model", request: { clientRequestId: "legacy-request" }, latencyMs: 10 } }, + }, include: { messages: true } }); + const detail = await app.inject({ method: "GET", url: `/v1/chats/${chat.id}`, headers: auth }); + assert.deepEqual(detail.json().chat.messages[0].metadata.generation, { provider: "anthropic", model: "original-model" }); + assert.deepEqual((await prisma.message.findUniqueOrThrow({ where: { id: chat.messages[0]!.id } })).metadata, chat.messages[0]!.metadata); + const fork = await app.inject({ method: "POST", url: `/v1/chats/${chat.id}/fork`, headers: auth, payload: {} }); + const forkDetail = await app.inject({ method: "GET", url: `/v1/chats/${fork.json().chat.id}`, headers: auth }); + assert.deepEqual(forkDetail.json().chat.messages[0].metadata, { citations: ["https://example.test"], generation: { provider: "anthropic", model: "original-model" } }); +}); + +test("missing, ambiguous, failed and unfinished calls never guess a reply's model", () => { + const call = { provider: "openai", model: "model-one", request: { clientRequestId: "request" }, error: null, latencyMs: 10 }; + const message = { role: "assistant", metadata: { clientRequestId: "request" } }; + for (const calls of [[], [call, { ...call, provider: "xai" }], [{ ...call, error: "failed" }], [{ ...call, latencyMs: null }]]) { + assert.deepEqual(withMessageGenerations([message], calls), [message]); + } + const existing = { ...message, metadata: { ...message.metadata, generation: { provider: "gemini", model: "original" } } }; + const aside = { ...message, metadata: { ...message.metadata, agentAside: true } }; + const user = { ...message, role: "user" }; + assert.deepEqual(withMessageGenerations([existing, aside, user], [call]), [existing, aside, user]); +});