diff --git a/docs/telemetry.md b/docs/telemetry.md index 6cbdbc39..08a9da30 100644 --- a/docs/telemetry.md +++ b/docs/telemetry.md @@ -2,6 +2,24 @@ [Back to README](../README.md) · [Contributing](../CONTRIBUTING.md) +## Standard MCP Apps hosts + +Explicit host profiles preserve the existing UI/server/native initialization, +release environment, identity, sampling and scrubbing. Standard context and chat +handoffs retain their existing timing and handled-error boundaries. Workspace +display-mode changes use `ui.workspace.display_mode_changed`; resize failures use +`workspace.resize`. Surface/device attribution and hidden-panel cleanup remain +on the active rendering and streaming paths. The Codex updater and its polling +are disabled outside Codex. No native telemetry changes are required. + +VS Code prepares messages in the composer without submitting a turn. Its success +counters are `ui.annotations.prepare_success`, `ui.logs.prepare`, +`ui.recording.ask_prepared`, `ui.recording.open_prepared`, +`ui.comparison.ask_prepared`, `ui.tracesift.investigate_prepared` and +`ui.tracesift.pick_investigate_prepared`. Existing send/action counters retain +their behavior on other hosts. Message-acknowledgement timings still measure the +host round trip; they do not measure model response time or prove submission. + ## Volume policy Since 0.1.164, application metrics in the UI, Node servers and native helpers use diff --git a/docs/vscode.md b/docs/vscode.md new file mode 100644 index 00000000..33273b64 --- /dev/null +++ b/docs/vscode.md @@ -0,0 +1,76 @@ +# VS Code / GitHub Copilot development preview + +Runs the shared Mobile Dev MCP server and native helpers on an Apple Silicon +Mac. Requires Node >=22.19, Xcode for iOS and the Android SDK for Android. +Use a current VS Code with GitHub Copilot and MCP Apps enabled +(`chat.mcp.apps.enabled`). This preview has automated protocol/package coverage; +actual VS Code rendering, device streaming and input still need validation. + +## Run from source + +Complete repository setup, then: + +```sh +npm run build +npm run vscode:config +``` + +Use **MCP: Open User Configuration** and merge the `servers` entry from +`.local-dev/vscode-mcp.json`. Preserve existing servers. Alternatively use your +app project's `.vscode/mcp.json`. For SSH/dev-container work, keep this stdio +server in the **local** user profile so it can reach the Mac's Xcode/ADB/devices. +The generated Node and server paths are absolute. Restart the MCP server after +rebuilding; regenerate configuration after moving the checkout or Node. + +Ask: **Open Mobile Dev and list available devices without booting anything.** +Choose ambiguous targets in chat or in the device picker. + +## Relocatable bundle + +```sh +npm run package:vscode +npm run test:vscode +``` + +Unzip `release/mobile-dev--vscode-darwin-arm64.zip` into a persistent +directory, run `node configure.mjs` there, then merge the printed configuration +into VS Code. It uses the Node running that command and requires no Codex +installation. Replace the bundle and restart the server to update. Codex update +checks and installation are disabled. This is an MCP bundle, not a VSIX. + +Local builds retain development telemetry. Public packaging requires +`npm run build:release` and `npm run package:vscode -- --release`, plus the normal +source-map and native-symbol upload workflow. + +## VS Code behavior + +- The workspace runs inline at a reported bounded height, with responsive + device/tools layout. It declares only inline mode. A docked editor panel or + fullscreen workspace would require separate VS Code extension work. +- **Prepare question** / **Prepare investigation** fills the chat input. Review + the text and press **Send** to start. Logs, screenshot context and annotations + remain selected until explicitly cleared; a prepared question is not a sent + turn. Questions include their selected log/recording details. +- The explicit VS Code profile accommodates versions that handle `ui/message` + but omit `message.text`, and context updates without a `text` subcapability. + Other hosts retain capability checks. Rejections/timeouts are shown without + automatically retrying an uncertain handoff. Existing copy actions remain. + +## Validation before declaring support + +- Open an existing iOS simulator, stream for five minutes, tap and drag. +- Confirm the inline height, keyboard/menu behavior and device/tool selectors. +- Prepare a log/annotation/recording question, verify it does not submit, then + submit manually and check the agent receives the correct device and reference. +- Check image attachments and explicit clearing, including a manually removed + composer attachment. Standard context provides no composer readback; local + selections remain app-owned and can be published again on a later update. +- Hide/reopen cards, switch chats and verify collectors and observers clean up. +- Validate Android H.264 and physical-iOS HEVC independently of simulator JPEG. + +The smoke test relocates the archive into a path with spaces and verifies MCP +startup, UI resources, telemetry opt-out and update isolation without booting +devices. Existing stream/input telemetry remains; prepared-message outcomes use +separate counters described in `docs/telemetry.md`. + +Source: [VS Code MCP developer guide](https://code.visualstudio.com/api/extension-guides/ai/mcp). diff --git a/integrations/vscode/INSTALL.md b/integrations/vscode/INSTALL.md new file mode 100644 index 00000000..7d917834 --- /dev/null +++ b/integrations/vscode/INSTALL.md @@ -0,0 +1,5 @@ +Run `node configure.mjs` from this extracted bundle. Merge the printed `servers` +entry into the local user-profile MCP configuration in VS Code. The generated +configuration uses absolute paths and does not modify any editor settings. + +See README.md for prerequisites, update instructions and preview limitations. diff --git a/package.json b/package.json index 221c458f..de40ff87 100644 --- a/package.json +++ b/package.json @@ -31,6 +31,9 @@ "test:ios-cpu": "node scripts/smoke-ios-cpu.mjs", "package": "node scripts/package.mjs", "package:release": "node scripts/package.mjs --release", + "vscode:config": "node scripts/vscode-config.mjs", + "package:vscode": "node scripts/package-vscode.mjs", + "test:vscode": "node scripts/smoke-editor-package.mjs vscode", "vendor:serve-emu": "node scripts/vendor-serve-emu.mjs", "dev": "node scripts/dev-ui.mjs", "dev:off": "node scripts/dev-ui.mjs --off", diff --git a/scripts/local-host-config.mjs b/scripts/local-host-config.mjs new file mode 100644 index 00000000..9526910c --- /dev/null +++ b/scripts/local-host-config.mjs @@ -0,0 +1,12 @@ +import { access } from "node:fs/promises"; +import { resolve } from "node:path"; + +// Generate data only. The caller decides where to save it; user settings are +// never overwritten. Resolve paths now so GUI hosts need no shell PATH or cwd. +export async function localHostConfig(host, root = ".", node = process.execPath) { + if (host !== "cursor" && host !== "vscode") throw new Error("Unsupported editor host."); + const server = resolve(root, "dist/server.mjs"); + await access(server); + const entry = { type: "stdio", command: node, args: [server], env: { MOBILE_DEV_HOST: host } }; + return host === "vscode" ? { servers: { "mobile-dev": entry } } : { mcpServers: { "mobile-dev": entry } }; +} diff --git a/scripts/package-editor.mjs b/scripts/package-editor.mjs new file mode 100644 index 00000000..a4d1c2d5 --- /dev/null +++ b/scripts/package-editor.mjs @@ -0,0 +1,34 @@ +import { execFileSync } from "node:child_process"; +import { cp, mkdir, readFile, rm, writeFile } from "node:fs/promises"; +import { resolve } from "node:path"; +import { telemetryBuildEnvironment } from "./telemetry-build.mjs"; + +export async function packageEditor(host) { + if (host !== "cursor" && host !== "vscode") throw new Error("Unsupported editor host."); + telemetryBuildEnvironment(); + // Retain the canonical native inventory, licenses and environment checks. + execFileSync(process.execPath, ["scripts/package.mjs", ...process.argv.slice(2)], { stdio: "inherit" }); + const { version } = JSON.parse(await readFile("package.json", "utf8")); + const source = resolve("release/marketplace/plugins/mobile-dev"); + const directory = resolve(`release/${host}/mobile-dev`); + await rm(directory, { recursive: true, force: true }); + await mkdir(directory, { recursive: true }); + for (const path of ["dist", "THIRD_PARTY_NOTICES.md", "PRIVACY.md", "TERMS.md", "SUPPORT.md"]) { + await cp(`${source}/${path}`, `${directory}/${path}`, { recursive: true, verbatimSymlinks: true }); + } + await cp(`integrations/${host}`, directory, { recursive: true }); + await cp(`docs/${host}.md`, `${directory}/README.md`); + await cp("scripts/local-host-config.mjs", `${directory}/local-host-config.mjs`); + await writeFile(`${directory}/configure.mjs`, `import { localHostConfig } from "./local-host-config.mjs";\nconsole.log(JSON.stringify(await localHostConfig(${JSON.stringify(host)}, import.meta.dirname), null, 2));\n`); + if (host === "cursor") { + const path = `${directory}/.cursor-plugin/plugin.json`; + const manifest = JSON.parse(await readFile(path, "utf8")); + manifest.version = version; + await writeFile(path, JSON.stringify(manifest, null, 2) + "\n"); + } + const archive = resolve(`release/mobile-dev-${version}-${host}-darwin-arm64.zip`); + await rm(archive, { force: true }); + execFileSync("zip", ["-qr", archive, "."], { cwd: directory }); + execFileSync("unzip", ["-tq", archive], { stdio: "inherit" }); + console.log(`${host} bundle: ${archive}`); +} diff --git a/scripts/package-vscode.mjs b/scripts/package-vscode.mjs new file mode 100644 index 00000000..28d23444 --- /dev/null +++ b/scripts/package-vscode.mjs @@ -0,0 +1,2 @@ +import { packageEditor } from "./package-editor.mjs"; +await packageEditor("vscode"); diff --git a/scripts/smoke-editor-package.mjs b/scripts/smoke-editor-package.mjs new file mode 100644 index 00000000..22f49228 --- /dev/null +++ b/scripts/smoke-editor-package.mjs @@ -0,0 +1,60 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { access, mkdir, mkdtemp, readFile, realpath, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js"; + +const host = process.argv[2]; +if (host !== "cursor" && host !== "vscode") throw new Error("Pass cursor or vscode."); +const { version } = JSON.parse(await readFile("package.json", "utf8")); +const archive = resolve(process.argv[3] ?? `release/mobile-dev-${version}-${host}-darwin-arm64.zip`); +const temporary = await realpath(await mkdtemp(join(tmpdir(), `mobile-dev-${host}-`))); +try { + const directory = join(temporary, "bundle with spaces"); + await mkdir(directory); + execFileSync("unzip", ["-q", archive, "-d", directory]); + const config = JSON.parse(execFileSync(process.execPath, [join(directory, "configure.mjs")], { cwd: temporary, encoding: "utf8" })); + const entry = config[host === "cursor" ? "mcpServers" : "servers"]["mobile-dev"]; + assert.equal(entry.command, process.execPath); + assert.equal(entry.env.MOBILE_DEV_HOST, host); + assert.equal(entry.args[0], join(directory, "dist/server.mjs")); + if (host === "cursor") { + const manifest = JSON.parse(await readFile(join(directory, ".cursor-plugin/plugin.json"), "utf8")); + assert.equal(manifest.version, version); + const pluginConfig = JSON.parse(await readFile(join(directory, "mcp.json"), "utf8")); + assert.equal(pluginConfig.mcpServers["mobile-dev"].env.MOBILE_DEV_HOST, host); + assert.equal(pluginConfig.mcpServers["mobile-dev"].args[0].replaceAll("${CURSOR_PLUGIN_ROOT}", directory), entry.args[0]); + } + for (const path of ["dist/app.html", "dist/baguette/Baguette", "dist/ios-mirror/darwin-arm64.node", "dist/serve-emu/src/cli.mjs"]) await access(join(directory, path)); + const home = join(temporary, "home"); + await mkdir(home); + for (const environment of ["development", "release"]) { + const client = new Client({ name: `${host}-package-smoke`, version: "1" }); + const transport = new StdioClientTransport({ + command: entry.command, args: entry.args, cwd: temporary, stderr: "pipe", + env: { PATH: "/usr/bin:/bin:/usr/sbin:/sbin", HOME: home, ...entry.env, MOBILE_DEV_TELEMETRY: "off", MOBILE_DEV_ENVIRONMENT: environment }, + }); + let diagnostics = ""; + transport.stderr?.on("data", chunk => { diagnostics = (diagnostics + chunk).slice(-4000); }); + try { + await client.connect(transport); + const { tools } = await client.listTools(); + assert.ok(!tools.some(tool => tool.name === "mobile_choose_devices")); + const workspace = tools.find(tool => tool.name === "mobile_open_workspace"); + assert.ok(workspace); + const resource = await client.readResource({ uri: workspace._meta.ui.resourceUri }); + const html = resource.contents[0].text; + assert.ok(html.includes(`data-host="${host}"`)); + assert.ok(html.includes(`name="mobile-dev-environment" content="${environment}"`)); + assert.ok(html.includes('name="mobile-dev-telemetry" content="off"')); + const check = await client.callTool({ name: "mobile_check_plugin_update", arguments: {} }); + assert.equal(check.structuredContent.update.status, "disabled"); + const install = await client.callTool({ name: "mobile_install_plugin_update", arguments: {} }); + assert.equal(install.isError, true); + console.log(`${host}/${environment}: relocated startup, UI resource, telemetry opt-out and updater isolation passed.`); + } catch (error) { if (diagnostics) console.error(diagnostics); throw error; } + finally { await client.close(); await transport.close(); } + } +} finally { await rm(temporary, { recursive: true, force: true }); } diff --git a/scripts/vscode-config.mjs b/scripts/vscode-config.mjs new file mode 100644 index 00000000..0ff98abb --- /dev/null +++ b/scripts/vscode-config.mjs @@ -0,0 +1,9 @@ +import { mkdir, writeFile } from "node:fs/promises"; +import { resolve } from "node:path"; +import { localHostConfig } from "./local-host-config.mjs"; + +const config = await localHostConfig("vscode"); +await mkdir(".local-dev", { recursive: true }); +const path = resolve(".local-dev/vscode-mcp.json"); +await writeFile(path, JSON.stringify(config, null, 2) + "\n"); +console.log(`Merge the mobile-dev entry from ${path} into VS Code's local user-profile mcp.json or your project's .vscode/mcp.json, preserving existing servers.`); diff --git a/src/server/plugin.ts b/src/server/plugin.ts index d592bdc8..d1a36244 100644 --- a/src/server/plugin.ts +++ b/src/server/plugin.ts @@ -1,3 +1,4 @@ +import { configuredHost } from "../shared/host.ts"; import { parseResourceInput } from "./resource-input.ts"; import { openRequestSession } from "./request-session.ts"; import type { ServerRequestContext } from "./request-session.ts"; @@ -90,22 +91,28 @@ function guarded(handler: (input: T, context: ServerRequestContext) => Promis } export async function createPlugin(html: string | (() => Promise), baguette = new Baguette(), simulatorInput: SimulatorInput = new SimulatorInputService(udid => baguette.repairInput(udid)), logs = new LogSessions(), android = new ServeEmu(), copyScreenshot = copyPNGToClipboard, providedCpu?: CpuSessions) { + const host = configuredHost(process.env.MOBILE_DEV_HOST); + const standardHost = host !== "codex"; const streams = new StreamSessions(baguette); let selectedCpu = providedCpu; if (selectedCpu === undefined) selectedCpu = await createCpuSessions(); const cpu = selectedCpu; const fps = new DisplayFpsSessions(); const server = new McpServer({ name: "mobile-dev", version: PLUGIN_VERSION }, { - instructions: "For mobile app development, open mobile_open_simulator beside the chat before the first device launch, or reuse the panel. Use selected device IDs. If a task has ambiguous device targets, discover candidates and ask with mobile_choose_devices; proceed only after action accept. For a local CPU or React profile path, use mobile_analyze_profile to show TraceSift findings, then wait for the user to choose Investigate. See the Mobile Dev skill for device control, logs, profiles, and performance workflows.", + instructions: standardHost + ? "For mobile app development, call mobile_open_workspace to show Mobile Dev in the conversation. Use the display modes supported by the host. Use selected device IDs. If device targets are ambiguous, discover candidates and ask the user in chat; wait for their answer before starting the task. For a local CPU or React profile path, use mobile_analyze_profile to show TraceSift findings, then wait for the user to choose Investigate." + : "For mobile app development, open mobile_open_simulator beside the chat before the first device launch, or reuse the panel. Use selected device IDs. If a task has ambiguous device targets, discover candidates and ask with mobile_choose_devices; proceed only after action accept. For a local CPU or React profile path, use mobile_analyze_profile to show TraceSift findings, then wait for the user to choose Investigate. See the Mobile Dev skill for device control, logs, profiles, and performance workflows.", }); instrumentMcpServer(server); - const updates = new PluginUpdates(); + const updates = new PluginUpdates(standardHost ? { enabled: false } : {}); addToolIcons(server, { mobile_open_workspace: PHONE_ICONS, mobile_open_simulator: PHONE_ICONS, mobile_choose_devices: PHONE_ICONS }); registerPluginUpdateTools(server, updates); - const extensions = new OpenAIExtensions(server); - registerDeviceChoiceTools(server, extensions, { - simulators: () => baguette.start(), android: () => android.list(), physicalIos: listIosDevices, - }); + if (!standardHost) { + const extensions = new OpenAIExtensions(server); + registerDeviceChoiceTools(server, extensions, { + simulators: () => baguette.start(), android: () => android.list(), physicalIos: listIosDevices, + }); + } registerIosDeviceTools(server); const closeIosMirror = registerIosMirrorTools(server, APP_URI); registerLogTools(server, logs, baguette); @@ -159,6 +166,7 @@ export async function createPlugin(html: string | (() => Promise), b } function configureUI(content: string): string { + if (standardHost) content = content.replace(" Promise), b ? content.replace('data-view="panel"', 'data-view="workspace"').replace('data-layout="stacked"', 'data-layout="split"') : content, _meta: { - ui: { csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }, + ui: { ...(standardHost ? { permissions: { clipboardWrite: {} } } : {}), csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }, "openai/ui": { preferredDisplayMode: "fullscreen", availableDisplayModes: ["inline", "fullscreen"] }, }, }], @@ -206,7 +214,7 @@ export async function createPlugin(html: string | (() => Promise), b const configured = configureUI(resource.html); const content = configured.replace('data-view="panel"', `data-view="${view}"`); return { contents: [{ uri: uri.href, mimeType: RESOURCE_MIME_TYPE, text: content, _meta: { - ui: { prefersBorder: true, csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }, + ui: { prefersBorder: true, ...(standardHost ? { permissions: { clipboardWrite: {} } } : {}), csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }, "openai/ui": { preferredDisplayMode: "inline", availableDisplayModes: ["inline", "fullscreen"] }, } }] }; }; @@ -249,7 +257,9 @@ export async function createPlugin(html: string | (() => Promise), b registerAppTool(server, "mobile_open_workspace", { title: "Mobile Dev", - description: "Open the fullscreen Mobile Dev workspace when the user asks for fullscreen, with logs and performance tools beside the devices. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. For app development beside a chat, prefer mobile_open_simulator. Starts the bundled backend without booting a device.", + description: standardHost + ? "Open Mobile Dev in the conversation, with devices, logs and performance tools. Expand when supported by the host. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. Starts the bundled backend without booting a device." + : "Open the fullscreen Mobile Dev workspace when the user asks for fullscreen, with logs and performance tools beside the devices. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. For app development beside a chat, prefer mobile_open_simulator. Starts the bundled backend without booting a device.", inputSchema: panelTargetSchema, outputSchema: { ...statusOutput, panelTarget: panelTargetSchema }, annotations: write, _meta: { ui: { resourceUri: WORKSPACE_URI, visibility: ["app", "model"] }, @@ -259,7 +269,9 @@ export async function createPlugin(html: string | (() => Promise), b registerAppTool(server, "mobile_open_simulator", { title: "Mobile Dev", - description: "Open Mobile Dev beside the chat for local mobile app development, before the first device launch unless a panel is open or the workflow is tool-only. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. Starts the bundled backend without booting devices.", + description: standardHost + ? "Open a compact Mobile Dev view in the conversation. Prefer mobile_open_workspace for devices with logs and performance tools. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. Starts the bundled backend without booting devices." + : "Open Mobile Dev beside the chat for local mobile app development, before the first device launch unless a panel is open or the workflow is tool-only. Pass established platform, kind, or discovered deviceId to select the initial device; omit undecided fields. Starts the bundled backend without booting devices.", inputSchema: panelTargetSchema, outputSchema: { ...statusOutput, panelTarget: panelTargetSchema }, annotations: write, _meta: { ui: { resourceUri: APP_URI, visibility: ["app", "model"] }, diff --git a/src/shared/host.ts b/src/shared/host.ts new file mode 100644 index 00000000..0cfc525c --- /dev/null +++ b/src/shared/host.ts @@ -0,0 +1,6 @@ +// Only explicitly configured distributions opt out of Codex behavior. +export function configuredHost(value: string | undefined): "codex" | "claude-desktop" | "cursor" | "vscode" { + if (value === undefined || value === "codex") return "codex"; + if (value === "claude-desktop" || value === "cursor" || value === "vscode") return value; + throw new Error("Unsupported Mobile Dev host."); +} diff --git a/src/ui/app.tsx b/src/ui/app.tsx index 4e637831..607f8da9 100644 --- a/src/ui/app.tsx +++ b/src/ui/app.tsx @@ -1,4 +1,3 @@ -import "@openai/mcp-extensions/app/styles.css"; import "@fontsource-variable/inter"; import "./theme.css"; import "./style.css"; diff --git a/src/ui/chat-message.ts b/src/ui/chat-message.ts new file mode 100644 index 00000000..70cacb48 --- /dev/null +++ b/src/ui/chat-message.ts @@ -0,0 +1,27 @@ +import type { App } from "@modelcontextprotocol/ext-apps"; + +export function preparesChatMessages() { + return typeof document !== "undefined" && document.documentElement?.dataset.host === "vscode"; +} + +export function canMessage(app: Pick) { + const capabilities = app.getHostCapabilities(); + // VS Code implements ui/message but some versions omit message.text in the + // handshake. Limit this adapter to the explicitly configured distribution. + return !!capabilities?.message?.text || !!(preparesChatMessages() && capabilities?.serverTools && capabilities?.serverResources); +} + +export function canUpdateTextContext(app: Pick) { + const context = app.getHostCapabilities()?.updateModelContext; + return !!context?.text || !!(preparesChatMessages() && context); +} + +// Standard hosts receive a self-contained user message. Codex can additionally +// route it to the active composer when it advertises that extension. +export function chatMessage(app: Pick, text: string): Parameters[0] { + const routed = !preparesChatMessages() && app.getHostCapabilities()?.experimental?.["openai/message"] != null; + return { + role: "user", content: [{ type: "text", text }], + ...(routed ? { _meta: { "openai/message": { target: "active", send: true } } } : {}), + }; +} diff --git a/src/ui/comparison-controller.ts b/src/ui/comparison-controller.ts index 9f86bd5d..377242b3 100644 --- a/src/ui/comparison-controller.ts +++ b/src/ui/comparison-controller.ts @@ -1,6 +1,6 @@ +import { chatMessage, canMessage, preparesChatMessages } from "./chat-message.ts"; import type { App } from "@modelcontextprotocol/ext-apps"; import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; -import type { OpenAIMessageParams } from "@openai/mcp-extensions/app"; import { comparisonDuration, comparisonSchema } from "../shared/performance-comparison.ts"; import type { PerformanceComparison } from "../shared/performance-comparison.ts"; import { recordingRangeSchema } from "../shared/recordings.ts"; @@ -36,8 +36,7 @@ export class ComparisonController { } hostChanged() { const capabilities = this.app.getHostCapabilities(); - const canMessage = Boolean(capabilities?.message?.text); - this.update({ canMessage }); + this.update({ canMessage: canMessage(this.app) }); } accept(result: CallToolResult) { if (this.disposed) return; @@ -100,12 +99,12 @@ export class ComparisonController { const reference = JSON.stringify({ recordingIds, title: comparison.title, ...(range ? { range } : {}) }); const interval = range ? `${range.start}–${range.end}s` : "the entire runs"; const prompt = `Compare CPU, memory, and device-wide Display FPS for ${interval}, aligned at recording start. Read the original runs and summaries with mobile_compare_performance_recordings using ${reference}. Account for different durations, devices, apps and memory definitions. Distinguish measurements from hypotheses about their cause; device-wide display data cannot attribute a slowdown to one app.`; - const message: OpenAIMessageParams = { role: "user", content: [{ type: "text", text: prompt }], _meta: { "openai/message": { target: "active", send: true } } }; + const message = chatMessage(this.app, prompt); const startedAt = performance.now(); try { const result = await this.app.sendMessage(message); if (result.isError) throw new Error("The host could not send this comparison to chat."); - countUiEvent("ui.comparison.ask"); + countUiEvent(preparesChatMessages() ? "ui.comparison.ask_prepared" : "ui.comparison.ask"); const elapsed = performance.now() - startedAt; recordUiTiming("ui.comparison.message_ack", elapsed); } catch (error) { diff --git a/src/ui/components/chat-preparation-notice.tsx b/src/ui/components/chat-preparation-notice.tsx new file mode 100644 index 00000000..e5d55309 --- /dev/null +++ b/src/ui/components/chat-preparation-notice.tsx @@ -0,0 +1,5 @@ +import { preparesChatMessages } from "../chat-message.ts"; + +export function ChatPreparationNotice() { + return preparesChatMessages() ?

Questions are prepared in chat. Review the text and press Send to start.

: null; +} diff --git a/src/ui/components/comparison-card.tsx b/src/ui/components/comparison-card.tsx index bb01aacc..2cd08e37 100644 --- a/src/ui/components/comparison-card.tsx +++ b/src/ui/components/comparison-card.tsx @@ -1,3 +1,5 @@ +import { preparesChatMessages } from "../chat-message.ts"; +import { ChatPreparationNotice } from "./chat-preparation-notice.tsx"; import { useLayoutEffect, useMemo, useRef, useState, useSyncExternalStore } from "react"; import type { PointerEvent } from "react"; import { CartesianGrid, Line, LineChart, ReferenceArea, ResponsiveContainer, Tooltip, XAxis, YAxis } from "recharts"; @@ -127,6 +129,7 @@ export function ComparisonCard({ controller }: { controller: ComparisonControlle const hasDisplayData = derived.total.some(run => run.summary !== null && (run.summary.averageFps !== null || run.summary.frameStats !== null)); const differentDurations = comparison.recordings.some(recording => recording.durationSeconds !== derived.duration); return
+

{comparison.title}

{comparison.recordings.length} runs · Aligned at recording start · {derived.duration}s timeline

@@ -164,7 +167,7 @@ export function ComparisonCard({ controller }: { controller: ComparisonControlle {mixedMemory &&

RSS and physical footprint measure different memory definitions and use separate tracks.

} {hasDisplayData &&

Display FPS and frame statistics measure the whole device. Jank rate uses classified presented frames; — means unavailable.

} {comparison.recordings.map(recording => recording.error &&

{recording.title}: {recording.error}

)} -
+
{state.canMessage === false &&

This host does not support sending chart selections to chat.

} {state.error &&

{state.error}

} diff --git a/src/ui/components/context-attachments.tsx b/src/ui/components/context-attachments.tsx new file mode 100644 index 00000000..d848e8d1 --- /dev/null +++ b/src/ui/components/context-attachments.tsx @@ -0,0 +1,20 @@ +import { useState, useSyncExternalStore } from "react"; +import type { PanelContext } from "../model-context.ts"; +import { Button } from "./ui/button"; +import { captureUiError } from "../telemetry.ts"; + +export function ContextAttachments({ context }: { context: PanelContext }) { + const count = useSyncExternalStore(context.subscribe, () => context.attachmentCount); + const [busy, setBusy] = useState(false); + const [error, setError] = useState(false); + if (context.usesHostAttachments || !count) return null; + return
+ + {error && Could not clear attachments. Try again.} +
; +} diff --git a/src/ui/components/display-mode-button.tsx b/src/ui/components/display-mode-button.tsx new file mode 100644 index 00000000..ff9b9ad8 --- /dev/null +++ b/src/ui/components/display-mode-button.tsx @@ -0,0 +1,20 @@ +import type { App } from "@modelcontextprotocol/ext-apps"; +import { useState } from "react"; +import { Button } from "./ui/button"; +import { captureUiError, countUiEvent } from "../telemetry.ts"; + +export function DisplayModeButton({ app }: { app: App }) { + const host = app.getHostContext(); + const [busy, setBusy] = useState(false); + const mode = host?.displayMode === "fullscreen" ? "inline" : "fullscreen"; + if (!host?.availableDisplayModes?.includes(mode)) return null; + return ; +} diff --git a/src/ui/components/logs-view.tsx b/src/ui/components/logs-view.tsx index 73678c64..5732ec9b 100644 --- a/src/ui/components/logs-view.tsx +++ b/src/ui/components/logs-view.tsx @@ -1,3 +1,4 @@ +import { preparesChatMessages } from "../chat-message.ts"; import { CopyIcon, MessageCircleIcon, ArrowDownToLineIcon, CircleAlertIcon, InfoIcon, TriangleAlertIcon, CheckIcon, LayersIcon, ListXIcon, PauseIcon, PlayIcon, RefreshCwIcon, SearchIcon, SlidersHorizontalIcon, TerminalIcon, UnlinkIcon, XIcon, CircleHelpIcon } from "lucide-react"; import { memo, useCallback, useEffect, useRef, useState, useSyncExternalStore } from "react"; import { LegendList, type LegendListRef, type LegendListRenderItemProps } from "@legendapp/list/react"; @@ -77,7 +78,7 @@ export const LogsView = memo(function LogsView({ panel }: { panel: LogsPanel }) { setCopyStatus(""); void copyLog(item); }}>Copy void panel.list.sendToChat(item)}> - {item.level === "error" || item.level === "warn" ? "Fix in chat" : "Ask in chat"} + {preparesChatMessages() ? "Prepare question" : item.level === "error" || item.level === "warn" ? "Fix in chat" : "Ask in chat"} ; @@ -215,7 +216,7 @@ export const LogsView = memo(function LogsView({ panel }: { panel: LogsPanel }) {logs.selected && <> panel.list.select()}> - + } diff --git a/src/ui/components/recording-card.tsx b/src/ui/components/recording-card.tsx index b1a3df66..aa47f8f6 100644 --- a/src/ui/components/recording-card.tsx +++ b/src/ui/components/recording-card.tsx @@ -1,3 +1,5 @@ +import { preparesChatMessages } from "../chat-message.ts"; +import { ChatPreparationNotice } from "./chat-preparation-notice.tsx"; import { useCallback, useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react"; import type { PointerEvent, ReactNode } from "react"; import { Area, AreaChart, CartesianGrid, ReferenceArea, ResponsiveContainer, Tooltip, XAxis, YAxis } from "recharts"; @@ -187,6 +189,7 @@ export function RecordingCard({ controller, detailed = false }: { controller: Re const hasFrames = frameStats !== null && frameStats.frameCount > 0; const hasReadings = hasCpu || hasMemory || hasFps || hasFrames; return
+ {!detailed && }

{recording.title}

{recording.deviceName} · {recording.durationSeconds}s · {savedAt}

{status} @@ -219,7 +222,7 @@ export function RecordingCard({ controller, detailed = false }: { controller: Re } }
- + {detailed === false && } {state.canMessage === false &&

This host does not support sending chart selections to chat.

}
diff --git a/src/ui/components/screen-annotations.tsx b/src/ui/components/screen-annotations.tsx index 0fcd9009..c104e94f 100644 --- a/src/ui/components/screen-annotations.tsx +++ b/src/ui/components/screen-annotations.tsx @@ -1,3 +1,4 @@ +import { preparesChatMessages } from "../chat-message.ts"; import { useState, useSyncExternalStore } from "react"; import { ArrowUpIcon, CircleAlertIcon, MessageCircleIcon, MousePointer2Icon, SendIcon, Trash2Icon } from "lucide-react"; import type { ScreenAnnotationsStore } from "../screen-annotations"; @@ -60,7 +61,7 @@ export function ScreenAnnotationOverlay({ store }: { store: ScreenAnnotationsSto {!!state.annotations.length && !draft &&
{state.annotations.map(annotation =>
)} {state.sendError && } - {state.canSend && } + {state.canSend && }
} ; } diff --git a/src/ui/components/tracesift-cards.tsx b/src/ui/components/tracesift-cards.tsx index 7ddabe02..34a35535 100644 --- a/src/ui/components/tracesift-cards.tsx +++ b/src/ui/components/tracesift-cards.tsx @@ -1,3 +1,5 @@ +import { preparesChatMessages } from "../chat-message.ts"; +import { ChatPreparationNotice } from "./chat-preparation-notice.tsx"; /** * TraceSift's findings inline in the chat: the same card the TraceSift tab draws. * Only the actions differ — Explore opens here, Investigate asks the model. @@ -66,7 +68,7 @@ function CardFooter({ cardId, controller, canExplore }: { cardId: string; contro title={state.canInvestigate ? "Hand this card to the chat and have it read the task." : undefined} onClick={() => void controller.investigate(cardId)} > - {state.busyCard === cardId ? "Starting…" : "Investigate"} + {state.busyCard === cardId ? (preparesChatMessages() ? "Preparing…" : "Starting…") : preparesChatMessages() ? "Prepare investigation" : "Investigate"}
@@ -143,12 +145,14 @@ export function TraceSiftCards({ controller }: { controller: TraceSiftController if (state.explorer) return ; if (!analysis) return
+

Analyzing profile with TraceSift…

{state.error &&

{state.error}

}
; const cpu = analysis.profileType === "cpu"; return
+
TraceSift · {cpu ? "CPU profile" : "React profile"} diff --git a/src/ui/components/tracesift-explorer.tsx b/src/ui/components/tracesift-explorer.tsx index 2090b120..1954ead7 100644 --- a/src/ui/components/tracesift-explorer.tsx +++ b/src/ui/components/tracesift-explorer.tsx @@ -1,3 +1,5 @@ +import { ChatPreparationNotice } from "./chat-preparation-notice.tsx"; +import { preparesChatMessages } from "../chat-message.ts"; /** * The full Explore view inside the card's frame, drawn from the same package the * web app uses. Its data arrives over MCP: this frame cannot reach the service. @@ -27,6 +29,7 @@ export function TraceSiftExplorer({ controller }: { controller: TraceSiftControl const count = explorer.selected?.length ?? 0; const noun = count === 1 ? picks.one : picks.many; return
+