Agent GUIs

Agents need command surfaces built for coordination, review, state, and recovery; terminals are only the power-user bootstrap layer.

The a16z Speedrun "GUIs for Agents" request frames current agent work as MS-DOS-like: CLI sessions, file directories, scattered state, and brittle operator loops. The reviewed video shows the concrete failure class: terminal sessions as the primary control surface, file directories as fragile state, and a "deleted by OpenClaw" example used to make agent state loss legible. Linzumi's YC launch points at the same direction from the team side: shared chat threads that coordinate many coding agents and keep the fleet unblocked. Source: X/@Tocelot, 2026-06-24; Source: local video artifact review, 2026-07-01; Source: YC Launch, 2026-06-24

Routing Summary

Use this page when designing agent operator UIs, multi-agent command rooms, review surfaces, and resumable work controls. Pair with Agent Product Surface, private page, Linzumi, and OpenComputer Durable Agent Sessions; then verify the resulting UI with frontend/design skills and browser proof.

Design Implications

• Show state: what each agent is doing, blocked on, and ready to merge.
• Preserve resumability: every task should survive refreshes, handoffs, and model swaps.
• Make review explicit: diffs, artifacts, screenshots, tests, and approvals are first-class objects.
• Support cancel/retry honestly: long-running agents need visible stop, pause, and resume.
• Keep human steering cheap: command palettes, replies, and annotations beat form-heavy dashboards.

Cross-Surface Agents

Open Tag extends the same idea from coding-agent control rooms into chat clients: Slack and Teams become agent frontends with streaming replies, generated UI, human approvals, and full thread context. Photon Spectrum pushes the surface further into iMessage, SMS, WhatsApp, Telegram, Slack, terminal, and custom providers. The durable pattern is not "put chat everywhere." It is shared run state across clients, with the same approval and provenance vocabulary whether the user is in a web app, a team channel, or a personal messaging thread. Source: X/@ataiiam, 2026-06-25; Source: X/@daniel_tian1, 2026-06-17

Portable Agent Sessions

Omnara adds the portability requirement to the GUI category. A command surface is incomplete if it only shows a running agent while the original laptop, terminal, branch, and credentials stay alive. Long-running coding agents need session rehydration: conversation state, dirty worktree state, branch/ref, staged and untracked files, runtime process state, and approval context must move between local and cloud environments without losing provenance. Source: Omnara sandbox-sync blog, 2026-04-06; Source: X/@ishaansehgal, 2026-05-13

That does not make Omnara an install default. It makes "can the operator recover or resume this run from another screen?" a core review question for any agent GUI.

X Artifacts

Designer-Built Agent Surfaces

Beni's SF design demo dinners add a community-level signal for agent GUIs: designers are prototyping the surfaces they need around agents rather than waiting for platform defaults. The named themes — disposable user interfaces, ultra-configurable UIs, and systems to deslopify LLMs — map directly to this page's control-surface concerns: transient task-specific UI, user-controlled adaptation, and visible quality/review layers around generated output. The local photo artifact is event evidence, not a product demo; the durable content is the theme list and the evidence that designers are hiring/building from those gatherings. Source: X/@ben_issen and self-thread, 2026-06-23; Source: local image review, 2026-07-04

The Clicky Figma Motion demo adds an editor-embedded version of the same GUI category. The useful design rule is that an agent in a professional tool should expose intermediate state in the tool's native vocabulary: selected object, path, target, timeline, and motion properties. Voice alone is not the interface; the canvas proof and reversible controls are the interface. Source: X/@heyclicky, 2026-06-26; Source: local video contact sheet, reviewed 2026-07-04

Fieldwork Pattern

Kevin's July 2026 whiteboard turns these principles into Fieldwork, a concrete visual-development surface: a live website preview, full thread and version history, DOM or visual-region selection, native screenshots and drawing, inspiration/content context, direct design controls, and an outer feedback loop that preserves each result. Fieldwork adds typed annotation layers and distinct pen/highlighter/connector/text/measurement tools, so feedback remains editable and attached to the selected DOM node and UI version. The durable requirement is connected state: feedback must retain the selected UI context, visual evidence, prompt/response provenance, implementation version, and approval or revert action. The private source transcription and reconstructed wireframe live in private page. Source: private July 2026 workflow whiteboard; private Discord message 1527032191513792674, 2026-07-15

Agent cursors

Cua Driver ships six agent cursor motions chosen from 82 built in a playground after studying how people aim a mouse: a comet swoop with a short trail, a signature arc with a click ripple (the default), a spring settle, a magnetic pull toward the target, and an adaptive motion that slows on small targets. The cursor is set per session through MCP tools. Source: X/@trycua thread, 2026-10-05; Source: cua.ai docs, agent cursor tools

The point is legibility. When a person watches an agent drive a desktop, or reviews a recording later, a motion they can follow is how they know what the agent is doing. A teleporting pointer hides the action.

Generated overlays instead of chat

Clicky generates custom overlays, diagrams and interactive 3D scenes directly on the user's screen as they talk, with no chat window, using Sol 6.1 Ultrafast. The demo teaches how LLMs use hardware by drawing over what is already on screen. When a model is fast enough, the answer can be a drawing in place. Ask for a Clearer Format covers asking for diagrams, pages and video instead of prose. Source: X/@FarzaTV, 2026-10-28; Source: heyclicky.com

What a well-made harness includes

A user's list of what makes the Capy coding harness feel like a product rather than a model demo: a captain agent with a digest to catch up and unblock fast, tagging it in Slack to pick up work, a clean multi-repo workflow, seamless local and remote switching, in-app editing of AGENTS.md and skills, guided review, and an agent that actually waits for PR checks and answers review bots. Tailscale and Paper integrations count as taste signals. Source: X/@gergomoricz thread, 2026-10-06

A canvas for LLM conversations, built because chat cannot branch: fork from any point like a git branch without spoiling the main thread. Source: X/@maxleedev, 2025-09-02

A "compostable interface": a temporary, contextual design tool the agent builds to compare variations, collect feedback or explore parameters, then throws away. Work in progress for Impeccable. Source: X/@sheherenow_, 2026-07-20

In an agent workspace, the nav has to prove work is happening, because agents run out of sight. One pattern: the Agents icon wears a live ring when an agent picks up a task, pulsing in real bursts rather than spinning, and hovering shows the current step ("Running tests…"). Ambient status in a bar people already watch, with no modals. Source: X/@uxmateja, 2026-07-16

Lavish opens agent-written HTML in a local review page where you click the exact element or text that is wrong and queue the feedback for the agent; it audits layout for overflow and clipped text first, and ships playbooks for diagrams, tables, plans, diffs and slides. MIT, runs with npx. Source: X/@itsharmanjot, 2026-07-17; Source: X/@itsharmanjot, 2026-07-17

Building on tldraw means you can draw what you want and have the model make it real. tldraw also shipped an offline app, where its team trains small neural nets on the canvas. Source: X/@sheherenow_, 2026-07-18; Source: X/@tldraw, 2026-07-16; Source: X/@tldraw, 2026-07-17; Source: X/@callumflack, 2026-07-18; Source: X/@nicekate8888, 2026-07-18

---

Timeline

• 2026-07-16 | Renamed the Design IDE pattern to Fieldwork and added editable layer/pen semantics for agent-readable visual feedback. Source: Private Discord message 1527032191513792674, 2026-07-15
• 2026-07-14 | Added Kevin's Design IDE pattern: live preview plus DOM selection, visual annotation, versioned conversation/results, design controls, and reversible feedback connected as one stateful review surface. private page
• 2026-07-04 | Added Beni's SF design demo dinner themes as community evidence for designer-built agent surfaces: disposable UIs, ultra-configurable UIs, and systems to deslopify LLMs. Source: X/@ben_issen, 2026-06-23
• 2026-07-04 | Added the Clicky / HeyClicky voice-to-Figma-motion video as editor-embedded agent-GUI evidence: the canvas, selected object, path, target, and motion properties remain visible while voice steers the action. Source: X/@heyclicky, 2026-06-26; Source: local video contact sheet, 2026-07-04
• 2026-07-01 | Re-reviewed the a16z Speedrun GUIs-for-agents video artifact. Preserved the specific MS-DOS-era evidence: terminal sessions, file directories as state, and file deletion/recovery risk as the reason agent command surfaces need resumability and review artifacts. Source: X/@Tocelot, 2026-06-24
• 2026-06-30 | Added Omnara as portable-session evidence: agent GUIs need resumable session state across local/cloud clients, not only a prettier terminal view. Source: X bookmark artifact audit, 2026-06-30
• 2026-06-30 | Promoted Photon Spectrum as messaging-surface evidence for agent GUIs: rich iMessage app surfaces plus versioned TypeScript provider packages. Source: X bookmark artifact audit, 2026-06-30
• 2026-06-30 | Added Open Tag/CopilotKit as cross-surface evidence that agent GUIs include Slack/Teams-style clients, not only web dashboards. Source: X bookmark artifact audit, 2026-06-30
• 2026-06-24 | Captured a16z Speedrun GUIs-for-agents request and Linzumi YC launch as linked evidence for the same product category. Source: X bookmarks, 2026-06-24