AI UX Patterns for Design Systems (part 2)

Design systems need an AI upgrade

👋 Get weekly insights, tools, and templates to help you build and scale design systems. More: Design Tokens Mastery Course / Podcast / My Linkedin

---

In this part, we’ll go through the following patterns:

• Visual Workflow Builders (Canvas)

• Voice Interfaces

• Prompt-to-Output / Generation Interfaces

• Iterative Prompting/Feedback

• Context-Aware UI

• Assistants

• Multi-Agent Workflows

Images available on my blog (because this newsletter hit the limit)

Part 1 (on blog) →

Part 2 (on blog) →

Canvas builder

Interfaces where users can connect and automate AI agents visually through drag-and-drop tools.

Examples:

• Connecting a sentiment analysis model to trigger an email draft generator

• Creating a voice transcription → summarization → action item extraction pipeline (Check Gumloop)

• Connecting agents for creating a Yosemite family trip (Check Cove)

Tools: Make.com, Zapier AI actions, LangFlow, Cove, Gumloop

Interaction Patterns:

• Drag-and-drop interfaces for connecting AI agents

• Visual representation of data flow and logic

• Real-time previews of AI operations

• Version control for workflows

• Node-specific debugging and inspection tools

• Templates for common workflow patterns

• Permission controls for shared workflows (for every agent)

• Smart connectors that handle data transformation between agents
  

😅 Challenges:

• Visualizing processes

• Balancing simplicity with advanced control (different skill levels)

• Providing appropriate feedback (for single agent/node)

• Ensuring consistent performance across nodes and agents

• Managing compute/resource allocation

• Supporting collaboration and editing (imagine multi-user editing in Figma)
  

Voice Interfaces

Interfaces where users primarily interact via speech.

Examples

• Hands-free workflows for warehouse or field workers

• AI phone agents making calls to reschedule or collect data (Check Retell AI)

Interaction Patterns:

• Clear listening indicators

• Training conversation flows (and defining steps)

• Interruption handling and recovery

• Confirmation steps (only voice?)

• Fallbacks for noisy environments or failed recognition

• Transcripts for visibility

Tools: Google Assistant, Retell AI, Descript (Overdub), Otter.ai, Vapi

😅 Challenges:

• No visual feedback about system status or understanding

• Indicator that AI is actively listening

• Device is on mute → failure

• Latency can be frustrating (conversation can feel robotic)

• Maintaining context across fragmented or broken inputs

• Accessibility for users with speech impairments

• Making the system’s understanding visible (supporting multi-modality)

Prompt-to-output Interfaces

Systems that generate complex content (e.g., images, videos, designs, or code) from simple prompts.

Examples:

• Writing “Landing page for a fintech app” and getting a full layout

• Generating a blog post from a topic prompt

• Asking, “Make a 1-minute highlight reel from this 20-minute video”

Tools: Figma AI, Framer AI, Canva Magic, Notion AI, Claude, Lovable

More examples and tools in my previous newsletter →

Interaction Patterns:

• Progressive refinement controls (e.g., sliders, edit prompts)

• Previews before high-fidelity generation (especially for videos → generating low-fidelity for version 1)

• Cost and time estimates before processing

• Libraries of templates

• History tracking of successful generations

• Output versioning and branching

• Export in multiple formats

• Loading animations during long generations or clear feedback on how much time the process will take (with notification via email when the job is done)
  

😅 Challenges:

• Long processing times require clear progress feedback

• Resource-heavy outputs may consume user credits or quotas

• Initial outputs often need substantial refinement

• Managing expectations about output fidelity '

• Giving users meaningful control without overloading them

• Allowing for low-fidelity drafts during iteration

• Handling harmful, low-quality, or biased outputs safely
  
  

Iterative Prompting

Interfaces where users refine generated outputs over time—with branching, feedback, or manual edits.

Examples:

• Highlighting a paragraph and asking, “Change the tone… ”

• Giving feedback like “More playful” or “Add stats” to a generated email

• Comparing different versions of a product name and merging favorites

Tools: Figma AI (copy plugins), Canva, Framer AI, Copilot Chat, Midjourney Remix

Interaction Patterns:

• Highlight-and-refine tools

• Smart merging of user edits and AI generations

• Version history with a visual diff

• Undo/redo across versions

• Inline feedback fields (e.g., “make this shorter”)
  

😅 Challenges:

• Tracking small refinements without losing intent

• Handling branching versions and edits

• Encouraging exploration without wasting tokens/credits

• Helping users understand what can be edited

• Preventing version fatigue or loops/hallucinations
  

Context-Aware Interfaces

Systems that adapt interface behavior based on user actions, context, and patterns.

Examples:

• A writing tool suggesting citations when it sees “According to...”

• An email tool nudging you to attach a file after writing “see attached.”

• Canvas adapted to your use case

Tools: Github Copilot, GrammarlyGO, Notion AI, Superhuman, Zuni

Interaction Patterns:

• Dynamic layouts

• Context-sensitive help and guidance

• Override options to take manual control

• Interfaces that improve with repeated use

😅 Challenges:

• Balancing predictability with intelligent adaptability

• Managing cognitive load when interfaces shift (when to hide toolbars, etc.)

• How to check UX, if layout changes based on X factors

• How to respect accessibility needs
  
  

AI Assistants (inline, onboarding)

Personalized, intelligent guides embedded in your product.

• Onboarding tutorials that evolve based on user behavior

• Suggest the next steps after user analysis (e.g., “Want to create a chart from this data?”)

• Giving live performance tips during design or writing tasks

Tools: Notion AI, GrammarlyGO, Microsoft Copilot, Framer AI onboarding flows

Interaction Patterns:

• Contextual suggestions

• Progressive disclosure of advanced features (beginner vs expert)

• Multi-modal delivery: text, tooltip, chat, walkthrough

• Learning from user preferences over time

😅 Challenges:

• Helping without annoying

• Supporting voice, chat, visual, and written instructions

• Building credibility: when to speak up vs. stay silent
  

Multi-Agent Workflows

Systems that coordinate multiple AI agents to complete tasks together.

Examples:

• Generating a report: one agent researches, another summarizes, a third formats the doc

• Chained bots: detect mood → adjust copy tone → regenerate visual

Tools: LangChain, AutoGen, Make.com, CrewAI, Zapier

Interaction Patterns:

• Modular agents with defined roles and connectors

• Branching logic for decisions

• Logs, previews, and agent-specific debugging

• Shared workflows editable by multiple users

😅 Challenges:

• Explaining what each agent does and why

• Debugging workflows when multiple agents are involved

• Managing cost and latency

• Preventing circular logic or agent conflict

• Showing history and traceability clearly (+rollbacks)

🧠 All in all, we need to start adding:

• AI component libraries (prompt boxes, loading indicators, feedback controls)

• AI-specific design tokens -> update naming conventions

• Accessibility expectations for AI-generated interactions

• Decide how to document AI behaviors

• Guidelines for context-aware components – how to prototype, test, and scale components that adapt to conversations and user behavior

• Think about cognitive load when adding personalized/adaptive interfaces

• Rules for what AI can create, change, and when attribution is needed

Enjoy the weekend, and stay tuned for part 3. ✌️

---

---

💎 Community Gems

Figma Variables Resources for designers

🔗 Link

Supernova introduced analytics

Supernova now comes with built-in documentation analytics — so you can finally stop guessing whether people are actually reading your design system docs ✨

🔗 Link

The existential challenge of design system team morale

by PJ Onori

🔗 Link