ControlAnything: Generative UI and On-Demand Device Integration in Mixed Reality
November 2026 preprint generating both spatial controls and the connector code for physical devices when a Quest 3 wearer requests them. A coding agent tests connections over hand-written MIDI, Bluetooth, and key-press transports, then returns AR widgets, a per-frame hand program, or both.
Interface Quilt: Modifying Applications with Computer Use Agents and Generative UI
November 2026 preprint from Autodesk Research combining computer-use agents and GenUI to build persistent interface patches over existing applications. It recovers controls from rendered pixels, generates new interaction surfaces, and synchronizes them with the underlying app without requiring source code, APIs, or accessibility data.
ABRA: Agent-Bound UI for Human-Agent Communication beyond Chat with Reusable Views
October 2026 EMNLP system demonstration separating reusable graph, table, text, and display views from task-specific interaction meanings. ABRA publishes these bindings as shared canvas state so users and other agents can inspect and invoke task-specific actions.
Inflak: A Unified Orchestration Architecture for Human-Agent Interaction
October 2026 paper separating runtime human-agent interaction into routing, planning, design, and rendering. Inflak implements these layers as composable agent skills, letting task state determine when to request input and which generated interface to present.
Atelier: From Semantic Editing Intentions to Persistent Parametric Controls for AI-Assisted Image Editing
October 2026 HCI paper on adaptive controls for image editing. Atelier combines image context and a user's editing intent to compose persistent, adjustable widgets, exposing proposed parameters for refinement instead of returning only a finished image.
AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
October 2026 paper using generated reference images to guide editable HTML/CSS generation for dashboards, webpages, slides, and posters. AesCode keeps the prompt authoritative for content, represents the design as a graph, and trains with separate programmatic checks and visual rewards.
When Interfaces Speak: Data-Aware Generative UI Harness for Active Interaction
October 2026 paper pairing database tool execution with task-specific frontend generation. GenUI-Harness adds sandboxed interaction rollouts and reward auditing, while UI-TAU Bench tests whether generated interfaces help agents collect input and complete database-backed tasks across ten domains.
Design Creativity Bench: Measuring creativity in LLM-Generated UI
October 2026 Kombai paper evaluating originality, variation across product domains, and requirement satisfaction in LLM-generated interfaces. It compares 2,400 HTML designs from 15 models across 160 prompts with human reference designs, separating acceptable output from diverse design.
Automating HCI Principles: Skills for On-Demand UI Construction
Position paper proposing machine-readable skills that guide agents as they generate task-specific interfaces. Its shared Human-AI workspace connects dialogue, task decomposition, and UI state, while inspectable skill files encode usability heuristics, accessibility criteria, cognitive-load guidance, and mixed-initiative interaction principles.
AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces
September 2026 benchmark connecting aesthetic scoring, diagnosis, repair, and text-to-UI generation across 1,395 executable web interfaces. It pairs diagnosis and repair on 660 controlled-degradation cases to test whether models can translate design judgments into effective interface changes.
Elicitive User Interfaces: Designing How Users Shape Generative Interfaces
September 2026 HCI paper proposing that generated interfaces should also generate ways to ask users about unstated preferences. Its six-axis design space and Ditto prototype cover questions, alternatives, editable panels, and other prompts embedded in the interface, with findings from two user studies.
GUIDE: Designer-in-the-loop Authoring of Conformant Generative User Interfaces
September 2026 HCI paper introducing a Figma-based environment for steering generated interfaces toward a designer's intent. GUIDE turns canvas edits into prompt updates and adapts a design-conformance scorer that ranks candidate screens against project references.
Harnessing Generative UI for Education: Tailored Learning Interactives
September 2026 Google Research paper on generating customized STEM simulations with Gemini. Its teacher-led pipeline turns learning objectives into scaffolded, multi-level interfaces, then checks rendered HTML and JavaScript for visual, solution, telemetry, and mechanical failures before refining the result.
IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective
2026 benchmark for evaluating runnable LLM-generated web applications through coverage-guided browser exploration. IWC-Bench instruments each app, turns interaction traces into state-transition graphs, and scores visual aesthetics, usability, and requirement alignment across 369 real-world requirements and 5,088 acceptance criteria.
Toward Frontier-Quality Declarative UI Generation at Small-Model Cost
September 2026 Harness4GenUI paper studying whether small language models can generate production-oriented A2UI from approved component catalogs. Across task-management and cloud-console domains, the authors compare training-data strategies, model sizes, catalog sizes, cost, latency, binding correctness, and rendered quality.
EvoGenUI-Bench: Evaluating Multi-Turn Generative UI Assistants
August 2026 benchmark testing whether LLM assistants can maintain one executable web interface as user requirements evolve. EvoGenUI-Bench contains 150 five-turn tasks across information presentation, stateful interaction, and tool-grounded external state, with browser-based evaluation over screenshots, DOM and source evidence, actor traces, and runtime logs.
Maru: Information Architecture for Aligned, Persistent Generative UI
August 2026 HCI paper introducing information architecture as shared state for Generative UI. Maru captures how users partition, order, name, and prioritize information, turns those choices into persistent rules, and applies them across successive interface generations instead of rebuilding structure from each prompt.
SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation
August 2026 benchmark for evaluating LLM-generated interfaces with deterministic annotations instead of noisy human labels. SchemaGUI synthesizes bilingual instructions and function-call references from parameterized UI schemas, then measures feasibility, geometry, and layout complexity across six scenarios and five models.
RAGE-Vis: A Relation-Aware Generative Editing Interface for Chart Editing
ChinaVis 2026 paper introducing a Generative UI for natural-language chart editing. RAGE-Vis turns bitmap charts into editable parameterized representations, parses composite requests, and generates hierarchical panels with coordinated global and local controls for underspecified editing intents.
LEGOUI: Designing with UI-DSL Bricks for Transparent, Controllable Generation
August 2026 HCI paper introducing a staged Generative UI framework for early design ideation. LEGOUI records prompt-derived and model-inferred decisions in a provenance-aware UI DSL, lets users accept, reject, or add decisions across layout, interaction, relation, and style stages, and renders from the evolving specification.
Beyond a Single Judge: Simulating Social Persona Panels for Generative UI Evaluation
July 2026 paper introducing ESPP, an evaluation method that replaces one LLM judge with a panel of evidence-grounded, psychologically diverse personas. Panelists independently rate generated-interface screenshots, exchange opinions through a bounded-confidence mechanism, and are combined with social weighting.
Design Theater: A Benchmark for Generative UI
July 2026 paper introducing a benchmark for checking whether Generative UI tools implement the design rationales they present to users. Across 120 interfaces from five tools and 24 structural, styling, and functional tasks, the study measures gaps between claimed design decisions and the generated result.
MV-Bench: Benchmarking MLLMs for Coordinated Multi-View Interface Construction
July 2026 benchmark for testing whether multimodal language models can generate executable multi-view visualization interfaces from visual designs. MV-Bench uses Tableau workbooks as structured ground truth and evaluates visual fidelity, data bindings, coordinated interactions, and code executability.
Learning to Detect UI Principle Violations via Reinforcement Learning
July 2026 paper on a lightweight vision-language critic for generated web interfaces. The authors unify 19 principles spanning WCAG 2.2 accessibility, deceptive design, and cognitive usability, then train on roughly 10,000 generated pages with verified synthetic violations.
TaskArtisan: Designing Composable Generative Widgets for LLM-Assisted Analysis
July 2026 HCI paper examining how Generative UI can make long, multi-step chatbot analysis workflows easier to revisit and reuse. TaskArtisan lets people create analysis widgets and compose them into sequential or fan-out workflows, then compares the approach with a conventional chatbot in a 12-participant study.
AI Prototyper: A Figma Plugin for Decomposition-Based GUI Prototyping with LLMs
July 2026 paper introducing AI Prototyper, an open-source Figma plugin that turns natural-language screen descriptions into fully editable prototypes. Its pipeline decomposes a request into reviewable features, retrieves components from a fixed 32-primitive library, generates schema-constrained instances, and renders native Figma layers with auto-layout.
Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control
July 2026 HCI paper introducing Spatula, an LLM-powered system that generates context-aware controls directly on a motion-graphics canvas. Its elastic attribute control space supports progressive discovery, multiple levels of precision, semantic group editing, and new controls on demand.
Rethinking the UI of GenUI: A Tale of Two Designs
June 2026 HCI paper comparing the prompt-first, depth-first, high-fidelity pattern common in Generative UI tools with a structured, breadth-first, low-fidelity alternative. A study with 24 UX designers and product managers surfaces concrete trade-offs for early-stage design exploration.
The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization
June 2026 arXiv paper accepted at the AIED NextGen Learning Interfaces Workshop. It argues that adaptive education should use Generative UI at authoring time, not only at learner runtime, so instructors can verify accessible, modality-specific representations before students encounter them.
TaskLens: Generating Task-Conditioned Scaffolded Interfaces for Learning Professional Creative Software
DIS 2026 / arXivの論文です。TaskLensは、複雑なcreative software向けにtask-conditioned scaffolded UIを生成するLLMベースの方法です。workflow stageとdomain conceptを特定し、関連toolを選び、実装codeを生成して、Blenderのようなsoftware内で高度な機能を段階的に開示するinterfaceを作ります。
FlowEval: Reference-Based Evaluation of Generated User Interfaces
Apple-affiliated research on evaluating whether generated interfaces support realistic user journeys. FlowEval runs validated tasks on reference websites and generated analogs, compares their navigation traces with reference-based similarity metrics, and tests how well those scores align with expert UI judgments.
Macaron-A2UI: A Model for Generative UI in Personal Agents
パーソナルAgentにおける実行時Generative UI（ジェネレーティブUI）を扱う、2026年の新しいarXiv論文です。通常のチャットだけでは複雑なAgentタスクのボトルネックになると捉え、大規模なGenerative UIコーパス、A2UI-Bench、自然言語と軽量な実行可能UI actionを同時に生成するモデルを提示し、生成AI UIデザインの研究基盤としても参照できます。
Efficient Personalization of Generative User Interfaces
GenUIの未解決課題であるパーソナライズに焦点を当てたarXiv論文です。同じ600件の生成UIに対して訓練済みデザイナーのペアワイズ評価を集め、デザイン嗜好のばらつきを示したうえで、少ないサンプルで個人化できる preference model を提案します。
MAIC-UI: Making Interactive Courseware with Generative UI
教育向けcoursewareを対象にした、zero-code GenUIシステムのarXiv論文です。MAIC-UIは教科書、PPT、PDFをインタラクティブな学習ページへ変換し、generate-verify-optimizeパイプラインで教育内容の正確性を保ちながら、click-to-locateの差分編集を可能にします。
Generative UI as an Accessibility Bridge: C2C e-commerceからの教訓
user-generatedなmarketplace contentにおけるaccessibility gapを、runtime Generative UIがどう埋められるかを論じるarXiv / CHI 2026 workshop paperです。blind、low-vision、高齢ユーザーを対象にした6つの研究をもとに、adapted interface、audio guidance、regenerated HTMLをpolicy-drivenなaccessibility interventionとして位置づけます。
What does Generative UI mean for HCI Practice?
CHI EA 2026 workshop paper from Microsoft Research and academic HCI researchers on how AI-generated interfaces may reshape design methods, workflows, and user experiences. It is useful because it frames GenUI as a practice-level shift for researchers, designers, developers, and product teams, not only a model-output technique.
Software as Content: Dynamic Applications as the Human-Agent Interaction Layer
構造化され状態を持つ人間とAgentの作業に、チャットだけでは不十分だと論じる2026年のarXiv論文です。Software as Contentという考え方のもと、動的に生成されるagentic applicationを主なインタラクション層として扱い、turnをまたいで持続する操作可能なUIを提案します。
The Keyhole Effect: Why Chat Interfaces Fail at Data Analysis
複数ステップで状態を持つ分析作業に、チャットを標準UIとして使う危うさを論じる2026年のarXiv論文です。線形会話がどこで破綻するかを認知科学の観点から説明し、Generative UIやhybrid interaction patternが可視性、空間記憶、ユーザー制御を取り戻す方法だと位置づけます。
Gradual Generation of User Interfaces as a Design Method for Malleable Software
GenUIをmalleable softwareのデザイン手法として捉える、2026年のarXiv論文です。巨大なprompt boxや複雑なmenuに利用者を押し込めるのではなく、生成途中に中間的なUI layerを挿入し、カスタマイズ可能性を段階的に発見・制御できるようにします。
AlignUI: A Method for Designing LLM-Generated UIs Aligned with User Preferences
2026 arXiv paper on aligning LLM-generated interfaces with user tasks and preferences. The authors collect 720 UI-control preferences from 50 users across image-editing tasks, use that dataset to guide model reasoning, and evaluate generated UIs with 72 additional users.
Generative UI: LLMs are Effective UI Generators
Generative UIの概念を体系的に示したGoogle Researchの論文です。AI modelがcontentだけでなくuser experience全体を生成するという考え方を定義し、適切なpromptとtoolを使えば、LLMが高品質なcustom interactive interfaceを生成できることを実験で示します。生成UI評価用のPAGEN datasetも公開されています。
Improving User Interface Generation Models from Designer Feedback
Research paper on training LLM-based UI generators with feedback that matches professional design practice. The authors compare rankings, comments, sketches, and direct revisions from 21 designers, then use the resulting data to improve code models that generate rendered interfaces.
Generative Interfaces for Language Models
LLMがテキストだけで返答するのではなく、タスク固有のinterfaceを能動的に生成するparadigmを提案するACL 2026 Findings論文です。interface向け表現と反復的なrefinementを組み合わせ、情報密度の高いタスクや探索タスクで、会話型UIよりgenerative interfacesが高く評価されたと報告しています。
Towards a Working Definition of Designing Generative User Interfaces
文献レビューとcase analysisを通じて、Generative UI（GenUI）のworking definitionを提案したDIS 2025論文です。設計時には人間とAIが協調してinterfaceを生成し、実行時にはユーザーがAI生成interfaceと対話するという新しいinterface creation paradigmとして整理しています。
Generative and Malleable User Interfaces with Generative AI
ユーザーのタスクに応じてAIがmalleable user interfacesを生成する方法を提案したCHI 2025論文です。LLMがpromptからユーザー意図を解析し、UI specificationを生成して具体的なinterfaceへmappingするtask-driven data modelを設計しています。
BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks
Apple研究者によるVL/HCC 2024論文で、JupyterLab workflowに一時的なUI layerを挿入するBISCUITを紹介しています。promptからすぐsource codeへ飛ぶのではなく、生成されたtemporary controlsを先に操作し、選択肢を理解し、要求を洗練し、変数を探索してからcodeを生成します。