ControlAnything: Generative UI and On-Demand Device Integration in Mixed Reality
November 2026 preprint generating both spatial controls and the connector code for physical devices when a Quest 3 wearer requests them. A coding agent tests connections over hand-written MIDI, Bluetooth, and key-press transports, then returns AR widgets, a per-frame hand program, or both.
Interface Quilt: Modifying Applications with Computer Use Agents and Generative UI
November 2026 preprint from Autodesk Research combining computer-use agents and GenUI to build persistent interface patches over existing applications. It recovers controls from rendered pixels, generates new interaction surfaces, and synchronizes them with the underlying app without requiring source code, APIs, or accessibility data.
ABRA: Agent-Bound UI for Human-Agent Communication beyond Chat with Reusable Views
October 2026 EMNLP system demonstration separating reusable graph, table, text, and display views from task-specific interaction meanings. ABRA publishes these bindings as shared canvas state so users and other agents can inspect and invoke task-specific actions.
Inflak: A Unified Orchestration Architecture for Human-Agent Interaction
October 2026 paper separating runtime human-agent interaction into routing, planning, design, and rendering. Inflak implements these layers as composable agent skills, letting task state determine when to request input and which generated interface to present.
Atelier: From Semantic Editing Intentions to Persistent Parametric Controls for AI-Assisted Image Editing
October 2026 HCI paper on adaptive controls for image editing. Atelier combines image context and a user's editing intent to compose persistent, adjustable widgets, exposing proposed parameters for refinement instead of returning only a finished image.
AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
October 2026 paper using generated reference images to guide editable HTML/CSS generation for dashboards, webpages, slides, and posters. AesCode keeps the prompt authoritative for content, represents the design as a graph, and trains with separate programmatic checks and visual rewards.
When Interfaces Speak: Data-Aware Generative UI Harness for Active Interaction
October 2026 paper pairing database tool execution with task-specific frontend generation. GenUI-Harness adds sandboxed interaction rollouts and reward auditing, while UI-TAU Bench tests whether generated interfaces help agents collect input and complete database-backed tasks across ten domains.
Design Creativity Bench: Measuring creativity in LLM-Generated UI
October 2026 Kombai paper evaluating originality, variation across product domains, and requirement satisfaction in LLM-generated interfaces. It compares 2,400 HTML designs from 15 models across 160 prompts with human reference designs, separating acceptable output from diverse design.
Automating HCI Principles: Skills for On-Demand UI Construction
Position paper proposing machine-readable skills that guide agents as they generate task-specific interfaces. Its shared Human-AI workspace connects dialogue, task decomposition, and UI state, while inspectable skill files encode usability heuristics, accessibility criteria, cognitive-load guidance, and mixed-initiative interaction principles.
AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfaces
September 2026 benchmark connecting aesthetic scoring, diagnosis, repair, and text-to-UI generation across 1,395 executable web interfaces. It pairs diagnosis and repair on 660 controlled-degradation cases to test whether models can translate design judgments into effective interface changes.
Elicitive User Interfaces: Designing How Users Shape Generative Interfaces
September 2026 HCI paper proposing that generated interfaces should also generate ways to ask users about unstated preferences. Its six-axis design space and Ditto prototype cover questions, alternatives, editable panels, and other prompts embedded in the interface, with findings from two user studies.
GUIDE: Designer-in-the-loop Authoring of Conformant Generative User Interfaces
September 2026 HCI paper introducing a Figma-based environment for steering generated interfaces toward a designer's intent. GUIDE turns canvas edits into prompt updates and adapts a design-conformance scorer that ranks candidate screens against project references.
Harnessing Generative UI for Education: Tailored Learning Interactives
September 2026 Google Research paper on generating customized STEM simulations with Gemini. Its teacher-led pipeline turns learning objectives into scaffolded, multi-level interfaces, then checks rendered HTML and JavaScript for visual, solution, telemetry, and mechanical failures before refining the result.
IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective
2026 benchmark for evaluating runnable LLM-generated web applications through coverage-guided browser exploration. IWC-Bench instruments each app, turns interaction traces into state-transition graphs, and scores visual aesthetics, usability, and requirement alignment across 369 real-world requirements and 5,088 acceptance criteria.
Toward Frontier-Quality Declarative UI Generation at Small-Model Cost
September 2026 Harness4GenUI paper studying whether small language models can generate production-oriented A2UI from approved component catalogs. Across task-management and cloud-console domains, the authors compare training-data strategies, model sizes, catalog sizes, cost, latency, binding correctness, and rendered quality.
EvoGenUI-Bench: Evaluating Multi-Turn Generative UI Assistants
August 2026 benchmark testing whether LLM assistants can maintain one executable web interface as user requirements evolve. EvoGenUI-Bench contains 150 five-turn tasks across information presentation, stateful interaction, and tool-grounded external state, with browser-based evaluation over screenshots, DOM and source evidence, actor traces, and runtime logs.
Maru: Information Architecture for Aligned, Persistent Generative UI
August 2026 HCI paper introducing information architecture as shared state for Generative UI. Maru captures how users partition, order, name, and prioritize information, turns those choices into persistent rules, and applies them across successive interface generations instead of rebuilding structure from each prompt.
SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation
August 2026 benchmark for evaluating LLM-generated interfaces with deterministic annotations instead of noisy human labels. SchemaGUI synthesizes bilingual instructions and function-call references from parameterized UI schemas, then measures feasibility, geometry, and layout complexity across six scenarios and five models.
RAGE-Vis: A Relation-Aware Generative Editing Interface for Chart Editing
ChinaVis 2026 paper introducing a Generative UI for natural-language chart editing. RAGE-Vis turns bitmap charts into editable parameterized representations, parses composite requests, and generates hierarchical panels with coordinated global and local controls for underspecified editing intents.
LEGOUI: Designing with UI-DSL Bricks for Transparent, Controllable Generation
August 2026 HCI paper introducing a staged Generative UI framework for early design ideation. LEGOUI records prompt-derived and model-inferred decisions in a provenance-aware UI DSL, lets users accept, reject, or add decisions across layout, interaction, relation, and style stages, and renders from the evolving specification.
Beyond a Single Judge: Simulating Social Persona Panels for Generative UI Evaluation
July 2026 paper introducing ESPP, an evaluation method that replaces one LLM judge with a panel of evidence-grounded, psychologically diverse personas. Panelists independently rate generated-interface screenshots, exchange opinions through a bounded-confidence mechanism, and are combined with social weighting.
Design Theater: A Benchmark for Generative UI
July 2026 paper introducing a benchmark for checking whether Generative UI tools implement the design rationales they present to users. Across 120 interfaces from five tools and 24 structural, styling, and functional tasks, the study measures gaps between claimed design decisions and the generated result.
MV-Bench: Benchmarking MLLMs for Coordinated Multi-View Interface Construction
July 2026 benchmark for testing whether multimodal language models can generate executable multi-view visualization interfaces from visual designs. MV-Bench uses Tableau workbooks as structured ground truth and evaluates visual fidelity, data bindings, coordinated interactions, and code executability.
Learning to Detect UI Principle Violations via Reinforcement Learning
July 2026 paper on a lightweight vision-language critic for generated web interfaces. The authors unify 19 principles spanning WCAG 2.2 accessibility, deceptive design, and cognitive usability, then train on roughly 10,000 generated pages with verified synthetic violations.
TaskArtisan: Designing Composable Generative Widgets for LLM-Assisted Analysis
July 2026 HCI paper examining how Generative UI can make long, multi-step chatbot analysis workflows easier to revisit and reuse. TaskArtisan lets people create analysis widgets and compose them into sequential or fan-out workflows, then compares the approach with a conventional chatbot in a 12-participant study.
AI Prototyper: A Figma Plugin for Decomposition-Based GUI Prototyping with LLMs
July 2026 paper introducing AI Prototyper, an open-source Figma plugin that turns natural-language screen descriptions into fully editable prototypes. Its pipeline decomposes a request into reviewable features, retrieves components from a fixed 32-primitive library, generates schema-constrained instances, and renders native Figma layers with auto-layout.
Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control
July 2026 HCI paper introducing Spatula, an LLM-powered system that generates context-aware controls directly on a motion-graphics canvas. Its elastic attribute control space supports progressive discovery, multiple levels of precision, semantic group editing, and new controls on demand.
Rethinking the UI of GenUI: A Tale of Two Designs
June 2026 HCI paper comparing the prompt-first, depth-first, high-fidelity pattern common in Generative UI tools with a structured, breadth-first, low-fidelity alternative. A study with 24 UX designers and product managers surfaces concrete trade-offs for early-stage design exploration.
The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization
June 2026 arXiv paper accepted at the AIED NextGen Learning Interfaces Workshop. It argues that adaptive education should use Generative UI at authoring time, not only at learner runtime, so instructors can verify accessible, modality-specific representations before students encounter them.
TaskLens: Generating Task-Conditioned Scaffolded Interfaces for Learning Professional Creative Software
DIS 2026 / arXiv Paper zu TaskLens, einer LLM-basierten Methode fuer task-conditioned scaffolded UIs in komplexer Kreativsoftware. Das System erkennt Workflow-Stufen und Domaenenkonzepte, waehlt relevante Tools, generiert Implementierungscode und erzeugt Interfaces, die fortgeschrittene Funktionen in Software wie Blender schrittweise offenlegen.
FlowEval: Reference-Based Evaluation of Generated User Interfaces
Apple-affiliated research on evaluating whether generated interfaces support realistic user journeys. FlowEval runs validated tasks on reference websites and generated analogs, compares their navigation traces with reference-based similarity metrics, and tests how well those scores align with expert UI judgments.
Macaron-A2UI: A Model for Generative UI in Personal Agents
Aktuelles arXiv Paper aus 2026 zu runtime Generative UI fuer Personal Agents. Die Autorinnen und Autoren zeigen, warum reiner Chat bei komplexen Agent-Aufgaben an Grenzen stoesst, fuehren ein grosses Generative UI Korpus sowie A2UI-Bench ein und trainieren Modelle, die natuerliche Sprache mit leichtgewichtigen, ausfuehrbaren UI-Aktionen fuer Bestaetigung, Praeferenzklaerung und Multi-Goal-Koordination verbinden.
Efficient Personalization of Generative User Interfaces
Aktuelles arXiv Paper zu einem Kernproblem von GenUI: Personalisierung. Die Studie sammelt Paarvergleiche trainierter Designer ueber 600 generierte UIs, zeigt starke Unterschiede in Designpraeferenzen und stellt ein stichprobeneffizientes Praeferenzmodell vor, das generierte Interfaces besser personalisiert als Baseline-Evaluator und direktes Prompting.
MAIC-UI: Making Interactive Courseware with Generative UI
Aktuelles arXiv Paper ueber ein Zero-Code-GenUI-System fuer interaktive Lernmaterialien. MAIC-UI verwandelt Lehrbuecher, PPTs und PDFs in interaktive Lernseiten, nutzt eine Generate-Verify-Optimize Pipeline fuer didaktische Genauigkeit und unterstuetzt punktgenaue inkrementelle Bearbeitung statt teurer Vollregenerierung.
Generative UI as an Accessibility Bridge: Lessons from C2C E-Commerce
arXiv / CHI 2026 Workshop Paper dazu, wie Runtime Generative UI Accessibility-Luecken in user-generierten Marketplace-Inhalten schliessen kann. Auf Basis von sechs Studien mit blinden, sehbehinderten und aelteren Nutzern beschreibt es angepasste Interfaces, Audio Guidance und regeneriertes HTML als policy-getriebene Accessibility Interventionen.
What does Generative UI mean for HCI Practice?
CHI EA 2026 workshop paper from Microsoft Research and academic HCI researchers on how AI-generated interfaces may reshape design methods, workflows, and user experiences. It is useful because it frames GenUI as a practice-level shift for researchers, designers, developers, and product teams, not only a model-output technique.
Software as Content: Dynamic Applications as the Human-Agent Interaction Layer
arXiv Paper aus 2026, das argumentiert, dass Chat allein kein gutes Medium fuer strukturierte, zustandsbehaftete Human-Agent-Arbeit ist. Software as Content beschreibt dynamisch generierte agentische Anwendungen als primaere Interaktionsschicht, die ueber Turns hinweg bestehen bleiben und direkt nutzbare Controls anbieten statt alles in linearen Text zu pressen.
The Keyhole Effect: Why Chat Interfaces Fail at Data Analysis
arXiv Paper aus 2026, das Chat als schwache Standardoberflaeche fuer mehrstufige, zustandsbehaftete Analysearbeit beschreibt. Es liefert eine kognitionswissenschaftliche Erklaerung fuer die Bruchstellen linearer Konversationen und positioniert Generative UI sowie andere hybride Interaktionsmuster als Wege zu mehr Sichtbarkeit, raeumlichem Gedaechtnis und Nutzerkontrolle.
Gradual Generation of User Interfaces as a Design Method for Malleable Software
arXiv Paper aus 2026 ueber Gradually Generating User Interfaces, eine GenUI-Designmethode fuer malleable software. Der Ansatz fuegt waehrend der Generierung Zwischenschichten ein, damit Nutzer Anpassungsoptionen entdecken und steuern koennen, ohne von einem riesigen Prompt-Feld oder dichten Menues ueberfordert zu werden.
AlignUI: A Method for Designing LLM-Generated UIs Aligned with User Preferences
2026 arXiv paper on aligning LLM-generated interfaces with user tasks and preferences. The authors collect 720 UI-control preferences from 50 users across image-editing tasks, use that dataset to guide model reasoning, and evaluate generated UIs with 72 additional users.
Generative UI: LLMs are Effective UI Generators
Google Research Paper, das das Konzept Generative UI frueh systematisch ausformuliert. Es beschreibt, wie AI Modelle nicht nur Inhalte, sondern ganze Nutzererfahrungen erzeugen koennen, und zeigt in Experimenten, dass LLMs mit passenden Prompts und Tools hochwertige interaktive Interfaces generieren, die in vielen Faellen menschlichen Expertenergebnissen nahekommen. Das Paper veroeffentlicht ausserdem den PAGEN Datensatz.
Improving User Interface Generation Models from Designer Feedback
Research paper on training LLM-based UI generators with feedback that matches professional design practice. The authors compare rankings, comments, sketches, and direct revisions from 21 designers, then use the resulting data to improve code models that generate rendered interfaces.
Generative Interfaces for Language Models
ACL 2026 Findings Paper, das ein Paradigma beschreibt, in dem LLMs proaktiv aufgabenspezifische Interfaces generieren statt nur Textantworten zu liefern. Die Arbeit kombiniert interface-spezifische Repraesentationen mit iterativer Verfeinerung und berichtet, dass menschliche Evaluatoren generative Interfaces bei informationsdichten und explorativen Aufgaben deutlich gegenueber reinen Konversationen bevorzugten.
Towards a Working Definition of Designing Generative User Interfaces
DIS 2025 Paper, das ueber Literaturreview und Fallanalyse eine fruehe Arbeitsdefinition von Generative UI (GenUI) entwickelt. Es beschreibt GenUI als neues Paradigma der Interface-Erstellung, in dem Menschen und AI zur Designzeit Interfaces erzeugen und Nutzer zur Laufzeit mit AI-generierten Interfaces interagieren.
Generative and Malleable User Interfaces with Generative AI
CHI 2025 Paper ueber generative and malleable user interfaces auf Basis von Nutzeraufgaben. Die Autorinnen und Autoren entwerfen ein task-driven data model, in dem LLMs Prompts interpretieren, UI-Spezifikationen erzeugen und diese auf konkrete Interfaces abbilden. Experimente zeigen dynamische Elemente wie Formulare und Visualisierungen sowie natuerlichsprachliche Nachbearbeitung.
BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks
VL/HCC 2024 Paper von Apple Forschenden zu BISCUIT, einem JupyterLab Workflow mit einer temporaeren UI-Schicht zwischen Nutzerintention und LLM-generiertem Code. Statt direkt vom Prompt in Quelldateien zu springen, interagieren Nutzer zunaechst mit generierten Controls, um Optionen zu verstehen, Anforderungen zu praezisieren und Variablen zu erkunden.