Empirical Study
Traditional Interfaces vs. Conversational AI
Comparing Intent-Driven and Interface-Driven Interaction: An Empirical Study of Traditional UI and Conversational AI Using the Model Context Protocol (MCP)
Two ways to complete the same tasks: click through menus and forms, or tell an AI what you need in plain language. We're studying which approach works better—and for whom.
Empirical comparison of interaction modalities
We compare a traditional web interface (menus, forms, buttons) with an MCP-based conversational system. Both handle the same four tasks—register or drop a course, book or cancel a facility room—and share the same backend, so any differences come from how you interact, not what the system can do.
Functional Equivalence
Shared backend, differences arise from modality, not functionality.
Which made you feel more in control?
Where was it clearer what the system did on your behalf?
Which behaved more predictably?
Which would you trust more without close supervision?
Which would you prefer depending on the task?
Traditional UI System
Web-based course registration and facility booking using standard navigation menus, forms, and buttons.
Facility Booking
Book or cancel facility rooms through menus and forms.
Course Registration
Register or drop courses through menus and forms.
Express Intent in Natural Language
Say what you need—e.g., "Book a study room tomorrow at 2 PM"—and the system interprets your request and carries it out via MCP. No menus or forms to navigate.
Intent-Driven
Natural language requests are turned into MCP tool calls
Facility Booking
Book or cancel rooms by describing what you need
Course Registration
Register or drop courses by expressing your intent
Shared Backend
Same database as the traditional interface
Traditional UI vs MCP Conversational System
Both systems share the same backend and business logic, so differences come from how you interact, not what the system does.
| Aspect | Traditional | MCP Chat |
|---|---|---|
| Interface | Menus, forms, buttons | Chat, natural language |
| Execution | Navigate to the page, select options, submit forms | Describe your goal; system interprets and executes via MCP |
| Backend | REST / GraphQL | MCP |
| Control | Higher for general users | Higher for advanced users |
| Trust | More trusted across users | Lower for unsupervised use |
| Data | Shared database | Same database |
System Architecture and Workflow
Three layers: user interaction (web apps + chat agent), application services (REST/GraphQL + MCP), and data/analytics. Both pathways connect to the same database.
System Design
Two web applications and a conversational agent share one backend. The same database powers both interaction paths, so we can compare how modality affects experience—not functionality.
Interaction
Facility Booking
Course Registration
Chat Agent
Service
REST / GraphQL
MCP Tool Invocation
Backend
Data
Shared Database
Task Time & Progress
Survey & Logs
Functional Equivalence
Same backend; differences come from modality, not functionality.
Unified Analytics
We record task time, progress, interaction logs, and survey responses so we can compare both systems directly.
Experimental Platform
Open-source prototype. Within-subject design with counterbalanced order.
Participate in the Experiment
Complete the same tasks with both systems. We measure usability, how in control you feel, and cognitive workload using standard scales (SUS, SDT, NASA-TLX).
What You'll Do
- •Pre-task questionnaire (demographics)
- •Four tasks: register or drop a course, book or cancel a facility room
- •SUS, SDT, and NASA-TLX after each system
- •Comparative preference questions
Why Participate
- •Contribute to HCI research
- •Help shape how conversational AI and traditional interfaces work together in the future
Takes about 15-20 minutes
Main Contributions
Empirical insights into how conversational AI and traditional interfaces may coexist in future hybrid interaction environments.
Empirical Comparison
Controlled comparison of MCP-based conversational interaction and traditional web interfaces.
Prototype System
A system where you can complete the same tasks through either menus and forms or natural language.
Hybrid Design Insights
How conversational AI and traditional interfaces can coexist—and when each works best.
SUS, SDT, NASA-TLX
Usability, perceived autonomy and competence, and cognitive workload.