Building AI Trio: A Multi-LLM Debate Platform Built with AI-Assisted Development
I kept running into the same frustration: every LLM has blind spots. Ask GPT a question, you get GPT’s particular style and biases. Same with Claude, same with Gemini. The answers aren’t wrong, but they’re shaped by each model’s training, tendencies, and assumptions.
I wanted a tool where I could ask all three the same question simultaneously and watch them debate each other - like a group chat where AI models challenge each other’s reasoning.
But I also had a second question I wanted to answer: Can a PM really go from idea to working prototype using AI-assisted development, without a large engineering team?
So I built AI Trio to answer both questions.
Spoiler: The answer to the second question is “yes, but…” and that “but” is where things get interesting.
The Problem: Single-Model Blind Spots
When you ask ChatGPT a question, you get ChatGPT’s answer. It sounds authoritative. It’s well-written. But it carries that model’s particular biases, knowledge gaps, and reasoning patterns.
The same applies to Claude, Gemini, or any other model. Each has strengths and weaknesses. Each makes different assumptions. Each emphasizes different tradeoffs.
For simple factual questions, this doesn’t matter much. But for ambiguous decisions - “Should I buy or lease a car?”, “What’s the right architecture for this system?”, “How should I approach this negotiation?” - a single model’s answer hides the complexity of the decision.
You’re not getting the full picture. You’re getting one perspective presented as the answer.
The Product: Multi-Model Debate
AI Trio puts GPT-5, Gemini, and Claude in the same conversation. You ask a question, and all three respond simultaneously. They can see each other’s answers and challenge each other’s reasoning.
A sample chat session: asking “Should I buy or lease a new car?” Each model frames the tradeoffs differently - GPT focuses on ownership vs. payments, Gemini adds driving habits, Claude emphasizes financial situation.
Core features:
- Multi-Model Streaming: Real-time responses from three models simultaneously, each aware of the others' answers
- Voice Room: Talk to all three models at once - like a conference call with AI. You speak, they all respond, and you can have a back-and-forth conversation
- Context-Aware Web Search: Automatic web search when models detect they need current information
- Comparison Mode: Side-by-side evaluation of responses and reasoning styles
- Multi-Layer Safety: Defense-in-depth approach with prompt injection detection, output validation, and rate limiting
The key insight is that disagreement reveals more than consensus. When all three models align, it confirms the obvious. When they diverge, it surfaces hidden tradeoffs and assumptions that single answers mask.
How It Works
The architecture is built around real-time streaming. When you submit a question, it fans out to all three model APIs simultaneously. Responses stream back in parallel and render in a shared conversation view where each model can reference what the others said.
Voice Room uses speech-to-text to capture your input, sends it to all three models, and streams back audio responses. It feels like a conference call where you’re talking to three different experts at once.
The safety layer runs in parallel with responses - checking for prompt injection attempts, validating outputs, and enforcing rate limits to prevent abuse.
The Development Journey: AI-Assisted Building
I built this over a few weekends using Claude Code with Opus 4.5. My workflow was deliberately AI-driven with human oversight:
- Ideate a feature with ChatGPT
- Write high-level requirements, asking ChatGPT to generate a clear PRD
- Feed the PRD to Claude Code with Opus 4.5
- Review, test, iterate
Results: Roughly 95% of specified features worked on first generation. UI styling often exceeded my expectations.
But here’s what “95%” hides:
- Edge cases weren’t in my PRD, so they weren’t in the code
- Security considerations required explicit specification
- Architecture was sound only because I designed it - the AI executed, it didn’t architect
The real shift: The bottleneck has moved from “writing syntax” to “specifying requirements completely." If you can write a comprehensive PRD that anticipates edge cases and scale, you can ship remarkably fast. But that “if” is doing heavy lifting.
What still required human judgment:
- Deployment configuration and CI/CD pipelines
- Cost monitoring and optimization
- Security architecture design
These aren’t minor details - they’re the difference between a demo and a production app. AI accelerated the 80%, but the final 20% still required expertise.
Product Insights
Disagreement as a Feature
Early user feedback revealed something surprising: the value wasn’t just getting multiple perspectives - it was seeing where and why models diverged.
For simple questions (“What’s the capital of France?"), three identical answers just add noise. But for ambiguous decisions, watching models emphasize different tradeoffs exposes the assumptions hidden in each response.
The caveat: This only works when users can evaluate the disagreement. For highly technical questions outside their expertise, three conflicting answers create decision paralysis, not clarity.
Key learning: Multi-model debates expose tradeoffs and assumptions - they’re a “second opinion generator,” not a replacement for expertise.
The Economics Tension: Pricing Predictability
Building this surfaced a core pricing tradeoff in AI products: pay-per-token is fair, but it’s not predictable.
I kept second-guessing product decisions: “Should I enable web search?” “What if someone spams it and my costs explode?”
With a 10-messages/day per user limit, roughly 500 tokens per request/response, and three models queried per message, I estimated about $0.01 to $0.03 per user query (depending on the model mix). At max usage (around 300 queries/month), that’s $3 to $9 per user per month in API costs alone.
The catch: Token usage is hard to police. One detailed question that triggers a long answer can easily push costs to 3x the baseline.
That experience convinced me a hybrid pricing model is the right default for most AI products:
- A base tier with included usage (a predictable floor)
- Usage-based pricing after the included amount (fair scaling)
- User-set spending caps (reduces “bill shock” anxiety)
Stripe’s write-up on AI pricing strategies articulates this hybrid approach well.
Competitive Landscape and Future Directions
Most multi-model tools today are either simple API aggregators (pick a model, get a response) or comparison tools (run the same prompt through multiple models and see results side-by-side).
AI Trio is different because the models are in conversation with each other. They see what the others said. They can disagree, build on each other’s points, or challenge assumptions. That changes the interaction from “get multiple answers” to “watch a debate unfold.”
Where could this go? A few directions seem promising:
- Specialized model panels: Instead of three general-purpose models, assemble panels of domain-specific models (a legal expert, a financial analyst, a technical architect) for complex decisions
- Structured disagreement: Automatically surface where models diverge and why, rather than making users parse through three full responses
- Consensus detection: Flag when all models agree (high confidence) vs. when they diverge (worth investigating further)
The underlying bet is that multi-model interfaces become more valuable as models get more capable but remain differently biased. More powerful models with different blind spots means more value in seeing them debate.
The Takeaway
The barrier to validating product ideas has collapsed. PMs who can write clear requirements can now build functional prototypes in days, not weeks.
But the prototype is just the beginning. AI ships exactly what you specify and quietly skips what you don’t. Productization still depends on human judgment around security, scale, reliability, and UX.
The tools have changed. The hard problems haven’t - they’ve just shifted from “can we build it?” to “can we trust it enough to ship?”
Check out AI Trio: try it here
(Personal side project built for learning purposes)