Large enterprise sales organizations may have hundreds or thousands of representatives to onboard, train, and continuously evaluate. At that scale, sales managers can't provide personalized feedback to every rep, while reps have limited opportunities to practice realistic sales conversations on their own. I built from scratch an AI-powered sales roleplay platform that lets reps practice cold calls and other sales scenarios with voice-based AI personas, then receive automated, personalized feedback on their performance. This product was selected by Singapore Business Review as a 2026 Technology Excellence Award winner for AI Training & Development.
Challenges and Solutions
The platform was built around a course-based training workflow. Sales representatives could enroll in and complete courses, with each course composed of multiple roleplay scenarios configured by an administrator. In each scenario, reps conducted a real-time voice conversation with an AI persona designed to simulate a realistic customer. Administrators could configure the persona's identity, attitude, agreeableness, and contextual information, allowing the same system to simulate a wide range of customer types and sales situations. After each call, reps were evaluated against sales methodologies such as SPIN and MEDDPIC, as well as custom evaluation frameworks defined by administrators. This was entirely a 0-to-1 product!
A particularly challenging part of the product was the evaluation system: translating subjective sales frameworks into consistent, useful feedback. For established frameworks such as SPIN and MEDDPIC, the challenge was determining how to quantify performance while still providing meaningful qualitative feedback. For custom frameworks, the challenge was making evaluations both deterministic and simple for administrators to configure. To address this, I developed a separate agentic workflow that helped generate and structure custom evaluation frameworks, turning high-level criteria into more explicit, machine-evaluable rubrics.
I also built a benchmarking platform to systematically evaluate and improve the evaluation pipeline. Early versions relied on broad evaluation prompts that produced inconsistent quantitative scores and highly variable qualitative feedback. The benchmarking system allowed me to run evaluation experiments across different prompts and configurations, measure the spread and consistency of quantitative scores, and use an LLM-as-a-judge framework to compare the quality and sentiment of qualitative feedback. Through these experiments, I moved toward scoped, explicit rubrics with clear instructions for what the evaluator should look for, while keeping the evaluation architecture modular enough to support arbitrary custom frameworks. The resulting evaluations were substantially more consistent and produced actionable feedback with citations back to specific portions of the call transcript.
I applied a similar experimental approach to the voice AI itself. Early prompting was difficult to tune because a customer's attitude and behavior could change throughout a conversation. I found that representing the persona with a stateful set of parameters that could be updated during the call provided a more effective way to model changes in mood and attitude. Giving each persona a well-defined identity, combined with fine-grained controls for administrators, was critical to producing realistic and controllable conversations. The voice layer was initially implemented using Vapi and ElevenLabs, with the longer-term architecture designed around migrating toward a self-hosted voice system.
Another area of work focused on the cost and quality tradeoffs of LLM-based evaluation. I built a comprehensive testing suite that ran a fixed golden dataset against different prompts, temperatures, and models, comparing evaluation cost, scores, and the quality and consistency of transcript citations. This provided a systematic way to benchmark changes to the evaluation pipeline.
In addition to the roleplay and evaluation platform, I also worked on the architecture of a RAG-based sales research platform, improving retrieval quality through context-aware chunking, cross-encoder reranking, and OCR preprocessing for unstructured documents.
React · Next.js · TypeScript · Go · Vapi · ElevenLabs · PostgreSQL · WebSockets · RAG · LLM Evaluation · Agentic Workflows · Prompt Benchmarking · LLM-as-a-Judge