PROBLEM
Sales managers couldn't effectively evaluate calls.
Sales managers were struggling to evaluate if reps followed company-specific sales playbooks. Amotions’ existing AI system gave generic scores (e.g., "87/100") that felt arbitrary and lacked context.
Arbitrary Metrics: A high score didn't guarantee the rep followed the specific "Discovery" or "Closing" steps required by the company.
AI Fatigue: Managers spent more time "auditing" the AI's mistakes (like flagging small talk as "off-track") than actually coaching.
Actionability Gap: Reps received "Low Scores" without clear instructions on what specific behavior to change for the next call.
BREAKTHROUGH INSIGHT
Managers don't want a score; they want a checklist.
Through 7 deep-dive interviews and 4 live call reviews, I discovered that managers were manually "checking off" steps in their heads while listening to the AI.
The Pivot:
I moved away from Granular Segment Scoring (which tried to analyze every second of the call) to a Visual Playbook Comparison. This allowed the AI to map the transcript directly against the company’s unique methodology (e.g., MEDDIC or BANT).
THE SOLUTION
3-month pilot, 4 companies, 73 users
Smart contextual analysis
Instead of a single score, managers see a "Yes/No" status for every required step in their playbook.
Why it works: It matches the mental model of a sales leader.
Transparent AI reasoning
Beside every "No," the AI provides a timestamped explanation (e.g., "Rep mentioned ROI but failed to quantify it with the customer’s specific data").
Why it works: It builds trust by showing the "Work" behind the AI's judgment.
The "Context Nudge"
I designed a low-friction pre-call modal that encouraged reps to add 1-2 goals for the meeting.
Result: Context provision increased from 12% to 41%, leading to significantly more accurate AI evaluations.
IMPACT
3-month pilot, 4 companies, 73 users
70% Faster Reviews
Average review time dropped from 8 mins to 2.5 mins per call.
100% Retention
All 4 pilot companies renewed their contracts based on the new Checklist feature.
94% Approval
17 out of 18 managers reported they would recommend the "Playbook Checklist" over the previous scoring system.
23% Better Adherence
Sales teams showed a measured increase in following the company playbook within 8 weeks.
Before - Hard to assess
After - Easy to assess
LEARNINGS
Key Learnings
Simple Beats Sophisticated
Sales managers need binary answers to move fast, not complex data visualizations.
AI Needs a Human Pilot
By allowing users to correct the AI, we turned a "mistake" into a training opportunity for the model, building long-term user trust.
B2B is About Workflow
Transparent human-AI partnership builds trust better than hiding AI mistakes
Early alignment is critical
The best AI features are the ones that disappear into the user's existing habits rather than forcing new ones.
Reflection
Effective AI product design isn't about showcasing technical capability, it's about solving real human problems with appropriate tools. The most successful features fit seamlessly into existing workflows and gave users control.
What I'd do differently: Involve pilot customers even earlier, potentially saving 4-6 weeks to final solution.
This is just one story. See what else I’ve designed
There’s a lot more where that came from. Browse through my other projects to see how I tackle problems, experiment with ideas, and create meaningful experiences.
