When we first conceived of the AI Assistant, our vision was simple: build the best AI Assistant for exam prep and lifelong learning. Not the best LLM. Not the best AI agent platform. Not a chatbot for your website or a tool to generate content. We wanted to focus on learner intelligence, understanding how a specific learner learns, where they got stuck, and how to help them through it.
So in April 2025, we launched the AI Assistant beta in limited availability to a select group of customers, a built-in tool that lets learners ask questions about course content anytime, in plain language, without leaving the platform. We wanted to get this right. Starting earlier this year, we’ve opened it up to all customers and learners. A year and 70,000 conversations later, here's what we're seeing, and what it means for your program.
1. AI Assistant drives deeper course engagement
To ensure an apples-to-apples comparison, we examined a specific course in the financial services industry. The engagement gap between AI and non-AI users was hard to ignore.
AI Assistant users progressed further through the course, completed more practice exams, and spent more time on the platform. That’s not a marginal difference. Learners who had access to the AI Assistant were more invested in finishing what they started.
2. Learner satisfaction is meaningfully higher with the AI Assistant
When we compared NPS scores between AI and non-AI users on the same course, the gap was clear.
3. Learners show up more prepared.
When we looked at assessment outcomes, AI Assistant users scored higher—modestly, but consistently.
While the score lift is minimal, it aligns with the intention of the tool. The AI Assistant is a supportive guide and not a shortcut. Learners are arriving better prepared, more confident, and less likely to second-guess themselves mid-exam.
4. Every conversation is a signal about your content
We analyzed the thousands of learner messages to understand what people actually use the AI Assistant for. The answer was clear: 87.5% of intentional questions were learners asking to have a concept explained. Not navigating the platform. Not building a study plan. Just trying to understand something they were stuck on.
Questions like:
- "What is a Keogh Plan?"
- "Explain ex-dividend date"
- "What's the difference between a mutual fund and an ETF?"
- "Can you summarize what I just read?"
The remaining 12.5% were learners asking the AI to generate practice questions (4.3%), summarize a lesson (2.7%), or help them navigate the course and track their progress (10%).
Here's the angle that often surprises people: learners struggling the most—those scoring in the 65–75% range—have 2× more AI Assistant conversations than the highest performers.
Every one of those conversations is a learner flagging a gap. Maybe the explanation wasn't clear enough, or a concept was introduced but never properly reinforced. When hundreds of learners ask variations of the same question, that's no longer an individual struggle. It's a signal your content hasn't answered that concept well enough yet.
Before the AI Assistant, those signals were lost to Google or ChatGPT. You'd see them eventually reflected in aggregate test scores, but by then, they were a lagging indicator at best. Now every conversation is a data point you own, and the most valuable thing the AI Assistant does might not be answering questions, but surfacing the questions your content hasn't answered yet.
For exam prep and training program providers, that means tightening the focus on the exact concepts where learners are losing points. For continuing education providers, it means smarter, faster content updates without waiting for the next cohort to tell you something's broken.
Schedule a demo to learn more about BenchPrep's AI Assistant →
