What We Learned Shipping AI Features to 2 Million Learners

When we launched AI recommendations, the biggest lift was not the model — it was defining what "relevant" meant across 500+ courses and wildly different learner goals.

Adaptive quizzes needed guardrails: an over-eager difficulty adjustment felt punishing rather than helpful. We settled on gentle, confidence-based pacing instead.

The single highest-impact change was surfacing "why this was recommended" — transparency increased click-through more than any ranking tweak.