AI tutoring results are strong, monetization is not

A Dartmouth study posted as a PDF showed a new AI tutor achieving a 0.71 to 1.30 standard deviation improvement in a statistics course, which is a large effect size by educational research standards. The thread was genuinely impressed. One commenter noted that good LLMs are 'a game changer for people motivated to learn.'

But the thread also quickly surfaced the central business problem: educational use cases don't make money. The comment got upvoted without much pushback, which is telling. The effect sizes are there. The willingness to pay, at least through traditional EdTech channels, is not.

The study also raised questions about generalizability. Statistics has objective grading, which makes it an easy testbed. Commenters asked whether the same results would hold for subjects requiring subjective evaluation. That's an open question and a real limitation on how broadly you can extrapolate.