Recommender System for Data Science Learning and Research
This paper presents a sequential recommendation system for data science education that models temporal learning patterns to surface relevant papers, tools, and tutorials based on learner history. The evaluation methodology distinguishes signal from noise in learner behavior using precision and recall metrics on held-out interaction sequences. Published in IJAITL, it reflects the eval-first standards Fact AI Lab applies to recommendation AI: measure what matters, audit what you ship.
Recommendation systems for education face a distinct challenge compared to e-commerce recommendation: the goal is long-term competency development, not short-term engagement. A learner who gets distracted by tangentially interesting content is not well-served, even if they click on it.
Problem Formulation
We model the learner's interaction history as a sequence of resource engagements - papers read, tutorials completed, tools installed - and predict the next resource that would best advance their stated learning objectives. The sequential formulation captures the temporal dependency in learning: what a learner needs next depends on what they have already mastered.
Model
The recommendation model is a transformer-based sequential encoder fine-tuned on learner interaction logs from a data science course platform. Content embeddings are derived from paper abstracts and tutorial metadata using a frozen language model, enabling generalization to new resources not seen during training.
Key design choice: The model explicitly separates engagement signals (clicks, saves) from mastery signals (completion, follow-on activity). Training only on engagement leads to recommending engaging but low-value content; incorporating mastery signals aligns recommendations with actual learning outcomes.
Evaluation
We evaluate offline using leave-one-out evaluation on held-out interaction sequences (Hit Rate@10, NDCG@10) and a manual quality review of top-10 recommendations for a sample of learner profiles. The manual review step is intentional: automated metrics alone do not capture whether recommended content is educationally appropriate.
This combination of automated metrics and manual auditing mirrors the evaluation approach Fact AI Lab uses for production AI systems in regulated settings.