Senior Applied Data Scientist, Learner Modeling

Senior Applied Data Scientist

At Instructure, we believe in the power of people to grow and succeed throughout their lives. Our goal is to amplify that power by creating intuitive products that simplify learning and personal development, facilitate meaningful relationships, and inspire people to go further in their education and careers. We do this by giving smart, creative, passionate people opportunities to create awesome. And that's where you come in:

Our team builds AI-native capabilities, reusable AI systems, and shared infrastructure that power multiple products and workflows across the platform.

We are looking for a Senior Applied Data Scientist to develop and validate knowledge tracing and longitudinal learner models, defining what "mastery" means operationally and ensuring the outputs are trustworthy, calibrated, and fair before they reach learners and educators. This is a measurement role as much as a modeling role, closer to computational psychometrics than to generic data science: a prediction that is accurate on average but miscalibrated, unstable, or unfair is not product-ready, and judging that difference is the core of the job. You will partner with AI platform engineers to productionize training and scoring pipelines and to monitor quality in live environments.

You will work closely with product, engineering, and research partners to turn advanced AI ideas into reliable product capabilities used at scale.

Important note on scope: This role is judged on the validity of what the models claim about a learner (calibration, fairness, and stability over time), not on predictive accuracy alone, and not on BI/reporting or experimentation analytics. We are looking for someone who can define a construct, model it rigorously, and stand behind the result in a live product.

What You'll Do

• Design and build knowledge tracing and longitudinal learner models that support mastery and progression features surfaced to customers
• Define what "mastery" and "progression" mean operationally: constructs, model targets, and evaluation criteria aligned to learning outcomes and product requirements
• Build robust training and scoring approaches for noisy, incomplete, and evolving learner interaction data
• Own model trustworthiness: lead evaluation for validity, calibration, fairness, stability over time, interpretability, and failure modes (not predictive accuracy alone), and set the bar for what is trustworthy enough to ship
• Partner with engineering to productionize learner models into reliable services, including deployment, monitoring, and iteration loops
• Collaborate with product and learning partners to translate learning theory into scalable product systems, and to communicate model behavior, assumptions, and limitations clearly

What You'll Need

• 6+ years of experience in applied machine learning, data science, or applied research, with ownership of models shipped into real products
• Strong depth in at least one of: knowledge tracing, sequence modeling, probabilistic modeling, temporal modeling, Bayesian approaches, or psychometrics / item response theory
• A working understanding of computational psychometrics: the rigorous measurement of a construct, not only the optimization of a predictive metric
• Demonstrated ability to evaluate a model on more than accuracy, including calibration, uncertainty, fairness, robustness, and interpretability, and to make a defensible ship/no-ship call on that basis
• Experience working with longitudinal data and designing models that remain stable, meaningful, and interpretable over time
• Strong Python and ML stack skills, with the ability to implement and iterate on modeling pipelines
• Ability to communicate modeling choices, assumptions, and uncertainty clearly to technical and non-technical stakeholders

It Would Be a Bonus If You Had

• Deep experience with computational psychometrics, psychometrics, educational measurement, or item response theory, with a track record of bringing that rigor into machine-learned models
• Experience with learning science or adaptive learning systems
• Experience building customer-facing learner progress or mastery products
• Experience combining structured knowledge representations (skills, standards, concept graphs) with learner models
• Experience designing experiments or observational validation strategies for learning impact
• Experience partnering with platform teams to run models reliably at scale on AWS

Growth & Impact - In This Role, You'll Be Expected To

In this role, you will define how mastery and progression are modeled, validated, and responsibly surfaced to learners and educators. You will build a core differentiator: scientifically grounded learner intelligence that is calibrated, fair, interpretable, stable over time, and production-ready.

Why Join Us

Join us and help shape the future of education by turning cutting-edge AI into reliable product capabilities.

At Instructure, we're on a mission to help educators and students learn together, anytime, anywhere, and however works best. You'll join our research-driven team tackling education's biggest challenges with cutting-edge technology. Our projects have included making sense of unstructured feedback, applying large language models to save teachers' time and improve student experiences, classifying partner networks for smarter recommendations, and detecting fraud to protect resources for real learners.

We value diversity, creativity, and passion, and invest in our teams through mentorship, hack weeks, internal conferences, and a culture where innovation thrives. Here, you'll have the chance to build the next generation of LMS features that make a real impact on students and teachers, and do it in a collaborative, supportive environment that encourages experimentation and growth.

Get in on all the awesome at Instructure!

We offer competitive, meaningful benefits in every country where we operate. While they vary by location, here's a general idea of what you can expect:

• Competitive compensation, plus all full-time employees participate in our ownership program - because everyone should have a stake in our success.
• Flexible work culture. Our remote, hybrid and in-office collaboration spaces vary by role, team and location.
• Generous time off, including local holidays and our annual "Dim the Lights" period in late December, when teams are encouraged to step back and recharge based on departmental needs.
• Comprehensive wellness programs and mental health support
• Learning and development resources, including professional development tools and tuition reimbursement, to support your growth
• The technology and tools you need to do your best work
• Motivosity employee recognition program
• A culture rooted in inclusivity, support, and meaningful connection

We believe in hiring great people and treating them right. The more diverse we are, the better our ideas and outcomes.

Instructure is an Equal Opportunity Employer. We comply with applicable employment and anti-discrimination laws in every country where we operate.

All employees must pass a background check as part of the hiring process. To help protect our teams and systems, we've implemented identity verification measures. Candidates may be asked to verify their legal name, current physical location, and provide a valid contact number and residential address, in accordance with local data privacy laws.

Any attempt to misrepresent personal or professional information will result in disqualification.