Proctor Quality and Learning Manager, Scaling Operations
$142.8k - $193.2kDuolingo
Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in!
At Duolingo, you’ll join a team that cares about educating our users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems. You’ll have limitless learning opportunities and daily collaborations with world-class minds — while doing work that’s both meaningful and fun.
Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world.
Read our blog to learn more.
About the role...
The Duolingo English Test (DET) is a high-stakes assessment used by institutions around the world. Its integrity depends on proctors making accurate, consistent, and defensible decisions — even when sessions present ambiguous evidence or new forms of cheating. As the Proctor Quality and Learning Manager (L3) in Scaling Operations, you will own the end-to-end quality standard and learning system that keep our global proctor network aligned and effective as security threats, technology, and workflows evolve. You’ll define what “correct” looks like, run our calibration and adjudication program, and build ongoing training that measurably improves decision quality. You will partner with Psychometrics, Security, Operations, Product, and Engineering to ensure our human and AI-assisted workflows meet rigorous accuracy standards before and after deployment. This is an individual-contributor role within DET Proctoring Operations, with functional leadership over Training Specialists and adjudicators executing the program.
You will...
- Own the DET proctoring quality standard — maintain rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases.
- Lead calibration and adjudication — operate recurring Proctor Certification Test cycles, resolve disputed decisions, identify patterns of disagreement, and ensure outcomes are consistent and auditable.
- Build and run an ongoing learning program — publish a forward-looking training calendar, deliver monthly interventions, and launch targeted training in response to quality signals, threats, or workflow changes.
- Measure effectiveness — use calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance.
- Guide specialist execution — set expectations for Training Specialists and adjudicators, assign deliverables, review outputs, and operate onboarding and certification that ramp new specialists quickly.
- Maintain durable sources of truth — keep calibration, adjudication, and training runbooks current so critical processes are documented and auditable.
- Evolve quality alongside proctoring — translate new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering.
- Represent the proctoring craft — bring frontline proctor evidence into product experiments and workflow decisions; assess whether launched changes preserve decision quality.
- Connect human and technology-assisted quality — establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment.
✅ You have...
- 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs.
- Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials).
- Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance.
- Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions.
- Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls.
- Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes.
- Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability.
- Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes.
- Adaptability — comfort revising standards and programs as risks, products, and technology change.
⭐ Exceptional candidates will have...
- Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments.
- Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations.
- Familiarity with assessment concepts such as inter-rater agreement, reliability, standardization, and performance monitoring.
- Experience designing quality systems that span human judgment and technology-assisted workflows.
- Familiarity with learning-management systems, assessment platforms, quality-audit tools, or operational analytics.
The offered salary is dependent upon several factors, including work experience, skills, and internal peer comparisons. The posted range is subject to change in the future. For this role, base salary is supplemented by equity compensation. We encourage you to talk with your recruiter for more information related to compensation for this role!
Salary Range:
$142,800—$193,200 USD
Benefits: Take a peek at how we care for our employees' holistic well-being with our benefits here .
Job Alerts: Sign up for job alerts here . Accommodations: We will do everything we can within reason to make sure that your interview takes place in an environment that fairly and accurately assesses your skills. If you need assistance or accommodation, please contact View email address on careers.duolingo.com . Equal Employment Opportunity: Duolingo is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. Fraud Warning: Unfortunately, there is a rise in scammers pretending to be real Duolingo employees. Duolingo and our employees will never ask for your Social Security number, bank details, or passport info, and we’ll never ask you to deposit a check, purchase equipment, or exchange money during the interview process. Real Duolingo employees always use an email that ends in @duolingo.com or @recruiting.duolingo.com. Stay alert and double-check these details before sharing any information. By applying for this position your data will be processed as per the Duolingo Applicant Privacy Notice .$200k - $230k
...Director of Operations – Commercial Mechanical Construction Location... ...Operations to lead its project management organization and oversee the successful delivery of large-scale commercial HVAC and plumbing... ..., resource allocation, quality, and successful project delivery...QualityFull timeContract workFor contractorsFor subcontractorMonday to Friday- ...this role, you will help make history by enabling the Legal Department to deliver high-quality legal services efficiently, consistently, and cost-effectively. As Manager, Legal Operations, you will serve as a central operational partner to the Chief Legal Officer for...QualityFull timeContract workWork at office
$120.9k - $181.3k
...apply your existing skills and learn new ones in an environment... ...marketing, sales, supply chain, operations, HR, finance, and more across... ...special for you.The Regional Manager is a senior-level operational... ...objectives across safety, quality, delivery, cost & financial performance...QualityFull timeLocal area- ...Details: $93,937 - $126,815 AnnuallyAs Manager, Human Resources (HR) Operations, you will lead the delivery of HR... ...to improve efficiency, service quality, compliance, and employee experience... ...has the opportunity to thrive. Learn more about Working at Wesco here and...QualityImmediate startWorldwideShift work
$136k - $176k
...developing humanoid robots designed to operate in real-world environments.... ..., and beyond. As we scale our manufacturing and deployments... ...behind. The Senior Test Operations Manager role is a critical role in... ...work will directly influence the quality and reliability of Digit in...QualityFull timeTemporary workWork at officeImmediate startRelocation packageFlexible hours$180k - $258k
...and supports our communities. With qualities like those, it's no wonder we're consistently... ...is currently seeking an Associate Managing Director, Global AI Operations Lead to join our Global Digital... ..., production-grade operations at scale, covering AI models, generative AI...QualityH1bLocal area$110k
...Field Operations Manager II Summary With a focus on the Truck Maintenance facilities and processes... ...(internal and external) Aptitude to learn a new business and systems Receptive to... ...strategic labor, financial, operating, quality, safety and asset processes as well as...QualityFull timeLocal area- ...roofing company is hiring an Operations Manager to run its production... ...a production operation and scale it. Key Responsibilities... ...efficiently while maintaining quality and profitability Skills,... ...platform, with the ability to learn systems quickly Strong organizational...QualityWork at office
$50k
...Description Job Description Remote Scanning Operations Manager The Manager of Remote Scanning... ...ramp, scheduling and coverage, scan quality and throughput, OEM compliance, and the... ...diagnostic/workflow platforms. Experience scaling a technical team through a phased...QualityWork at officeRemote workHome officeDay shift- ...-powered prescription operating system. RxOS connects... ...exceptional momentum:Built for scale: Over the past decade,... ...Center Operations Manager, you won't just... ...: ensure timely, high-quality resolution of customer... ...We are a relentlessly learning, constantly curious, and...QualityFull timeWork at officeShift work
$26 per hour
...our team as an Assistant Store Manager, you’ll take on key store... ...managing schedules, and developing operational action plans while... ...maximize sales • Ensures the quality and freshness of products for... ...by state Click here to learn more about the benefits ALDI...QualityHourly payFull timeTemporary workPart timeLocal area$26 per hour
...our team as an Assistant Store Manager, you’ll take on key store... ...managing schedules, and developing operational action plans while... ...maximize sales • Ensures the quality and freshness of products for... ...by state Click here to learn more about the benefits ALDI...QualityHourly payFull timeTemporary workPart timeLocal area- ...and supports our communities. With qualities like those, it's no wonder we're consistently... ...is currently seeking a Contract Manager to join our Strategic Deal... ...contract review and deal enablement operations by designing and scaling standardized contracting processes,...QualityContract workH1bWork at officeLocal area
- ...to create value at speed and scale. We are a talent- and innovation... ...momentum and a clear path to Managing Director for high performers.... ...the future-state human + AI operating model, and ensuring every... ...builds to a consistent, high-quality blueprint. This role sits at...QualityLive inWork at officeLocal areaImmediate start
- ...capabilities spanning strategy, process, operating model, change management, talent, data, technology and AI. We... ...setting the methodology, governing quality, and making sure every engagement... ...of building, evaluating, and scaling AI-enabled solutions into client work...QualityWork experience placementLive inWork at officeLocal area
$128k - $252.5k
...passionate program / project manager with a focus on... ...and streamline business operations in all areas of project... ...driving execution for large-scale client programs. Manage... ...to detail and quality of work productAbility... ...broad range of benefits. Learn more about what working...QualityLocal areaRemote work$33 per hour
...Operations Supervisor-Patient Shipment Setup Location: Pittsburgh, Pennsylvania (Must live... ...maintaining the highest standards of quality and compliance. Location : Pittsburgh... ...develop SOPs and ensure compliance. Manage staff scheduling and maintain data...QualityFull timeWork experience placementWork at officeRemote workMonday to FridayFlexible hoursDay shift- ...Operations SupervisorOPEN AVAILABILITY REQUIRED This role you will be required to learn Housekeeping, Front Desk, as well as the Bistro. As an... ...years to Assistant General Manager roles.ResponsibilitiesInspect... ...they maintain the high quality and standards of a Marriott...QualityFull timeTemporary workShift workAfternoon shift
- ...Pay Grade:** 16 CulinArt Group operates over 250 onsite corporate,... ...teams in a union environment, manage financial performance, and build... ...initiatives. + Ensure exceptional food quality, service standards,... ...Compass Group USA_** Click here to Learn More about the Compass Story!...QualityContract workWork at officeLocal areaRemote workFlexible hours
- ...Operations SupervisorAt PNC, our people are our greatest differentiator... ...their recruiter and hiring manager to understand workplace expectations... ...identify exceptions, monitor quality and ensure compliance.... ...level; and years of service.To learn more about these and other...QualityFull timeTemporary workPart timeWork experience placementWork at office
- ...Director, Hotel Operations This role reports to the Chief Executive Officer and is full... ...Historic Downtown North Star Lodging Management is a hospitality company built on a... ...readiness reviews to ensure execution of quality, service, and safety standards Leading...QualityFull timeRemote workHome officeNight shiftAfternoon shiftEarly shift
$25 - $31 per hour
...Operations Manager Here's your chance to get in on the ground floor and help to shape and create this new business! PIRTEK, the nation'... ...the Operations Manager ensures customers receive the highest quality of service to get their equipment back up and running. Lead...QualityHourly payLocal area- ...role you will be required to learn Housekeeping, Front Desk, as well as the Bistro. As a Operations Supervisor at Concord Hospitality... ...years to Assistant General Manager roles. Responsibilities... ...ensure they maintain the high quality and standards of a Marriott hotel...QualityFull timeTemporary workShift workAfternoon shift
- ...000 Omega Drive Working Environment: On-Site/Hybrid Operations - Management NPK International Inc. (NYSE: NPKI) is the only site access... ...value through performance, innovation, and service quality, and that commitment starts with YOU! NPK is where you can...QualityFull timeLocal area
$19 - $29 per hour
...purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger... ...and one community at a time. Position Summary The Operations Manager is a member of the Store Management team, and as such, fulfills...QualityHourly payFull timeTemporary workLocal areaFlexible hoursShift workNight shiftWeekend workDay shiftEarly shift- The Director of Operation’s duties, roles and responsibilities are dynamic. The Director of... ..., financial performance, and achieving quality, specifically for our North American Rail... .... Other responsibilities include managing daily operations, planning how both material...QualityFull timeShift work
- ...Job-ID: 2352350 Category: Operations Management - District Location: Pittsburgh, PA, United States;... ...and Power BI familiarity) and ability to learn other job-related technical software.... ...maintenance work at assigned sites to ensure the quality and timeliness of the work. Conducts,...QualityFull timeContract workTemporary workFor contractorsWork experience placementLocal areaNight shift
- ...measurable outcomes at scale. Through our... ...can change the way it operates. You're comfortable working... ...current through continuous learning. Apply AX Design... ...process redesign, or management consulting Minimum 1 year... ...building agents, for higher-quality deliverables to...QualityWork experience placementLive inWork at officeLocal area
- ...reinvention real at enterprise scale.This is a high-visibility,... ...momentum and a clear path to Managing Director for high performers.... ...stakeholders. You are comfortable operating in ambiguous environments and... ...Life Sciences industry.The Learning & Development Partner designs...Live inWork at officeLocal area
$163.4k - $322.1k
Position Summary Enterprise Operations & Risk - Investment Management and Trust Operations Senior Manager... ..., artificial intelligence, machine learning, data platforms, and digital client... ...experiences for effects on controls, data quality, operational resilience, and...QualityWork at officeNight shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Proctor Quality and Learning Manager, Scaling Operations. Be the first to apply!
- director quality management Pittsburgh, PA
- director of quality improvement Pittsburgh, PA
- quality manager Pittsburgh, PA
- director of quality Pittsburgh, PA
- senior manager quality engineering Pittsburgh, PA
- quality supervisor Pittsburgh, PA
- quality manager healthcare Pittsburgh, PA
- director quality assurance regulatory affairs Pittsburgh, PA
- quality manager aerospace Pittsburgh, PA
- training and quality manager Pittsburgh, PA





