Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Proctor Quality and Learning Manager, Scaling Operations

$142.8k - $193.2k

Duolingo

Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in!

At Duolingo, you’ll join a team that cares about educating our users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems. You’ll have limitless learning opportunities and daily collaborations with world-class minds — while doing work that’s both meaningful and fun.

Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world. 

Read our blog to learn more.

About the role...

The Duolingo English Test (DET) is a high-stakes assessment used by institutions around the world. Its integrity depends on proctors making accurate, consistent, and defensible decisions — even when sessions present ambiguous evidence or new forms of cheating. As the Proctor Quality and Learning Manager (L3) in Scaling Operations, you will own the end-to-end quality standard and learning system that keep our global proctor network aligned and effective as security threats, technology, and workflows evolve. You’ll define what “correct” looks like, run our calibration and adjudication program, and build ongoing training that measurably improves decision quality. You will partner with Psychometrics, Security, Operations, Product, and Engineering to ensure our human and AI-assisted workflows meet rigorous accuracy standards before and after deployment. This is an individual-contributor role within DET Proctoring Operations, with functional leadership over Training Specialists and adjudicators executing the program.

You will...

  • Own the DET proctoring quality standard — maintain rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases.
  • Lead calibration and adjudication — operate recurring Proctor Certification Test cycles, resolve disputed decisions, identify patterns of disagreement, and ensure outcomes are consistent and auditable.
  • Build and run an ongoing learning program — publish a forward-looking training calendar, deliver monthly interventions, and launch targeted training in response to quality signals, threats, or workflow changes.
  • Measure effectiveness — use calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance.
  • Guide specialist execution — set expectations for Training Specialists and adjudicators, assign deliverables, review outputs, and operate onboarding and certification that ramp new specialists quickly.
  • Maintain durable sources of truth — keep calibration, adjudication, and training runbooks current so critical processes are documented and auditable.
  • Evolve quality alongside proctoring — translate new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering.
  • Represent the proctoring craft — bring frontline proctor evidence into product experiments and workflow decisions; assess whether launched changes preserve decision quality.
  • Connect human and technology-assisted quality — establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment.

✅  You have...

  • 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs.
  • Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials).
  • Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance.
  • Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions.
  • Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls.
  • Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes.
  • Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability.
  • Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes.
  • Adaptability — comfort revising standards and programs as risks, products, and technology change.

⭐  Exceptional candidates will have...

  • Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments.
  • Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations.
  • Familiarity with assessment concepts such as inter-rater agreement, reliability, standardization, and performance monitoring.
  • Experience designing quality systems that span human judgment and technology-assisted workflows.
  • Familiarity with learning-management systems, assessment platforms, quality-audit tools, or operational analytics.

The offered salary is dependent upon several factors, including work experience, skills, and internal peer comparisons. The posted range is subject to change in the future. For this role, base salary is supplemented by equity compensation. We encourage you to talk with your recruiter for more information related to compensation for this role!

Salary Range:

$142,800—$193,200 USD

Benefits: Take a peek at how we care for our employees' holistic well-being with our benefits here .

Job Alerts: Sign up for job alerts here .

Accommodations: We will do everything we can within reason to make sure that your interview takes place in an environment that fairly and accurately assesses your skills. If you need assistance or accommodation, please contact View email address on careers.duolingo.com .

Equal Employment Opportunity: Duolingo is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

Fraud Warning: Unfortunately, there is a rise in scammers pretending to be real Duolingo employees. Duolingo and our employees will never ask for your Social Security number, bank details, or passport info, and we’ll never ask you to deposit a check, purchase equipment, or exchange money during the interview process. Real Duolingo employees always use an email that ends in @duolingo.com or @recruiting.duolingo.com. Stay alert and double-check these details before sharing any information.

By applying for this position your data will be processed as per the Duolingo Applicant Privacy Notice .

Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Proctor Quality and Learning Manager, Scaling Operations in Pittsburgh, PA vacancy
  • $200k - $230k

     ...Director of Operations – Commercial Mechanical Construction Location...  ...Operations to lead its project management organization and oversee the successful delivery of large-scale commercial HVAC and plumbing...  ..., resource allocation, quality, and successful project delivery... 
    Quality
    Full time
    Contract work
    For contractors
    For subcontractor
    Monday to Friday

    Stealth

    Pittsburgh, PA
    1 day ago
  •  ...this role, you will help make history by enabling the Legal Department to deliver high-quality legal services efficiently, consistently, and cost-effectively. As Manager, Legal Operations, you will serve as a central operational partner to the Chief Legal Officer for... 
    Quality
    Full time
    Contract work
    Work at office

    Eos Energy Enterprises, Inc.

    Pittsburgh, PA
    2 days ago
  • $120.9k - $181.3k

     ...apply your existing skills and learn new ones in an environment...  ...marketing, sales, supply chain, operations, HR, finance, and more across...  ...special for you.The Regional Manager is a senior-level operational...  ...objectives across safety, quality, delivery, cost & financial performance... 
    Quality
    Full time
    Local area

    IDEX

    Pittsburgh, PA
    3 days ago
  •  ...Details: $93,937 - $126,815 AnnuallyAs Manager, Human Resources (HR) Operations, you will lead the delivery of HR...  ...to improve efficiency, service quality, compliance, and employee experience...  ...has the opportunity to thrive. ​ Learn more about Working at Wesco here and... 
    Quality
    Immediate start
    Worldwide
    Shift work

    Wesco International

    Pittsburgh, PA
    37 minutes ago
  • $136k - $176k

     ...developing humanoid robots designed to operate in real-world environments....  ..., and beyond. As we scale our manufacturing and deployments...  ...behind. The Senior Test Operations Manager role is a critical role in...  ...work will directly influence the quality and reliability of Digit in... 
    Quality
    Full time
    Temporary work
    Work at office
    Immediate start
    Relocation package
    Flexible hours

    Energize Group

    Pittsburgh, PA
    8 days ago
  • $180k - $258k

     ...and supports our communities. With qualities like those, it's no wonder we're consistently...  ...is currently seeking an Associate Managing Director, Global AI Operations Lead to join our Global Digital...  ..., production-grade operations at scale, covering AI models, generative AI... 
    Quality
    H1b
    Local area

    KPMG

    Pittsburgh, PA
    1 day ago
  • $110k

     ...Field Operations Manager II Summary With a focus on the Truck Maintenance facilities and processes...  ...(internal and external) Aptitude to learn a new business and systems Receptive to...  ...strategic labor, financial, operating, quality, safety and asset processes as well as... 
    Quality
    Full time
    Local area

    Ryder

    Pittsburgh, PA
    2 days ago
  •  ...roofing company is hiring an Operations Manager to run its production...  ...a production operation and scale it. Key Responsibilities...  ...efficiently while maintaining quality and profitability Skills,...  ...platform, with the ability to learn systems quickly Strong organizational... 
    Quality
    Work at office

    Hire Right Recruiting

    Pittsburgh, PA
    28 days ago
  • $50k

     ...Description Job Description Remote Scanning Operations Manager The Manager of Remote Scanning...  ...ramp, scheduling and coverage, scan quality and throughput, OEM compliance, and the...  ...diagnostic/workflow platforms. Experience scaling a technical team through a phased... 
    Quality
    Work at office
    Remote work
    Home office
    Day shift

    Clear Drive Solutions

    Homestead, PA
    18 days ago
  •  ...-powered prescription operating system. RxOS connects...  ...exceptional momentum:Built for scale: Over the past decade,...  ...Center Operations Manager, you won't just...  ...: ensure timely, high-quality resolution of customer...  ...We are a relentlessly learning, constantly curious, and... 
    Quality
    Full time
    Work at office
    Shift work

    Blink Health

    Pittsburgh, PA
    24 minutes ago
  • $26 per hour

     ...our team as an Assistant Store Manager, you’ll take on key store...  ...managing schedules, and developing operational action plans while...  ...maximize sales • Ensures the quality and freshness of products for...  ...by state Click here to learn more about the benefits ALDI... 
    Quality
    Hourly pay
    Full time
    Temporary work
    Part time
    Local area

    Aldi Inc.

    Carnegie, PA
    1 day ago
  • $26 per hour

     ...our team as an Assistant Store Manager, you’ll take on key store...  ...managing schedules, and developing operational action plans while...  ...maximize sales • Ensures the quality and freshness of products for...  ...by state Click here to learn more about the benefits ALDI... 
    Quality
    Hourly pay
    Full time
    Temporary work
    Part time
    Local area

    Aldi Inc.

    Pittsburgh, PA
    1 day ago
  •  ...and supports our communities. With qualities like those, it's no wonder we're consistently...  ...is currently seeking a Contract Manager to join our Strategic Deal...  ...contract review and deal enablement operations by designing and scaling standardized contracting processes,... 
    Quality
    Contract work
    H1b
    Work at office
    Local area

    KPMG

    Pittsburgh, PA
    3 days ago
  •  ...to create value at speed and scale. We are a talent- and innovation...  ...momentum and a clear path to Managing Director for high performers....  ...the future-state human + AI operating model, and ensuring every...  ...builds to a consistent, high-quality blueprint. This role sits at... 
    Quality
    Live in
    Work at office
    Local area
    Immediate start

    Accenture

    Pittsburgh, PA
    4 days ago
  •  ...capabilities spanning strategy, process, operating model, change management, talent, data, technology and AI. We...  ...setting the methodology, governing quality, and making sure every engagement...  ...of building, evaluating, and scaling AI-enabled solutions into client work... 
    Quality
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Pittsburgh, PA
    37 minutes ago
  • $128k - $252.5k

     ...passionate program / project manager with a focus on...  ...and streamline business operations in all areas of project...  ...driving execution for large-scale client programs. Manage...  ...to detail and quality of work productAbility...  ...broad range of benefits. Learn more about what working... 
    Quality
    Local area
    Remote work

    Deloitte

    Pittsburgh, PA
    2 days ago
  • $33 per hour

     ...Operations Supervisor-Patient Shipment Setup Location: Pittsburgh, Pennsylvania (Must live...  ...maintaining the highest standards of quality and compliance. Location : Pittsburgh...  ...develop SOPs and ensure compliance. Manage staff scheduling and maintain data... 
    Quality
    Full time
    Work experience placement
    Work at office
    Remote work
    Monday to Friday
    Flexible hours
    Day shift

    Kelly

    Pittsburgh, PA
    9 days ago
  •  ...Operations SupervisorOPEN AVAILABILITY REQUIRED This role you will be required to learn Housekeeping, Front Desk, as well as the Bistro. As an...  ...years to Assistant General Manager roles.ResponsibilitiesInspect...  ...they maintain the high quality and standards of a Marriott... 
    Quality
    Full time
    Temporary work
    Shift work
    Afternoon shift

    Concord Hospitality

    Pittsburgh, PA
    8 hours ago
  •  ...Pay Grade:** 16 CulinArt Group operates over 250 onsite corporate,...  ...teams in a union environment, manage financial performance, and build...  ...initiatives. + Ensure exceptional food quality, service standards,...  ...Compass Group USA_** Click here to Learn More about the Compass Story!... 
    Quality
    Contract work
    Work at office
    Local area
    Remote work
    Flexible hours

    Compass Group, North America

    Pittsburgh, PA
    3 days ago
  •  ...Operations SupervisorAt PNC, our people are our greatest differentiator...  ...their recruiter and hiring manager to understand workplace expectations...  ...identify exceptions, monitor quality and ensure compliance....  ...level; and years of service.To learn more about these and other... 
    Quality
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office

    PNC

    Pittsburgh, PA
    1 day ago
  •  ...Director, Hotel Operations This role reports to the Chief Executive Officer and is full...  ...Historic Downtown North Star Lodging Management is a hospitality company built on a...  ...readiness reviews to ensure execution of quality, service, and safety standards Leading... 
    Quality
    Full time
    Remote work
    Home office
    Night shift
    Afternoon shift
    Early shift

    Marriott International Inc

    Pittsburgh, PA
    1 day ago
  • $25 - $31 per hour

     ...Operations Manager Here's your chance to get in on the ground floor and help to shape and create this new business! PIRTEK, the nation'...  ...the Operations Manager ensures customers receive the highest quality of service to get their equipment back up and running. Lead... 
    Quality
    Hourly pay
    Local area

    PIRTEK

    Pittsburgh, PA
    1 day ago
  •  ...role you will be required to learn Housekeeping, Front Desk, as well as the Bistro. As a Operations Supervisor at Concord Hospitality...  ...years to Assistant General Manager roles. Responsibilities...  ...ensure they maintain the high quality and standards of a Marriott hotel... 
    Quality
    Full time
    Temporary work
    Shift work
    Afternoon shift

    Concord Hospitality

    Pittsburgh, PA
    8 hours ago
  •  ...000 Omega Drive Working Environment: On-Site/Hybrid Operations - Management NPK International Inc. (NYSE: NPKI) is the only site access...  ...value through performance, innovation, and service quality, and that commitment starts with YOU! NPK is where you can... 
    Quality
    Full time
    Local area

    NPK International Inc.

    Pittsburgh, PA
    8 hours ago
  • $19 - $29 per hour

     ...purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger...  ...and one community at a time. Position Summary The Operations Manager is a member of the Store Management team, and as such, fulfills... 
    Quality
    Hourly pay
    Full time
    Temporary work
    Local area
    Flexible hours
    Shift work
    Night shift
    Weekend work
    Day shift
    Early shift

    CVS Health

    Pittsburgh, PA
    1 day ago
  • The Director of Operation’s duties, roles and responsibilities are dynamic. The Director of...  ..., financial performance, and achieving quality, specifically for our North American Rail...  .... Other responsibilities include managing daily operations, planning how both material... 
    Quality
    Full time
    Shift work

    LB Foster

    Pittsburgh, PA
    3 days ago
  •  ...Job-ID: 2352350 Category: Operations Management - District Location: Pittsburgh, PA, United States;...  ...and Power BI familiarity) and ability to learn other job-related technical software....  ...maintenance work at assigned sites to ensure the quality and timeliness of the work. Conducts,... 
    Quality
    Full time
    Contract work
    Temporary work
    For contractors
    Work experience placement
    Local area
    Night shift

    Waste Management , Inc.

    Pittsburgh, PA
    4 days ago
  •  ...measurable outcomes at scale. Through our...  ...can change the way it operates. You're comfortable working...  ...current through continuous learning. Apply AX Design...  ...process redesign, or management consulting Minimum 1 year...  ...building agents, for higher-quality deliverables to... 
    Quality
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Pittsburgh, PA
    4 days ago
  •  ...reinvention real at enterprise scale.This is a high-visibility,...  ...momentum and a clear path to Managing Director for high performers....  ...stakeholders. You are comfortable operating in ambiguous environments and...  ...Life Sciences industry.The Learning & Development Partner designs... 
    Live in
    Work at office
    Local area

    Accenture

    Pittsburgh, PA
    4 days ago
  • $163.4k - $322.1k

    Position Summary Enterprise Operations & Risk - Investment Management and Trust Operations Senior Manager...  ..., artificial intelligence, machine learning, data platforms, and digital client...  ...experiences for effects on controls, data quality, operational resilience, and... 
    Quality
    Work at office
    Night shift

    Deloitte

    Pittsburgh, PA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Proctor Quality and Learning Manager, Scaling Operations. Be the first to apply!