Senior Evaluation Quality & Operations Lead
Harvey
Harvey is seeking a Senior Operator in Evaluation Operations to own the quality bar behind human evaluations as we scale globally. You will set standards for what good looks like, collaborate with ALR, Product, Engineering, and AI Research, and produce actionable analysis to guide product decisions. You will develop guidelines, calibrate a distributed pool of raters, and build repeatable evaluation templates and dashboards to monitor quality across jurisdictions and languages. #J-18808-Ljbffr Harvey
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Senior Evaluation Quality & Operations Lead in San Francisco, CA vacancy
$150.1k - $227k
...your career at the company leading workforce transformation in... ...strategy. The Digital Success and Operations Senior Lead owns the end-to-end... ...platform KPIs (uptime, stream quality, registration throughput),... ...prototypes etc., and the ability to evaluate AI capabilities in vendor...QualitySeniorFull timeContract work$155.4k - $233.2k
...transforming how legal and professional services operate. By combining frontier agentic AI, an... ...getting started. Role Overview We’re looking for a senior operator to own the quality bar behind Harvey’s human evaluations. As we scale globally, the volume of eval work...QualitySeniorContract work- ...products. About the Role In this role, you’ll lead development of the systems we use to evaluate the quality of our AI models and products. You’ll help us... ...accelerate robust iteration Work with a small, senior team of engineers and researchers to combine existing...QualitySeniorFull timeWork at officeRelocation package
OpenAI
San Francisco, CA20 hours ago - ...transforming how legal and professional services operate. Role Overview We’re looking for a... ...‑minded operator to build and scale the evaluation engine behind Harvey’s platform. As we... ...‑attorney pipeline, ensuring evaluation quality meets legal standards Work with...QualitySeniorContract work
- Harvey is seeking a senior operator to own the quality bar behind human evaluations as we scale globally. You will define what good looks like, author guidelines, and maintain data analyses to produce decision-grade outputs for stakeholders. You will work with Evaluation...QualitySenior
- Obsidian seeks experienced AI Safety Practitioners to evaluate safety, quality, and alignment of frontier AI models across grey-area topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured evaluations and...QualitySenior
- Mercor seeks expert Evaluators in User/customer research and feedback synthesis to review AI-generated artifacts (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise to grade outputs. This is a...QualitySeniorRemote jobHourly pay
- Coinbase is seeking a senior operations professional to elevate the FCM operating model for US derivatives businesses. You will lead cross-functional efforts with Product, Engineering,... ...reducing manual work, improving control quality, and ensuring operational resilience...QualitySenior
- ...You will train CLS trainees and assist in developing new test procedures while working with minimal supervision. The role emphasizes quality control, proficiency testing, and collaboration with physicians. A California CLS license and relevant lab experience are required,...QualitySeniorTraineeship
- Obsidian is hiring expert Evaluators in Investment analysis / valuation / credit to review AI-generated work products for accuracy and quality. This remote, hourly position requires deep subject-matter expertise and professional fluency in English to provide structured...QualitySeniorHourly payWork at officeRemote work
- iRhythm Technologies, Inc. is seeking an experienced Clinical Operations Manager to lead and optimize the daily services team's operations,... ...across clinical, operational, and leadership teams to ensure high-quality service. #J-18808-Ljbffr iRhythm Technologies, Inc.QualitySenior
$281k - $356k
...is to ensure that we can continue to evaluate safety and service quality across new cities and countries, each... ...manager with strong quantitative skills to lead a small team and work with partners... ...hybrid role, you will report to a Senior Staff manager. You will: Develop...QualitySeniorFull timeWork experience placementRemote work$204k - $259k
...public roads and tens of billions in simulation across 15+ U.S. states. The Planner Evaluation team works on one of the key challenges in autonomous driving: measuring and improving the quality of the software that drives the car. We are looking for experienced data-minded...QualitySeniorFull timeRemote work- Dormont Manufacturing Co in San Francisco is seeking a Senior Manager for IT Operations. In this role, you'll lead the delivery of exemplary IT support and manage... ...on aligning IT with business goals, ensuring high-quality service delivery, and implementing efficient...QualitySeniorWork at office
- Expedia Group is seeking a Senior Experience Design Operations Manager in San Francisco to lead operational programs that connect design strategy with delivery. You... ...planning, reviews, and coordination, enabling high-quality experiences at scale. The role emphasizes...QualitySenior
- Expedia Group is seeking a Senior Manager, Design Operations to lead operational programs across design teams in Seattle, New York, San Jose, or San Francisco... ...enable designers to move from insight to impact while maintaining quality and velocity. #J-18808-Ljbffr Expedia, Inc.QualitySenior
$132k - $163k
...excellence. Backed by leading investors like Google... ...team includes US-based Senior Specialists who handle... ...day and hit our core operational KPIs. You will contribute... ...strategy for CX: evaluate tools, configure integrations... ...that improve response quality and reduce manual...QualitySeniorFull timeFor contractorsWork at officeLocal areaFlexible hours$127k - $223k
...impact the world in a positive way. To learn more visit: The Evaluation Algorithms team is responsible for building the algorithms &... ...simulator. - Champion engineering excellence, ensuring high-quality, well structured and tested code. - Assist in project roadmap...QualitySeniorFull timeWork at officeWork from homeFlexible hoursWaabi
San Francisco, CAa month ago- Bird in San Francisco is seeking a Senior City Operations Manager to drive operational excellence, market... ...efficiency in local markets. You will lead day-to-day operations, optimize... ...team while upholding Bird’s safety and quality standards. Reporting to the Regional Operations...QualitySeniorLocal area
- ...Specialist III to oversee both forward and reverse logistics operations. This role involves managing resources, developing operational... ...will have 5-8 years of logistics experience, strong leadership qualities, and the ability to drive process improvements. The position requires...QualitySeniorFull time
- Google is seeking a UX Program Manager to lead multi-disciplinary UX programs, coordinating with Engineering, Product Management, and... ...actionable insights to ensure on-time delivery while maintaining a high quality bar. This role emphasizes collaboration across many UX...QualitySenior
- Oakmont Management Group is seeking a Senior Living Assistant Executive Director for a community in San Francisco... ...supports the Executive Director, overseeing daily operations and guiding staff to deliver high-quality care and services to residents. It also offers a pathway...QualitySenior
- ...frontier AI lab researchers to create evaluations, publish benchmarks, and push the... ...together with engineers, scientists, operators, and more from Palantir, Meta, Scale... ...generative AI space, we’re looking for a Senior Growth Operations Lead to architect and scale strategic...SeniorFull timeWork at officeRemote workFlexible hours
$90k - $115k
...seeking a Warehouse Manager for their South San Francisco facility to enhance operational performance in fulfillment operations. This leadership role involves overseeing a large team, ensuring quality standards, and facilitating scaling processes. Responsibilities cover...QualitySenior$215k - $265k
...foundational role on Sierra's Operations team, sitting at the... ...Marketing Operations and Analytics Lead, you'll own two things: the measurement... ...—and aren't—generating high-quality opportunities.Performance... ...standards the team uses to evaluate new programs; translate...QualityFull timeFlexible hours$132.3k - $272.38k
...to learn more.The Strategy & Operations team is a key strategic partner... ...synthesizing relevant data, leading analyses and developing final... ...to ensure correctness and quality.What we’re looking for:Minimum... ...Strong track record of critical evaluation and verification of AI-...QualityWork experience placementWork at officeLocal areaRelocationRelocation packageFlexible hours- ...national news. We’re the leading free news streaming... ...looking for a Growth Operations & Analytics Lead to... ...tests, know how to evaluate statistical significance... ...thinkers with access to quality news. Be part of... ...directly with founders and senior leaders—no bureaucracy...QualityLocal area
Haystack News
San Francisco, CAa month ago - ...Job Description Job Description Operations Lead Location: San Francisco, CA Employment Type: Full-Time, Direct Hire... ...building maintenance teams. Supervise housekeeping teams, evaluate cleaning quality, and provide immediate feedback. Perform light...QualityFull timeLocal areaImmediate startFlexible hoursWeekend workWeekday work
Global IT Resources Inc.
San Francisco, CA7 days ago - ...foundational role on Sierra's GTM Operations team, embedded with our... ..., workflows, reporting, data quality, and automationRun partner business... ...building forecasts, evaluating commercial scenarios, and translating... ...Finance, Legal, Product, and senior leadershipA highly structured...QualityFull timeFlexible hours
Sierra
San Francisco, CA6 days ago - ...agents and workflows, and partner across the company to replace busywork with real leverage. You’ll design evaluation frameworks to measure effectiveness, own quality, and drive adoption by ensuring teams trust and use the AI tools. The role blends technical work with...Quality
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Evaluation Quality & Operations Lead. Be the first to apply!
Related searches
- operations leader San Francisco, CA
- operations lead San Francisco, CA
- network operations center team lead San Francisco, CA
- clinical operations lead San Francisco, CA
- senior maintenance supervisor San Francisco, CA
- senior lead project manager San Francisco, CA
- senior robotics software engineer San Francisco, CA
- senior devops engineer remote San Francisco, CA
- senior sas administrator San Francisco, CA
- senior IT manager San Francisco, CA