Senior AI Quality Engineer—Regression & Evaluation
$176k - $253kHarper
Harper, an AI-native commercial insurance company in San Francisco, seeks a Senior Member of Technical Staff focused on AI quality evaluation. This role requires building evaluation systems to measure agent performance and ensure high standards in service. The ideal candidate will have 3–6 years of software building experience, especially in LLM and agent evaluations, and should thrive in a dynamic environment. Compensation ranges from $176,000 to $253,000 plus equity, with extensive in-office benefits. #J-18808-Ljbffr Harper
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior AI Quality Engineer—Regression & Evaluation in San Francisco, CA vacancy
$240k - $280k
A leading software monitoring company is seeking a Senior Software Engineer on its AI/ML team to build evaluation infrastructure for measuring the performance of AI systems. This role involves designing datasets, creating benchmarks, and ensuring AI features behave reliably...Senior- Drata is seeking a Senior Applied Research Engineer to enhance the quality of AI systems through rigorous evaluation and experimentation. This role emphasizes applied research, focusing on information retrieval and reasoning strategies. The ideal candidate will bring 5+...Senior
- Cacheflow is seeking a Senior Applied Research Engineer to enhance AI systems through rigorous experimentation and applied research. This research-focused... ...will design information access strategies, evaluate innovative methodologies, and collaborate closely with...SeniorFlexible hours
$124k - $280k
...expertise, and network to deliver quality results. You motivate and... ...through innovative, AI-driven solutions. As a Senior Manager, you will lead... ...strategy, transformation and engineering projects and teams Design... ...with team members. We evaluate these factors thoughtfully...SeniorFull timeH1bPwC
San Francisco, CA3 days ago$80 - $120 per hour
Mercor is looking for an Engineering / Manufacturing / Technical Operations Evaluator to assess AI-generated artifacts. The role requires 5+ years of relevant experience... ...in identifying errors and providing quality feedback to enhance AI outputs. This is a remote...SuggestedRemote jobHourly payContract workWork at office$77k - $202k
...Privacy Management Level Senior Associate Job... ...clients, and to deliver quality. Embracing increased ambiguity... ...clients through innovative, AI-driven solutions. As a... ...development or AI/ML engineering What Sets You Apart... ...with team members. We evaluate these factors thoughtfully...SeniorH1b$150k - $250k
...Distyl AI Job Posting Distyl is an applied... ...build AI systems using Evaluation-Driven Development —an... ...production. AI Evaluation Engineers focus on designing and... ...Define how system quality is measured in each... ...golden test cases and regression suites in Python, using...Work at office3 days per weekDistyl AI
San Francisco, CA2 days ago$180k - $300k
A leading AI company in San Francisco seeks an experienced candidate to own and elevate core AI systems that power their services... ...requires expertise in TypeScript and Python, focusing on prompt engineering for text, image, and layout generation. With a hybrid work...Senior- ...Senior Principal Ai Agent / Ml Software Engineer The Senior Principal AI Agent / ML Software Engineer is a Senior... ...tools, APIs, memory, retrieval, evaluation, guardrails, and cloud services.... ...tracing, monitoring, eval suites, regression testing, experimentation, safety...Senior
Oracle
San Francisco, CA20 hours ago $150k - $180k
...Global (NYSE: ZETA) is the AI-Powered Marketing Cloud... ...a hands-on Agentic AI engineer to build bidder-... ...Observe → Reason → Act → Evaluate). This is not a classic... ...AgentOps: eval harnesses, regression suites, online... ...outcomes (CPA/ROAS, pacing, quality, margin). Add observability...SeniorZeta Global
San Francisco, CA2 days ago$144.5k - $230k
...Senior AI Engineer We are the better way to work in finance. As private... ...and RAG pipelines to evaluation frameworks and production observability... ...production AI system quality Travel to client site as... ...quality is measurable and regressions are caught early Demonstrate...SeniorWork at officeLocal areaRemote work2 days per weekAccordion USA
San Francisco, CA2 days ago- A global professional services firm based in San Francisco seeks a Senior Associate in Cybersecurity to develop innovative AI-driven solutions. You will leverage your skills in software development and AI/ML to address complex cybersecurity challenges, mentor team members...Senior
- ...Senior AI Engineer Disney Entertainment and ESPN Product & Technology is a global... ...governance, observability, and evaluation, so teams can deliver high-quality AI solutions quickly—without reinventing... ...output validation, and quality regression testing. ~ Strong...Senior
The Walt Disney Studios
San Francisco, CA2 days ago $141.9k - $190.3k
...global organization of engineers, product developers, designers... ...Summary: We're hiring a Senior AI Engineer to build the AI... ..., observability, and evaluation, so teams can deliver high-quality AI solutions quickly—... ...validation, and quality regression testing. * Strong collaboration...SeniorDisney Entertainment and ESPN Product & Technology Careers
San Francisco, CA4 days ago$191k - $223k
Nerdleveltech is looking for a Senior Software Engineer to join the Quality Platform team in San Francisco. In this role, you will contribute to building and evolving AI-native quality workflows, ensuring high-quality software delivery. Your responsibilities will include...SeniorRemote job- ...Fieldguide is building AI agents for the most complex... ...investors. As an AI Engineer, Quality , you will own the evaluation infrastructure that ensures... ...levels. We'll calibrate seniority during interviews based... ...that catch quality regressions before they reach customers...SeniorWork at officeRemote workWork from homeFlexible hours
$155k
...to the semantic and AI layers that sit on... ...work for everyone - engineers, analysts, and... ...We're looking for a Senior AI Data Engineer to... ...trained, aligned and evaluated (RLHF, fine-tuning,... ...harnesses, defining quality metrics, and catching regressions before they reach production...SeniorFor contractorsLocal areaHome officeFlexible hoursScribd
San Francisco, CA3 days ago$170k - $200k
...public benefit corporation and AI-enabled medical group, we... ...Complex NeedsAbout The RoleAs a Senior AI Engineer, you will help design and... ...through observability, evaluation, and continuous improvement... ..., simulation-based testing, regression frameworks, metrics design,...SeniorTemporary workLocal areaWork from home$96.8k - $306.4k
...Job Description The Senior Principal AI Agent / ML Software Engineer is a Senior Staff-level... ..., memory, retrieval, evaluation, guardrails, and cloud... ...monitoring, eval suites, regression testing,... ...ability to contribute high-quality production code, reviews...SeniorTemporary workFlexible hoursOracle
San Francisco, CA2 days ago$155k
...About the Team The Quality Engineering team builds the... ...We are looking for a Senior Software Engineer, Quality... ...in implementing how AI reshapes quality... ...contract tests, and regression gates keep pace as AI... ...Experience using or evaluating AI-powered engineering...SeniorContract workFor contractorsLocal areaHome officeFlexible hoursShift workEarly shiftScribd
San Francisco, CA3 days ago- Cacheflow is seeking a Senior Applied Research Engineer to enhance the effectiveness of our AI systems through focused research and experimentation. This role involves designing information retrieval strategies and collaborating with engineers to turn validated approaches...SeniorFlexible hours
$192k - $237.1k
...ambitious path to redefine how AI and General AI... ...are seeking an Applied AI Engineer to drive the quality and effectiveness of our AI... ...research, experimentation, and evaluation. In this role, you will optimize... ...quality metrics, regression detection Implement and tune...SeniorWork at officeImmediate startWorldwideMonday to FridayFlexible hours- Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market. Inference is just one piece...SeniorWork at officeImmediate start
- The Role We're hiring Senior Backend + Applied AI Engineers to build the core systems that... ...only the teams who ship quality products & features at... ...in terms of evals, model regression tests, traceability, and... ...orchestration patterns and evaluation frameworks the team standardizes...SeniorWork at officeLocal areaRelocation package
$200k - $240k
...blockchain analytics and AI solutions to help... ...for all. The AI Engineering Team is chartered... ...involved in evaluating and integrating cutting... ...the market. As a Senior or Staff AI... ...agents — including regression testing, cost monitoring... ...For Write high-quality, maintainable software...SeniorRemote workWorldwide$155k
...About the Team The Quality Engineering team builds the... ...We are looking for a Senior Software Engineer, Quality... ...in implementing how AI reshapes quality... ...contract tests, and regression gates keep pace as AI... ...Experience using or evaluating AI-powered engineering...SeniorContract workFor contractorsLocal areaHome officeFlexible hoursShift workEarly shiftScribd
San Francisco, CA3 days ago$150k - $180k
Location: Remote, located in the US Type: Full-time Department: Engineering Reports to: Director Of Engineering Responsibilities Build and maintain infrastructure and tooling for the AI evaluations platform used by internal teams, including automated testing platform for...Full timeRemote workFlexible hours- A pioneering AI technology firm based in San Francisco is seeking an AI Engineer to own the evaluation infrastructure for AI agents. This role requires designing automated pipelines and building observability systems, ensuring agent performance meets enterprise standards...Remote jobFlexible hours
- A cutting-edge AI firm in San Francisco is seeking a Research Engineer to develop evaluation systems and benchmarking pipelines for language models. Candidates should have a strong background in applied research, coding skills, and familiarity with ML models. You will work...
$240k
...Title : Senior AI Engineer Location : San Francisco, CA (Hybrid) Compensation : Up to... ...foundational decisions around systems, evaluation, and long-term technical direction, working... ...frameworks to measure retrieval quality, reasoning accuracy, and system performance...SeniorHarnham
San Francisco, CA21 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Quality Engineer—Regression & Evaluation. Be the first to apply!
Related searches
- senior ai engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- ai engineer San Francisco, CA
- ai prompt engineer San Francisco, CA
- ai developer San Francisco, CA
- ai research engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- senior manager quality engineering San Francisco, CA
- senior quality systems engineer San Francisco, CA