Senior Software Engineer Evaluator - AI Output Quality
Mercor Inc
Obsidian is seeking expert Evaluators in Software Engineering to review AI-generated slides, spreadsheets, and documents for real-world quality. You will apply deep subject-matter expertise to grade outputs and provide actionable feedback in a clear, structured format. This remote, hourly engagement requires 5+ years of software engineering experience at top firms in English-speaking markets and fluency in English. Proficiency with Microsoft Office and Google Workspace is essential. #J-18808-Ljbffr
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer Evaluator - AI Output Quality in San Francisco, CA vacancy
- ...systems that run millions of AI workflows. This means building... ...levels of ownership to ship high quality features You habitually... ...user experience as much as the engineering behind it You have a strong... ...model, every parameter, and every output. The most powerful workflow...QualitySenior
Mixpeek
San Francisco, CA4 days ago $200.7k - $250.9k
In 1965, an engineer in Scotland was given a mundane task... ...Command is builtBuild quality in:Write and expand... ...candidateHas 7 or more years of software engineering experience,... ...just whether the model outputs something... ...opinions about what makes an AI product trustworthy, not...QualitySeniorShift workMercury
San Francisco, CA4 days ago$165k - $226k
...Every Identity, from AI to HumanIdentity is the... ...too, let's talk.The Senior Software Engineer OpportunityThe team... ...build the harnesses and evaluations that make these... ...agent execution or model quality and help turn it into... ...model APIs, structured outputs and asynchronous...QualitySeniorLocal areaWorldwideFlexible hoursOkta
San Francisco, CA9 days ago$165k - $250k
...Meet Your Future NeighborsAs a Software Engineer at Nextdoor, you’ll join a... ...impact, without compromising on quality and reliability as guided by... ...Nextdoor, we operate in an AI-first environment and expect... ...full ownership of AI-assisted outputs.We also offer a warm and inclusive...QualitySeniorWork at officeLocal areaWork from homeFlexible hoursNextdoor
San Francisco, CA8 days ago$160k - $180k
.... About the RoleAs a Senior Software Engineer on the MarTech Engineering... ...building production AI agents that solve... ...how to ensure agent outputs translate into safe,... ...measured by the volume and quality of marketing work... ...reliability, observability, evaluation, and cost efficiency...QualitySeniorFull timeLocal areaRemote workJones Lang LaSalle
San Francisco, CA8 days ago- ...Senior Software Frontend Engineer, Market Pricing Who We Are At Pave, we're building... ...with deep expertise in AI and machine learning. Our... ...: performance, interaction quality, and accessibility Drive... ...that extend being your own output ~ You invest meaningfully...QualitySeniorFlexible hours3 days per week
Trov
San Francisco, CA4 days ago $117.2k - $223.9k
...SalesforceSalesforce is the #1 AI CRM, where humans... ....Your ImpactAs an engineer, your job... ...Build and ship high-quality, production-grade software ,with AI as a core part... ...AI jobs.Critically evaluate code (Human or AI-generated... ..., parseable LLM outputs (JSON classification...QualitySeniorFull timeWork at officeFlexible hours3 days per weekSalesforce
San Francisco, CA4 days ago$186.07k - $218.9k
...at Coinbase. We're hiring a Senior Software Engineer, Mobile QA to join the App Infra... ...meets an uncompromising quality bar. You’ll build automated... ...frameworks directly into the AI-assisted code generation loop... ...to deliver business-ready outputs and measurable gains in workflow...QualitySeniorLocal areaCoinbase
San Francisco, CA4 days ago$200k - $220k
...TRM Labs provides AI-powered intelligence... ...more secure. As a Senior Software Engineer on the Graph Analytics... ...native technology to evaluate benefit to data science... ...data to complex model outputs What we’re looking... ...Ensures system health and quality with operational...QualitySeniorWorldwideTRM Labs
San Francisco, CA2 days ago$180k - $220k
...production infrastructure our software runs on. You'll build... ...do Build CI for an AI-enabled team. Build... ...merges. Make quality part of the pipeline, not... ...this role is not A QA engineer writing tests and test... ...every parameter, and every output. The most powerful...QualitySeniorMixpeek
San Francisco, CA4 days ago$190k - $215k
...revolutionizing healthcare for senior living community... ..., we build software to make them cheap... ...Senior Software Engineer role at Pine Park... ...heavily on the AI tools the company... ...to make sure the quality of the code matches... ...is what makes that output production-grade...QualitySeniorPart timeFlexible hoursPINE PARK HEALTH MANAGEMENT INC
San Francisco, CA4 days ago- ...InsightRX | Engineering About the Role InsightRX is... ...to help build our Astra AI platform — a system... ...statistical modeling, data quality checks, and PK/PD... ...quirks of working with LLM output, even without deep AI/ML... ...or healthcare/life-sciences software. #J-18808-LjbffrQualitySenior
HealthX Ventures
San Francisco, CA20 hours ago $207k - $233k
...Senior Software Engineer II, Search & Matching (Alma) Spring Health... ...life stage. Our AI-native platform helps... ...should expand the reach, quality, and humanity of care... ...judgement to AI outputs, and keep building AI... ...quality standards. Evaluate new AI tools and techniques...QualitySeniorWork at officeWorldwideRelocationSleeping nights2 days per week3 days per weekSpring Health
San Francisco, CA4 days ago- ...Every Identity, from AI to HumanIdentity is the... ...too, let's talk.The Senior Software Engineer OpportunityThe team... ...build the harnesses and evaluations that make these... ...agent execution or model quality and help turn it into... ...model APIs, structured outputs and asynchronous...QualitySenior
Okta, Inc.
San Francisco, CA4 days ago $186.07k - $218.9k
...at Coinbase. Frontend Software Engineer - Coinbase Card Team As... ...and operate as a senior technical voice on a team... ...levels Utilizes generative AI responsibly,... ...deliver business-ready outputs and drive measurable improvements... ...efficiency, cost, and quality Pay Transparency...QualitySeniorLocal areaCoinbase
San Francisco, CA3 days ago- ...a mission to use AI to radically accelerate... ...of founders and engineers have previously... ...We're hiring a Senior Software Engineer to build... ...shaping and engineering quality, and mentor other... ...you verify their output rather than... ...and internal teams evaluate success across turnaround...QualitySeniorFlexible hours
ALT.com
San Francisco, CA3 hours ago $184.7k - $324.8k
...Machine Learning and AI Do you want to... ...with them. We are engineers, scientists, and... ...looking for seasoned AI Software Engineers to join... ...into measurable quality improvements... ...Contribute to the shared evaluation and telemetry loop... ...validating model output ~ Fluency with...QualitySeniorRelocationApple Inc.
San Francisco, CA4 days ago$186.07k - $218.9k
...working sessions called surges. Senior Software Engineer, Simple Trade Experience We'... ...grow Strengthen engineering quality across the team through... ...platforms Utilizes generative AI responsibly, maintaining... ...oversight to deliver business-ready outputs and drive measurable...QualitySeniorLocal areaWorldwideCoinbase
San Francisco, CA4 days ago$186.07k - $218.9k
...at Coinbase. We're hiring a Senior Software Engineer, Native Mobile to join the App... ...Swift and Kotlin, using AI-assisted code generation to... ...dramatically faster, higher-quality experience for millions of customers... ...ensure production-quality outputs. Partner with product teams...QualitySeniorLocal areaImmediate startCoinbase
San Francisco, CA2 days ago$10 per hour
...team is a brand new, AI-first engineering team in San Francisco... ...manual intervention.As a Senior Engineer, you will... ...the bar on code quality, helping junior engineers... ...quality of the team's output.Collaborate closely... ...years of professional software engineering experience...QualitySeniorFlexible hoursFlexport
San Francisco, CA4 days ago$184.7k - $324.8k
...California, United States Software and Services... ...organization. IS&T is the engine behind everything... ...highly experienced Senior Software Engineer... ..., security, and AI/ML teams—to define... ...s expectations for quality, security, performance... ...ensuring generated output remains governed by...QualitySeniorContract workLocal areaRelocationApple Inc.
San Francisco, CA1 day ago- ...seeking an entrepreneurial full-stack Software Engineer with the ability to move back and forth... ...high agency and be used to shipping high quality output at high velocity. This role is ideal... ...Strong desire to learn and integrate AI-driven features into products B2B high...QualitySeniorWork at office
Praxis
San Francisco, CA20 hours ago $138k - $194k
...We are seeking a Staff Software Engineer to join our People... ...behind PeopleOS — our AI-native internal HR platform... ..., handle structured outputs, and build safe, reliable... ...cases Collaboration & Quality Document integration... ...on the scope and seniority of the role, and from...QualitySeniorWork at officeLocal areaRemote workWork from home2 days per week3 days per weekGusto
San Francisco, CA4 days ago$160.9k - $260.7k
...that define how software gets built and delivered. As AI agents redefine software... ...looking for a Senior Software Engineer to build AI-... ...landscape and balance quality with pragmatism.... ..., validate their output, and stay... ...calibration and consistent evaluations. Recordings are...QualitySeniorFull timeLocal areaRemote workWork from homeHome officeVisa sponsorshipShift workDocker
San Francisco, CA4 days ago$210k - $240k
...TRM Labs provides AI-powered intelligence... .... The Product Engineering team is responsible... ...reliable and scalable software that will be used... ...the velocity, quality, and overall impact... ...the opportunity to evaluate TRM. Most interview... ...problems Improve output quality Increase speed...QualitySeniorWork at officeWorldwide3 days per weekTRM Labs
San Francisco, CA16 hours ago- Senior Software Engineer Roger is an AI platform that frees home health clinicians from paperwork so they can focus on what matters: delivering... ...AI-assisted development tools to accelerate high-quality engineering output Prototype MVPs and harden them into scalable,...QualitySenior
$187k
...scientists and ML engineers to develop, train... ...efficiently.As a Senior Software Engineer on the... ...learning and emerging AI workloads,... ...foundation-model access, evaluation, and agentic... ...testing, online quality signals, human feedback... ..., structured outputs, tool calling,...QualitySeniorFull timeWork at officeLocal areaRemote workChime
San Francisco, CA9 days ago$186.07k - $218.9k
...at Coinbase. We're hiring a Senior Software Engineer to join the Payments team within... ...Contribute to engineering quality through thoughtful code... ...auditability Utilizes generative AI responsibly, maintaining... ...oversight to deliver business-ready outputs and drive measurable...QualitySeniorLive inLocal areaCoinbase
San Francisco, CA3 days ago- ...World. TRM Labs provides AI-powered intelligence... ...products and services. As a Senior Software Engineer on Data Infrastructure (RDBMS... ...development and produce higher-quality output faster, with the judgment... ...and leverage You will be evaluated on applied AI fluency during...QualitySeniorWorldwideDay shift
TRM Labs
San Francisco, CA2 days ago $175k - $230k
..., delivering high-quality, physician-led multidisciplinary... ...The Team: The AI Platform Team... ...team of engineers, product managers,... ...hiring a full-stack Senior Software Engineer to build... ...management, structured outputs, tool use, agentic workflows, and evaluation. Develop the...QualitySeniorWork experience placementCOLOR
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer Evaluator - AI Output Quality. Be the first to apply!
Related searches
- software engineer amazon San Francisco, CA
- information technology software engineer San Francisco, CA
- federal - software developer San Francisco, CA
- graduate software developer San Francisco, CA
- rust software engineer San Francisco, CA
- software development engineer (robotics engineer) San Francisco, CA
- IT software developer San Francisco, CA
- software engineer remote San Francisco, CA
- junior software developer internship San Francisco, CA
- remote entry level software developer San Francisco, CA