Senior Software Engineer Evaluator - AI Output Quality
Mercor
Obsidian is seeking expert Evaluators in Software Engineering to review AI-generated slides, spreadsheets, and documents for real-world quality. You will apply deep subject-matter expertise to grade outputs and provide actionable feedback in a clear, structured format. This remote, hourly engagement requires 5+ years of software engineering experience at top firms in English-speaking markets and fluency in English. Proficiency with Microsoft Office and Google Workspace is essential. #J-18808-Ljbffr Mercor
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer Evaluator - AI Output Quality in San Francisco, CA vacancy
$200.7k - $250.9k
In 1965, an engineer in Scotland was given a mundane task... ...Command is builtBuild quality in:Write and expand... ...candidateHas 7 or more years of software engineering experience,... ...just whether the model outputs something... ...opinions about what makes an AI product trustworthy, not...QualitySeniorShift workMercury
San Francisco, CA2 days ago$165k - $226k
...Every Identity, from AI to HumanIdentity is the... ...too, let's talk.The Senior Software Engineer OpportunityThe team... ...build the harnesses and evaluations that make these... ...agent execution or model quality and help turn it into... ...model APIs, structured outputs and asynchronous...QualitySeniorLocal areaWorldwideFlexible hoursOkta
San Francisco, CA2 days ago- YO AI Labs seeks an AI Evaluation Specialist to assess enterprise AI outputs on a contractual, remote basis across the US, Canada, UK, Ireland, Australia, and New Zealand. You will apply rubrics, identify weaknesses, and provide concrete feedback to improve systems. The...QualityRemote job
$160k - $180k
.... About the RoleAs a Senior Software Engineer on the MarTech Engineering... ...building production AI agents that solve... ...how to ensure agent outputs translate into safe,... ...measured by the volume and quality of marketing work... ...reliability, observability, evaluation, and cost efficiency...QualitySeniorFull timeLocal areaRemote workJones Lang LaSalle
San Francisco, CA1 day ago$165k - $250k
...Meet Your Future NeighborsAs a Software Engineer at Nextdoor, you’ll join a... ...impact, without compromising on quality and reliability as guided by... ...Nextdoor, we operate in an AI-first environment and expect... ...full ownership of AI-assisted outputs.We also offer a warm and inclusive...QualitySeniorWork at officeLocal areaWork from homeFlexible hoursNextdoor
San Francisco, CA1 day ago- ...contract Customer success / support operations Evaluator to assess AI-generated artifacts against domain-specific quality rubrics, identify errors in documents,... ...and provide structured feedback to improve AI outputs. You will work independently and asynchronously...QualityRemote jobContract work
$117.2k - $223.9k
...SalesforceSalesforce is the #1 AI CRM, where humans... ....Your ImpactAs an engineer, your job... ...Build and ship high-quality, production-grade software ,with AI as a core part... ...AI jobs.Critically evaluate code (Human or AI-generated... ..., parseable LLM outputs (JSON classification...QualitySeniorFull timeWork at officeFlexible hours3 days per weekSalesforce
San Francisco, CA2 days ago- Mercor is seeking expert Evaluators in Humanities / arts / culture to review AI-generated work products for accuracy, rigor, and domain quality. This remote hourly engagement requires deep subject-matter expertise to grade outputs and provide actionable feedback. Candidates...QualityRemote jobHourly payWork at office
- Obsidian is seeking expert Evaluators in Document/Deck production QA to review AI-generated outputs and assess accuracy, rigor, and domain quality. You will apply deep subject-matter expertise to grade documents, spreadsheets, and slide decks.This remote, hourly engagement...QualityRemote jobHourly payWork at office
$10 per hour
...team is a brand new, AI-first engineering team in San Francisco... ...manual intervention.As a Senior Engineer, you will... ...the bar on code quality, helping junior engineers... ...quality of the team's output.Collaborate closely... ...years of professional software engineering experience...QualitySeniorFlexible hoursFlexport
San Francisco, CA2 days ago- ...systems that run millions of AI workflows. This means building... ...levels of ownership to ship high quality features You habitually... ...user experience as much as the engineering behind it You have a strong understanding... ..., every parameter, and every output. The most powerful workflow...QualitySenior
$187k
...scientists and ML engineers to develop, train... ...efficiently.As a Senior Software Engineer on the... ...learning and emerging AI workloads,... ...foundation-model access, evaluation, and agentic... ...testing, online quality signals, human feedback... ..., structured outputs, tool calling,...QualitySeniorFull timeWork at officeLocal areaRemote workChime
San Francisco, CA2 days ago$1,000 per month
...life stage. Our AI-native platform helps... ...the reach, quality, and humanity of... ...human judgment to AI outputs, and keep... ...Reporting to the Engineering Manager - Search... ...Matching , this Senior Software Engineer II will... ...quality standards. Evaluate new AI tools and...QualitySeniorWork at officeWorldwideRelocationSleeping nights2 days per week3 days per week$190k - $215k
...revolutionizing healthcare for senior living community... ..., we build software to make them cheap... ...Senior Software Engineer role at Pine Park... ...heavily on the AI tools the company... ...to make sure the quality of the code matches... ...is what makes that output production-grade...QualitySeniorPart timeFlexible hours$153k - $204k
...Essential Cloud for AI™. Built for... ...Do: Applications Engineering, part of CoreWeave... ...organization, builds the software CoreWeave uses to... ...data, plus the evaluation work that tells you... ...measured whether their output was correct ~... .... Here are a few qualities we've found compatible...QualitySeniorPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hours- ...working at Coinbase. Senior Software Engineer, Backend/Fullstack (Coinbase... ...trading system: an AI advisor living inside... ..., orchestration, or evaluation Proven ability to... ...deliver business-ready outputs and drive measurable... ..., cost, and quality. Pay Transparency Notice...QualitySenior
$124k - $186k
...positive mark on culture. Senior Software Engineer, CMS Platform Paramount... ...management platform - shipping high-quality code across a polyglot stack... ...— so both people and AI agents can ramp quickly. •... ...effectively, when to trust the output, and how to catch what it...QualitySeniorFull timeLocal areaSan Francisco, CAa month ago$186.07k - $218.9k
...at Coinbase. We're hiring a Senior Software Engineer to join the Payments team within... ...Contribute to engineering quality through thoughtful code... ...auditability Utilizes generative AI responsibly, maintaining... ...oversight to deliver business-ready outputs and drive measurable...QualitySeniorLive inLocal area$220k - $230k
...or Mountain View, CA, USAR&D - Engineering /Full-time /HybridResponsibilities... ...and foster a culture where quality is paramount.About You:Strong experience in backend software developmentDeep expertise in one... ...the JVM works.Experience using AI tools such as Claude Code.Nice...QualitySeniorFull timeAera Technology
San Francisco, CA13 hours ago$155k - $400k
...standard and our team is building its AI-native future.About the roleSentry's issue... ...needs to know to investigate.As a Senior Software Engineer on the Issue Detection team, you'll design... ...technical product decisions: grouping quality, search performance, migrations and backfills...QualitySeniorWork experience placementImmediate startSentry
San Francisco, CA4 days ago- Lead by example, driving engineering excellence across the team while... ...s goal, roadmap, and broader AI strategy.Stay ahead of the evolving... ...including Google's software supply chain integrity program... ...versatile, display leadership qualities and be enthusiastic to take on...QualitySenior
Google
San Francisco, CA3 days ago $152.5k - $205k
...exceptional support. As a member of a dedicated engineering team, you will contribute to Circle’s... ..., and maintaining a high standard for quality. You will collaborate closely with Product... ..., continuous delivery practices, and AI-assisted development tools to improve engineering...QualitySeniorFlexible hoursCircle
San Francisco, CA3 days ago$212.5k - $250k
...connected platform and AI-native ecosystem for private... ...brings together the software, services, and legal... .... You'll join a small, senior team that blends startup... ...SolveAs a Senior Software Engineer II, KYC, you will lead... ...workflow, improve code quality, and spend more of your...QualitySeniorFull timeCarta
San Francisco, CA1 day ago$192k - $240k
..., bill pay, and travel software, Brex enables founders... ...spend effortlessly. Brex’s AI-native automation and... ...to grow your career.Engineering at BrexEngineering at Brex... ....What you’ll doAs a Senior Frontend Engineer, you... ...technical vision with a high-quality bar.Where you’ll...QualitySeniorWork at officeRemote workWork from homeBrex
San Francisco, CA2 days ago$192k - $240k
..., bill pay, and travel software, Brex enables founders... ...spend effortlessly. Brex’s AI-native automation and... ...to grow your career.Engineering at BrexEngineering at Brex... ...as Okta and Oso.As a Senior Software Engineer on IAM... ...define the team’s technical quality, and help evolve the...QualitySeniorWork at officeRemote workWork from homeBrex
San Francisco, CA2 days ago$180k - $260k
...survey platform, rebuilt around AI agents.For two decades, teams... ...the RoleAgents pay off when engineers hand them whole changes, which... ...automatic flake handling, and quality checks that act as real gates.... ...agentic workflows within the software development lifecycle (SDLC).Ownership...QualitySeniorFull timeWork at officeLocal areaSprig
San Francisco, CA3 days ago$190k - $235k
...Handler, which empowers AI agents with secure... ...churn, and save engineering resources—allowing... ..., with candidate evaluation based on a... ...discussionsProduce high-quality, production-ready,... ...of full-time software engineering experience (looking for senior+ experience)An ability...QualitySeniorFull timeWork at officeHome officeMerge API
San Francisco, CA13 hours ago$192k - $240k
..., bill pay, and travel software, Brex enables founders... ...spend effortlessly. Brex’s AI-native automation and... ...to grow your career.Engineering at BrexEngineering at Brex... ....What you’ll doAs a Senior Software Engineer, Infrastructure... ...to elevate the overall quality of our...QualitySeniorWork at officeRemote workWork from homeBrex
San Francisco, CA4 days ago$174k - $252k
...user feedback.Build the software systems, automated pipelines, evaluation harnesses, and knowledge... ...context that accelerates AI agents and human... ...guides).Google's software engineers develop the next-generation... ...versatile, display leadership qualities and be enthusiastic to...QualitySeniorGoogle
San Francisco, CA2 days ago- ...dataset with deep expertise in AI and machine learning. Our... ...craft: performance, interaction quality, and accessibilityDrive... ...ll Bring5+ years of frontend engineering experienceYou have empathy for... ...compensation decision.We build software that transforms how companies...QualitySeniorFlexible hours
Trove Information Technologies
San Francisco, CA1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer Evaluator - AI Output Quality. Be the first to apply!
Related searches
- scientific software engineer San Francisco, CA
- intel software engineer San Francisco, CA
- director software engineering San Francisco, CA
- software engineer San Francisco, CA
- software engineer student San Francisco, CA
- senior robotics software engineer San Francisco, CA
- financial software developer San Francisco, CA
- graduate software developer San Francisco, CA
- part time software developer remote San Francisco, CA
- software engineer remote San Francisco, CA