Software Engineers: Paid Code Review for AI Agent Evaluation

$65 per hour

Temporary

Terac

San Francisco, CA

What You'll Do
Review realistic programming tasks for accuracy and difficulty Verify the logic and test cases within provided evaluation harnesses Assess whether coding environments effectively measure software engineering skills Walk us through your thought process while analyzing complex code structures
Who Should Apply
Active professional experience as a software engineer Hands-on experience building or maintaining test suites and evaluation harnesses Comfortable reviewing code and explaining technical concepts aloud Familiarity with complex real-world software architecture
Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.
Pay: $65

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Software Engineers: Paid Code Review for AI Agent Evaluation in San Francisco, CA vacancy

Software Engineer - AI Code Evaluation & Benchmarking (US candidates only)
...Evaluate and improve AI-generated code for accuracy and quality. About Turing Turing... ...looking for experienced Software Engineers to help evaluate, benchmark... ...engineers who enjoy code review, debugging, problem-... ...Contractor assignment (no medical/paid leave) Duration of...
Suggested
Contract work
Part time
For contractors
Freelance
Remote work
Turing
San Francisco, CA
1 day ago
South Asian Software Engineers: Coding Tasks for AI Evaluation
$40 per hour
...What You'll Do Review proposed coding tasks and programming environments for realism Provide... ...on the structure and difficulty of evaluation harnesses Walk us through your approach... ...Who Should Apply Professional software engineer based in South Asia Experience building...
Suggested
Contract work
Temporary work
For contractors
Remote work
Terac
San Francisco, CA
1 day ago
Senior Application Security Engineer - AI Code Evaluation
$75 - $90 per hour
...States | Canada + 21 more Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security... ...not required Professional code review, coding annotation, LLM/code evaluation...
Suggested
Temporary work
Remote work
Braintrust
San Francisco, CA
1 day ago
Senior Software Engineer - LLM Evaluation
...Evaluate and refine AI-generated code for efficiency and reliability. About... ...multimodality, and agents; and second, by... ...Role Overview As a Software Engineering evaluator, you will... ..., and code quality/review assessment.... ...Contractor (no medical/paid leave) Duration...
Suggested
Full time
Part time
For contractors
Flexible hours
Turing
San Francisco, CA
1 day ago
Software Engineer I, Build Review & Test
$136k - $170k
...started Handshake AI and built the... ...to create evaluations, publish benchmarks... ...together with engineers, scientists, operators... ...looking for a Software Engineer to... ...understand. Coding agents are a core part... ...and tests, review changes, and explain... ...Support: Paid parental leave,...
Suggested
Full time
Work at office
Remote work
Flexible hours
Handshake
San Francisco, CA
9 days ago
Software Engineer, Agent Builder
$230k - $390k
...platform for customer-facing AI agents, working with many of... ...create intuitive, no-code tools that allow anyone... ...we adapt traditional software development methodologies... ...:Flexible (unlimited) paid time offMedical, dental... .... We strive to evaluate all applicants consistently...
Full time
Flexible hours
Sierra
San Francisco, CA
2 days ago
Software Engineer, Code Quality
$159.8k - $235k
About the TeamThe Code Quality team sits... ...sure the signals engineers rely on before shipping... ...'re looking for Software Engineers to help... ...rolloutsExplore how AI agents and tools can... ...matching, 16 weeks of paid parental leave,... ...Covey tool has been reviewed by an independent...
Hourly pay
Work at office
Local area
Remote work
Flexible hours
Doordash
San Francisco, CA
23 hours ago
Backend Engineer - AI Coding Agent
$85 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: Backend Engineer (Coding Agent Experience) Type: Contract... ...AI coding agents to complete and evaluate complex engineering tasks. Review model-generated code for correctness...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
16 days ago
Software Engineer, Security Agents
...-first enterprise AI company. We build... ...team of researchers, engineers, designers, and... ...OverviewReady to build AI agents that revolutionize... ...for a Senior Software Engineer to create... ...triage, secure code reviews, threat modeling,... ...coaching.6 weeks of paid vacation (30 working...
Full time
Work at office
Local area
Remote work
Home office
Cohere
San Francisco, CA
a month ago
Software Engineering Architect - DevXP AI team
$218.4k - $365.2k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive customer... ...a trailblazing engineering leader who is... ..., evangelism, evaluation, and cost... ...like AI code review agents for use... ...AI accelerates software development at... ...health support, paid parental leave,...
Full time
Salesforce
San Francisco, CA
1 day ago
Senior Software Engineer - LLM Evaluation & Repository Validation
...Analyze GitHub issues and evaluate LLM performance on real code. Job Title: Senior Software Engineer – LLM Evaluation &... ...world’s fastest-growing AI companies accelerating... ...tools or automation agents. Perks of Freelancing... ...(no medical/paid leave) Duration of...
Contract work
Part time
For contractors
Freelance
Remote work
Turing
San Francisco, CA
1 day ago
Senior Software Engineer - Backend
$170k - $240k
...we are growing the engineering team and looking... ...and implement new software features to support... ...through design and code reviews to ensure best... ...us: Sigma is the AI Apps and agentic analytics... ...time off you need!Paid bonding time for... ...the purposes of evaluating your candidacy for...
Full time
Work at office
Sigma Computing
San Francisco, CA
3 days ago
Software Engineer, Agent
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...examples of projects that engineers on our team have worked on recently... ...:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
3 days ago
Software engineer, agents
$146k - $240k
...leading enterprises orchestrate AI-powered work. Our vision is... ...building and deploying AI agents that are grounded in their... ...a highly skilled fullstack software engineer to join our engineering team... ...and participate in rigorous code reviews to uphold quality and maintainability...
Full time
Work at office
Local area
Writer
San Francisco, CA
3 days ago
Software Engineer, Agents
$180k - $250k
...product-minded fullstack engineer to build the core... ...experiences that power our AI agents. You’ll work across... ...deployment.Research and evaluate new approaches to... ...practices.Participate in peer reviews, brainstorming... ...in a product-oriented software engineering role.A strong...
Work at office
11x AI
San Francisco, CA
3 days ago
Software Engineer, Agent - Healthcare
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...been a founder or founding engineer—you know what it means to balance... ...:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
2 days ago
Software Engineer, Agent - Travel & Hospitality
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...we have to meet. Every engineer on TTH is also a product-led... ...employees:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
2 days ago
Software Engineer, Codex Core Agents
$230k - $385k
...TeamThe Codex Core Agent team builds the kernel... ...’re looking for engineers to build the infrastructure... ...safely execute code, interact with... ...developer tooling, and AI, building... ...into a dependable software engineering agent.Bonus... ...birthing parents), plus paid medical and...
Work at office
Local area
Flexible hours
OpenAI
San Francisco, CA
3 days ago
Software Engineer, Agent Infrastructure
$230k - $385k
About the TeamThe Agent Infrastructure team at... ...deployment of highly useful AI agents, both... ...AI models to execute code, debug issues, and develop software just as human SWEs... ...the RoleAs a Software Engineer on the Agent... ...birthing parents), plus paid medical and caregiver...
Work at office
Local area
Worldwide
Relocation package
Flexible hours
OpenAI
San Francisco, CA
3 days ago
Software Engineer III - ServiceNow
...you to take your software engineering career to the next... ...quality production code and maintains algorithms... ...-authorized AI coding assist tools... ...through peer review, automated testing... ...ability to critically evaluate, validate, and refine... ...compensation, paid in the form of cash...
JP Morgan Chase
San Francisco, CA
2 days ago
Onboard Software Engineer - Hyperdrive
$155k - $225k
...CASoftware - Onboard Software /Full time /On-... ...robustness and reliability.Review code, provide clear... ...+ years' software engineering experience, using... ...:100% company-paid medical, dental, and... ...wide as we evaluate individuals based... ...connectivity, on-orbit AI, national security...
Full time
Temporary work
Loft Orbital
San Francisco, CA
3 days ago
Software Engineer - Compiler
$170k - $235k
...join a group of engineers dedicated to building... ...stakeholders to review designs and code, driving... ...engineering high-quality software... ...us: Sigma is the AI Apps and agentic... ...time off you need!Paid bonding time for... ...the purposes of evaluating your candidacy for...
Full time
Work at office
Sigma Computing
San Francisco, CA
3 days ago
Senior Software Engineer, Developer Experience
$172.43k - $209k
...vertically integrated AI infrastructure... ...future of Crusoe’s engineering muscle. We’re... ...efficiently. As a Senior Software Engineer you will... ...optimization, code review/CI experience and... ...Expert ability to evaluate technical tradeoffs... ...UnitsPaid time off & paid...
Full time
Temporary work
Work at office
Local area
Crusoe
San Francisco, CA
23 hours ago
Agentic Ontology Software Engineer
$50 per hour
...team as a Senior AI Engineer focused on... ...source data.Build AI agent workflows that create, evaluate, and improve ontology... ..., and human review.Integrate... ...Build production software across services,... ...Review designs and code, mentor other engineers... ..., 401(k) match, Paid time off,...
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
San Francisco, CA
2 days ago
Onboard Software Engineer - Deployment & Runtime Environments
$144k - $198k
...COSoftware - Onboard Software /Full time /On-... ...scrappy — the kind of engineer who digs into an... ...in code reviews, design proposals,... ...Benefits:100% company-paid medical, dental, and... ...intentionally wide as we evaluate individuals based... ...connectivity, on-orbit AI, national security...
Full time
Temporary work
Loft Orbital
San Francisco, CA
23 hours ago
Senior Software Engineer, AI & App Experience
$187k - $259k
...Senior Full-Stack Engineer to join our... ...Jade, Chime's AI-powered financial... ...designing new agent capabilities and... ...orchestration, and evaluation pipelines. The... ...through code reviews and mentorshipBring... ...years of backend software development... ...and company-wide paid days offð« 1% of...
Full time
Work at office
Local area
Remote work
Shift work
Chime
San Francisco, CA
4 days ago
Software Engineering LMTS
$172.5k - $260.1k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...is seeking engineers to architect,... ..., leadership reviews, or Tech... ..., testable code with an eye... ...8+ years of software development experienceExperience... ...assess and evaluate candidates’... ...support, paid parental...
Full time
Salesforce
San Francisco, CA
1 day ago
Software Engineer, iOS, Level 5
$209k - $313k
...digital services.Snap Engineering teams build fun... ...’re looking for a Software Engineer, iOS to... ...featuresCarefully evaluate the technical tradeoffs... ...out code reviews to guarantee code qualityUtilize AI tools and high velocity... ...needs and include paid parental leave, comprehensive...
Full time
Live in
Work at office
Local area
Snap
San Francisco, CA
1 day ago
Research Engineer - Code Generation & Model Evaluation
$1,000 per month
...is the leading AI data lab for... ...frontier models and evaluating AI agents. Experts... ...healthcare, STEM engineering, and more.... ...Research Engineer - Code Generation &... ...Develop and review real-world coding... ...collaborative software projects.... ...; experts are paid per task that...
Part time
For contractors
Remote work
micro1
San Francisco, CA
13 days ago
Senior Software Engineer - Observability and Reliability
$170k - $240k
Senior Software Engineer - Observability and ReliabilityAbout... ...stakeholders through design and code reviews to ensure best... ...us: Sigma is the AI Apps and agentic analytics... ...the time off you need!Paid bonding time for all new... ...for the purposes of evaluating your candidacy for employment...
Full time
Work at office
Sigma Computing
San Francisco, CA
3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineers: Paid Code Review for AI Agent Evaluation. Be the first to apply!