Software Engineers: Paid Code Review for AI Agent Evaluation

$65 per hour

Temporary

Terac

San Francisco, CA

What You'll Do
Review realistic programming tasks for accuracy and difficulty Verify the logic and test cases within provided evaluation harnesses Assess whether coding environments effectively measure software engineering skills Walk us through your thought process while analyzing complex code structures
Who Should Apply
Active professional experience as a software engineer Hands-on experience building or maintaining test suites and evaluation harnesses Comfortable reviewing code and explaining technical concepts aloud Familiarity with complex real-world software architecture
Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.
Pay: $65

Apply

Vacancy posted 2 days ago

Similar jobs that could be interesting for youBased on the Software Engineers: Paid Code Review for AI Agent Evaluation in San Francisco, CA vacancy

Software Engineers: Paid Code Review for AI Agent Evaluation
$65 per hour
...What You'll Do Review realistic programming tasks for accuracy and difficulty Verify the logic and test cases within provided evaluation harnesses Assess whether coding environments effectively measure software engineering skills Walk us through your thought process...
Suggested
Contract work
Temporary work
For contractors
Remote work
Terac
San Francisco, CA
2 days ago
Software Engineer - AI Code Evaluation & Benchmarking (US candidates only)
...Evaluate and improve AI-generated code for accuracy and quality. About Turing Turing... ...looking for experienced Software Engineers to help evaluate, benchmark... ...engineers who enjoy code review, debugging, problem-... ...Contractor assignment (no medical/paid leave) Duration of...
Suggested
Contract work
Part time
For contractors
Freelance
Remote work
Turing
San Francisco, CA
2 days ago
Senior Software Engineer - AI Evaluation / Coding Agents ($100-150/hr)
$100 - $150 per hour
...Evaluate and improve AI coding models by reviewing generated code. Freelance · Remote · North... ...data, evaluations, and engineering talent. Engagement... ...experienced, hands-on software engineers to help evaluate... ...’ll work with coding agents across real-world repositories...
Suggested
Hourly pay
Part time
For contractors
Freelance
Immediate start
Remote work
Turing
San Francisco, CA
14 days ago
South Asian Software Engineers: Coding Tasks for AI Evaluation
$40 per hour
...What You'll Do Review proposed coding tasks and programming environments for realism Provide... ...on the structure and difficulty of evaluation harnesses Walk us through your approach... ...Who Should Apply Professional software engineer based in South Asia Experience building...
Suggested
Contract work
Temporary work
For contractors
Remote work
Terac
San Francisco, CA
2 days ago
Senior Software Engineer - LLM Evaluation
...Evaluate and refine AI-generated code for efficiency and reliability. About... ...multimodality, and agents; and second, by... ...Role Overview As a Software Engineering evaluator, you will... ..., and code quality/review assessment.... ...Contractor (no medical/paid leave) Duration...
Suggested
Full time
Part time
For contractors
Flexible hours
Turing
San Francisco, CA
2 days ago
Software Engineer I, Build Review & Test
$136k - $170k
...started Handshake AI and built the... ...to create evaluations, publish benchmarks... ...together with engineers, scientists, operators... ...looking for a Software Engineer to... ...understand. Coding agents are a core part... ...and tests, review changes, and explain... ...Support: Paid parental leave,...
Full time
Work at office
Remote work
Flexible hours
Handshake
San Francisco, CA
10 days ago
Software Engineer, Agent Builder
$230k - $390k
...platform for customer-facing AI agents, working with many of... ...create intuitive, no-code tools that allow anyone... ...we adapt traditional software development methodologies... ...:Flexible (unlimited) paid time offMedical, dental... .... We strive to evaluate all applicants consistently...
Full time
Flexible hours
Sierra
San Francisco, CA
3 days ago
Software Engineer, Code Quality
$159.8k - $235k
About the TeamThe Code Quality team sits... ...sure the signals engineers rely on before shipping... ...'re looking for Software Engineers to help... ...rolloutsExplore how AI agents and tools can... ...matching, 16 weeks of paid parental leave,... ...Covey tool has been reviewed by an independent...
Hourly pay
Work at office
Local area
Remote work
Flexible hours
Doordash
San Francisco, CA
1 day ago
Backend Engineer - AI Coding Agent
$85 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: Backend Engineer (Coding Agent Experience) Type: Contract... ...AI coding agents to complete and evaluate complex engineering tasks. Review model-generated code for correctness...
Full time
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
2 hours ago
Senior Software Engineer - Ruby (LLM Evaluation & Repository Validation)
...GitHub issues and evaluate LLM performance in software projects. Job... ...Senior Software Engineer – LLM Evaluation... ...fastest-growing AI companies accelerating... ...with real code, influencing the... ...tools or automation agents. Perks of Freelancing... ...(no medical/paid leave) Location...
Part time
For contractors
Freelance
Remote work
Turing
San Francisco, CA
2 days ago
Software Engineering Architect - DevXP AI team
$218.4k - $365.2k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive customer... ...a trailblazing engineering leader who is... ..., evangelism, evaluation, and cost... ...like AI code review agents for use... ...AI accelerates software development at... ...health support, paid parental leave,...
Full time
Salesforce
San Francisco, CA
2 days ago
Software Engineering Architect
$218.4k - $365.2k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...change how software is designed,... ...and Context Engineering capabilities.... ...oversight, evaluation, and... ...SDLC, including code understanding... ...creation, code review, debugging, modernization... ...support, paid parental...
Full time
Salesforce
San Francisco, CA
14 hours ago
Software Engineer, Security Agents
...-first enterprise AI company. We build... ...team of researchers, engineers, designers, and... ...OverviewReady to build AI agents that revolutionize... ...for a Senior Software Engineer to create... ...triage, secure code reviews, threat modeling,... ...coaching.6 weeks of paid vacation (30 working...
Full time
Work at office
Local area
Remote work
Home office
Cohere
San Francisco, CA
a month ago
Senior Software Engineer - Backend
$170k - $240k
...we are growing the engineering team and looking... ...and implement new software features to support... ...through design and code reviews to ensure best... ...us: Sigma is the AI Apps and agentic analytics... ...time off you need!Paid bonding time for... ...the purposes of evaluating your candidacy for...
Full time
Work at office
Sigma Computing
San Francisco, CA
4 days ago
Senior Software Engineer, Backend (All Teams)
$159.8k - $235k
...demand, logistics engine for delivery! We're... ..., as well as evaluating and executing on trade... ...stacksProficiency in using AI coding tools (e.g.,... ...) in the full software development lifecycle... ..., 16 weeks of paid parental leave, wellness... ...tool has been reviewed by an independent...
Hourly pay
Work at office
Local area
Remote work
Flexible hours
Doordash
San Francisco, CA
2 days ago
Software Engineer, Agent
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...examples of projects that engineers on our team have worked on recently... ...:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
4 days ago
Software engineer, agents
$146k - $240k
...leading enterprises orchestrate AI-powered work. Our vision is... ...building and deploying AI agents that are grounded in their... ...a highly skilled fullstack software engineer to join our engineering team... ...and participate in rigorous code reviews to uphold quality and maintainability...
Full time
Work at office
Local area
Writer
San Francisco, CA
4 days ago
Software Engineer, Agent - Healthcare
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...been a founder or founding engineer—you know what it means to balance... ...:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
3 days ago
Security Software Engineer - Python & AI Evaluation
$75 - $90 per hour
...States | Canada + 21 more Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security... ...not required Professional code review, coding annotation, LLM/code evaluation...
Temporary work
Remote work
Braintrust
San Francisco, CA
2 days ago
Software Engineer, Agents
$180k - $250k
...product-minded fullstack engineer to build the core... ...experiences that power our AI agents. You’ll work across... ...deployment.Research and evaluate new approaches to... ...practices.Participate in peer reviews, brainstorming... ...in a product-oriented software engineering role.A strong...
Work at office
11x AI
San Francisco, CA
4 days ago
Software Engineer, Agent - Travel & Hospitality
$180k - $390k
...platform for customer-facing AI agents, working with many of the world... ...we have to meet. Every engineer on TTH is also a product-led... ...employees:Flexible (unlimited) paid time offMedical, dental, and... ...job description. We strive to evaluate all applicants consistently without...
Full time
Flexible hours
Sierra
San Francisco, CA
3 days ago
Software Engineer III - ServiceNow
...you to take your software engineering career to the next... ...quality production code and maintains algorithms... ...-authorized AI coding assist tools... ...through peer review, automated testing... ...ability to critically evaluate, validate, and refine... ...compensation, paid in the form of cash...
JP Morgan Chase
San Francisco, CA
3 days ago
Software Engineer, Agent Infrastructure
$230k - $385k
About the TeamThe Agent Infrastructure team at... ...deployment of highly useful AI agents, both... ...AI models to execute code, debug issues, and develop software just as human SWEs... ...the RoleAs a Software Engineer on the Agent... ...birthing parents), plus paid medical and caregiver...
Work at office
Local area
Worldwide
Relocation package
Flexible hours
OpenAI
San Francisco, CA
4 days ago
Software Engineer, Codex Core Agents
$230k - $385k
...TeamThe Codex Core Agent team builds the kernel... ...’re looking for engineers to build the infrastructure... ...safely execute code, interact with... ...developer tooling, and AI, building... ...into a dependable software engineering agent.Bonus... ...birthing parents), plus paid medical and...
Work at office
Local area
Flexible hours
OpenAI
San Francisco, CA
4 days ago
Onboard Software Engineer - Hyperdrive
$155k - $225k
...CASoftware - Onboard Software /Full time /On-... ...robustness and reliability.Review code, provide clear... ...+ years' software engineering experience, using... ...:100% company-paid medical, dental, and... ...wide as we evaluate individuals based... ...connectivity, on-orbit AI, national security...
Full time
Temporary work
Loft Orbital
San Francisco, CA
4 days ago
Software Engineer - Compiler
$170k - $235k
...join a group of engineers dedicated to building... ...stakeholders to review designs and code, driving... ...engineering high-quality software... ...us: Sigma is the AI Apps and agentic... ...time off you need!Paid bonding time for... ...the purposes of evaluating your candidacy for...
Full time
Work at office
Sigma Computing
San Francisco, CA
4 days ago
Software Engineering MTS
$117.2k - $176.7k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...Salesforce engineers use every day... ...maintainable code and raise the... ...through code review.-Partner with... ...professional software engineering... ...assess and evaluate candidates’... ...support, paid parental leave...
Full time
Immediate start
Salesforce
San Francisco, CA
3 days ago
Agentic Ontology Software Engineer
$50 per hour
...team as a Senior AI Engineer focused on... ...source data.Build AI agent workflows that create, evaluate, and improve ontology... ..., and human review.Integrate... ...Build production software across services,... ...Review designs and code, mentor other engineers... ..., 401(k) match, Paid time off,...
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
San Francisco, CA
3 days ago
Software Engineering SMTS - Brain Team (REMOTE)
$148.5k - $223.9k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive customer... ...of Salesforce.Engineering at Qualified... ...agentic software and embedding... ...collaboration, code review, and strong... ...engineering around evaluation, observability,... ...support, paid parental leave,...
Full time
Remote work
Salesforce
San Francisco, CA
13 hours ago
Software Engineering LMTS
$172.5k - $260.1k
...SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...is seeking engineers to architect,... ..., leadership reviews, or Tech... ..., testable code with an eye... ...8+ years of software development experienceExperience... ...assess and evaluate candidates’... ...support, paid parental...
Full time
Salesforce
San Francisco, CA
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineers: Paid Code Review for AI Agent Evaluation. Be the first to apply!