Software Engineer for AI Code Evaluation
$25 per hourSaidGig
Role Overview
Software Engineers will apply their real-world development experience to evaluate AI-generated code and technical content, providing structured, practical feedback that helps improve AI behavior on programming tasks, system design, and engineering best practices.
Key Responsibilities- Evaluate AI-generated code and technical explanations for correctness, efficiency, clarity, and alignment with real-world engineering practices.
- Design job-related coding questions and problem prompts that reflect realistic engineering scenarios.
- Provide structured feedback on AI responses, documenting issues and recommended improvements.
- Follow detailed technical guidelines and quality rubrics when reviewing outputs.
- Work independently in an asynchronous environment to meet review and evaluation goals for assigned projects.
- Based in India, with a background in Backend, Frontend, Full-Stack, or Systems engineering.
- At least 2 years of professional software engineering experience, internships excluded.
- Strong coding ability in one or more major languages, such as Python, Java, C++, JavaScript, or Go.
- Experience building, debugging, and maintaining production-level code.
- Ability to work independently, follow detailed instructions, and apply technical guidelines consistently.
- Strong written communication skills and attention to detail.
- Applicants will be required to pass a coding assessment as part of the selection process.
- Employment type: Part time, flexible, hourly contract work that can be done alongside primary employment.
- Remote and asynchronous, work independently from your location in India.
- No minimum weekly hours required, flexible scheduling.
- Ongoing, project-based engagement, with placement dependent on project availability.
- This engagement is invite only, not a traditional full-time software engineering position.
- Up to $25/hr (depending on the project)
- Eligibility: Candidates must be based in India and meet the qualifications listed above.
- Placement into projects depends on availability and successful completion of the selection steps.
- Application steps: create a candidate profile, complete identity verification, complete the required coding assessment, and complete onboarding for any project you are invited to. Once onboarded to a project, begin assigned review work and receive payment for hours worked.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Software Engineer for AI Code Evaluation in United States vacancy
- ...brilliance. Surge was founded by engineers and researchers who dreamed... ...the next generation AI. We're building a platform... ...to conducting rigorous evaluations that go beyond benchmarks.... ...funding. The Role As a Software Engineer, Coding Evaluation & Training Data...SuggestedFull time
$40 per hour
...specialists with project-based AI opportunities for... ..., focused on testing, evaluating, and improving AI... ...dataset to evaluate AI coding agents - how well a model... .... Not prompt engineering. Not writing code from... ...Qualifications ~5+ years in software development. ~Core...SuggestedPart time- ...Role Overview Evaluate and benchmark the coding abilities of frontier AI models by reviewing AI-generated solutions, validating them against real-world software engineering tasks, and shaping high-quality evaluation datasets and benchmarks. This role suits experienced...SuggestedContract workFor contractorsRemote work
$90 per hour
...Role Overview Software Engineers evaluate AI-generated code and technical content, providing structured, experience-based feedback to improve how AI systems understand programming tasks, system design, and engineering best practices. This is an ongoing, project-based,...SuggestedHourly payOngoing contractContract workPart timeInternshipRemote workFlexible hours$90 per hour
...Software Engineers evaluate AI-generated code and technical content, provide structured feedback, and help improve how AI understands programming tasks, system design, and engineering best practices. Role Overview This is an ongoing, project-based, hourly contract role...SuggestedHourly payOngoing contractFull timeContract workPart timeInternshipRemote workFlexible hours$50 per hour
...Role Overview Evaluate, benchmark, and help improve the coding capabilities of advanced AI models by assessing AI-generated solutions against real software engineering problems. You will review code for correctness and quality, reproduce and debug issues, design evaluation...Contract workFor contractorsRemote work$25 per hour
...Role Overview Software Engineers based in India will evaluate AI-generated code and technical content, provide structured feedback, and help improve how AI models understand programming tasks, system design, and engineering best practices. This is a part-time, project...Hourly payPart timeFor contractorsInternshipImmediate startRemote workFlexible hours- ...Senior Software Engineer — AI Coding Evaluator is a remote engineering review track for evaluating production code, debugging traces, and developer-facing AI outputs against real-world correctness standards. Reviewers reproduce failures, write the unit test the model should...Remote jobHourly payFor contractors10 hours per week
$60 - $100 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Software Engineering, Data Science, and Systems Design... ...Role Responsibilities Evaluate LLM-generated responses to coding and software engineering queries...Full timeContract workSummer workRemote work$175k - $215k
...state-of-the-art Generative AI to create a training ground for... ...Waymo Driver. The Simulator Evaluation team faces the ultimate data... ...real"? We are looking for a Software Engineer to build the metrics and pipelines... ...and AI, writing the code that processes petabytes of simulation...Full timeRemote work$320k
...interpretable, and steerable AI systems. We want AI to... ...committed researchers, engineers, policy experts, and... ...looking for a Software Engineer to help us build... ...and maintain new agentic coding tools for developers. The... ...experimenting with new tools, evaluating emerging techniques,...Full timeWork experience placementWork at officeVisa sponsorshipFlexible hours$170k - $216k
...the Waymo Driver. Our software allows the Waymo Driver... ...sensors, enabling software engineers like you to develop... ...an automated system Evaluate new hardware specifications... ...in industrial AI applications involving... ...with robust and efficient code We prefer: MS...Full timeRemote work$170k - $216k
...the Waymo Driver. Our software allows the Waymo Driver... ...sensors, enabling software engineers like you to develop... ...critical automation and evaluation frameworks that establish... ...in industrial AI applications involving... ...with robust and efficient code We Prefer MS or...Full timeRemote work- ...organize human intelligence to power the AI economy. We partner with leading AI... ...context that can't be captured in code alone. Today, more than 30,000 experts... ...About the Role As a Senior Software Engineer (AI Data & Evaluation) at Mercor, you will be at the core of...Full timeWork at officeRelocation package
$144.7k - $221.4k
...About the Organization The Evaluation team builds and evolves... ...clear feedback for engineering and leadership, and help... ...autonomous driving software performance at interfaces... ...thoughtful system design, code reviews, testing,... ...Experience leveraging AI-assisted development and...Full timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours$129.4k - $198.4k
...About the Organization:The Evaluation team builds and evolves... ...for autonomous vehicle software validation. Develop and... ...insights to engineering teams and leadership, including... ...high standards for code quality and software architecture... ...practices. Leverage AI-assisted development...Full timeLocal areaRemote workWork from homeFlexible hours$147k - $211k
...product or system development code.Participate in, or lead... ...years of experience with software development in one or more... ...of labeling, ratings, and evaluations.Experience with Natural Language... ...Learning, or Generative AI.Google's software engineers develop the next-...- AI Systems Modernization DeveloperBentley Systems... ...dedicated expert team in code modernization. This... ...support and guide other software developers in the company... ...: Manual evaluation of the quality of the conversion... ...solutions for architecture, engineering, and construction -...Worldwide
$224k - $356.5k
...autonomous driving, and evaluation is how we know the drive... ...our organization develops AI drivers!We are looking for a senior engineer to own the engine that... ...end by this person, in code and in the room, not from... ...experience).12+ years building software, with significant time...Full timeRemote work$80 - $100 per hour
...employment verification for this role. What You'll Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate frontier models on real-world...Remote jobFull timeContract workFor contractors$405k
...interpretable, and steerable AI systems. We want AI to... ...committed researchers, engineers, policy experts, and... ...re looking for a Staff Software Engineer to set... ...research on the Claude Code team. In this role, you... ...systems, tooling, and evaluation infrastructure that determine...Full timeWork at officeVisa sponsorshipFlexible hours$50 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Software Engineering Expert Type: Contract... ...Role Responsibilities Write and evaluate code across diverse programming languages...Full timeContract workSummer workRemote work- ...Software Engineer, Agent Evaluation and Quality Engineering · Full-time · San Francisco; New York Our mission is to automate coding. The first step in our journey is to build the best tool for professional... ...and building best-in-class AI evaluation system: curated...Full timeWork at office
$50 - $150 per hour
A fast-growing AI company in New York is seeking a contractor for a software engineering role focused on improving LLM performance. Responsibilities... ...and reviewing model-generated code. Ideal candidates will have... ...scalable applications and evaluating code quality. The position...Hourly payFor contractors10 hours per weekFlexible hours- A leading AI research accelerator in San Francisco is looking for an experienced Software Engineer to evaluate AI-generated code for efficiency and reliability. You will work closely with cross-functional teams to develop innovative coding solutions and design verification...Remote jobFor contractors10 hours per weekFlexible hours
- ...Job Description Join Code for Good, a social impact hackathon... ...and be considered for a 2027 Software Engineer Program Summer Internship.... ...effectively prompting approved AI-assisted development tools to... ...and validation. Critically evaluate AI-generated outputs and...Full timeSummer workInternshipSummer internshipVisa sponsorship
$70 - $120 per hour
...technical talent with leading AI research labs. Headquartered... ...Jack Dorsey . Position: Software Engineer Type: Freelance... ...multiple categories, such as code review, debugging, and error... ...Collaborate with AI labs to enhance evaluation datasets for AI reasoning and...Full timeSummer workFreelanceImmediate start$35 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Code-Data Eval Author — Software Engineer Type: Contract... ...quality through structured review. Evaluate the accuracy and depth of AI-generated...Full timeContract workSummer workRemote work- ...Senior Software Engineer, AI Code Modernization Location: Hybrid – Exton, PA/Philadelphia Position Summary Bentley is building... ...accelerate the conversion/test/correction process. Evaluate and improve AI systems for accuracy, reliability, performance...Worldwide
- ...development of high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate and author reference code, evaluate and refine AI-generated solutions across multiple programming...Full timeFor contractorsRemote work10 hours per weekFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer for AI Code Evaluation. Be the first to apply!
Related searches
- agile software developer United States
- software developer internship no experience United States
- intermediate software engineer United States
- software engineer staff United States
- experienced software developer United States
- software engineer co-op United States
- work from home software developer United States
- software developer no experience United States
- software developer fintech United States
- software data engineer United States




