Software Engineer - AI Evaluation
Virtual Vocations Inc
Evaluating and benchmarking AI-generated code, the full-time Software Engineer - AI Evaluation will assess correctness, efficiency, and maintainability of coding solutions while working remotely on a contractor basis. Key responsibilities Review and evaluate AI-generated code for correctness and adherence to requirements Analyze software engineering tasks and validate proposed solutions against expected outcomes Create and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks Required qualifications Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field 3+ years of professional software engineering experience Strong proficiency in two or more programming languages such as Python, Java, or C/C++ Experience performing code reviews and evaluating code quality in large-scale codebases Familiarity with version control systems and modern software development workflows
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Software Engineer - AI Evaluation in United States vacancy
$175k - $245k
...Senior Software Engineer II - Applied AI and Evaluations (Remote Eligible) -REMOTE, USA- For over 20 years, Smartsheet has helped people and teams achieve–well, anything. From seamless work management to smart, scalable solutions, we’ve always worked with flow. We’re building...SuggestedFull timeTemporary workLocal areaImmediate startRemote workSmartsheet
New York, NY3 days ago$50 - $150 per hour
A leading AI company is seeking a software engineer to review and evaluate model-generated code. This contract role requires several years of software engineering experience, particularly as a full-stack engineer at notable tech firms. You will assess code quality and...SuggestedHourly payContract workFlexible hours- ...© 2025 Repovive, Inc. All rights reserved. Back to Jobs Apply Now Compensation Not listed Posted April 25, 2026 Required Skills AI evaluation data pipelines agent instrumentation Requirements Mid/Senior Visa Sponsorship Not mentioned Relocation Not mentioned About the Role...SuggestedRelocationVisa sponsorship
Repovive, Inc.
New York, NY4 days ago - Senior Software Engineer - Siri Agentic Evaluation Platform Cambridge, England, United Kingdom • Machine Learning and AI Siri is the world’s most popular intelligent assistant. Available on iPhone, iPad, Mac, Apple Watch, HomePod, AppleTV and Vision Pro, hundreds of millions...Suggested
$147k - $211k
Software Engineer, Search Evaluation Platforms corporate_fare Google place Mountain View, CA, USA Apply Qualifications Bachelor’s degree or equivalent... ...that meets standards, applying artificial intelligence (AI) tools in the development flow, and ensuring your work is...SuggestedTemporary work$120k - $250k
...2016 in Silicon Valley, Pony.ai has quickly become a global leader... ..., and multi-dimensional evaluation. Design and implement high-performance... ...and optimize downstream engineering workflows for Large Language... ...skills in C/C++, Python, and software design Strong foundation in...Temporary work$148k - $356.5k
Senior Software Engineer, Metrics and Evaluation - Autonomous Vehicles page is loaded Senior Software Engineer, Metrics and Evaluation - Autonomous Vehicles... ...to tackle challenges no one else can solve. Our work in AI and digital twins is transforming the world's largest...Full timeRemote work$144.7k - $221.4k
...introspect autonomous driving software performance at interfaces... ...autonomy developers and systems engineers. Design and implement analysis... ...autonomy stack, including evaluation of perception, prediction, and... ...systems. Experience leveraging AI‑assisted development and...Local areaRemote workRelocationRelocation packageFlexible hours- Software Engineer (Model Evaluation & Benchmarking) About the Role We are hiring Engineers focused on AI Model Evaluation to build the systems that ensure multimodal AI behaves reliably, consistently, and predictably as it moves from research into production. This position...
$147.4k - $272.1k
Cupertino, California, United States Software and Services At Apple, great... ...to the next revolution in AI and Machine Learning. We're a dynamic team seeking engineers who are enthusiastic about influencing... ...signals, and advance the evaluation pipeline to the next level. Continuously...Relocation- Turing is searching for a Software Engineering evaluator based in Seattle, United States. You will work on AI model training by curating code examples and evaluating AI-generated code, ensuring efficiency and reliability. This role requires strong full-stack application...Remote jobFor contractors10 hours per weekFlexible hours
- Feedinkoo is looking for experienced software engineers to contribute their expertise to AI evaluation and improvement projects. This role involves applying software engineering skills to assess AI systems with no prior AI experience required, as training will be provided...Remote jobFreelance
- ...SupportFinity™ is seeking a Web Platform Engineer to train AI models in South Dakota. This role entails coding challenges for AI chatbots, evaluating their outputs for quality, and requires proficiency in programming languages like Python or JavaScript. As a flexible...Hourly payRemote workFlexible hours
SupportFinity
Sioux Falls, SD3 days ago - ...inventive research, design, and engineering. Our organization is very... ...shipping code. About the Role As a Software Engineer on the Agent Quality... ...’ll build the measurement, evaluation, and feedback-loop... ...Designing and building best-in‑class AI evaluation system: curated...
- An innovative AI company based in the US is seeking a Mid-Senior level developer. The role involves developing and maintaining evaluation servers, implementing logic for agent actions, and creating testing tools. Candidates should have 4+ years of Python experience, solid...Part timeRemote work
Mind Rift
Houston, TX5 days ago - A cybersecurity technology company is seeking experienced professionals to evaluate AI-generated security content and solve technical cybersecurity problems. This role offers a flexible schedule, with options for full-time or part-time remote work. Candidates should have...Full timePart timeRemote workFlexible hours
DataAnnotation
Louisiana, MO4 days ago - A cybersecurity consultancy is seeking experienced individuals to enhance AI capabilities by evaluating cybersecurity content and solving security-related challenges. You will play a pivotal role in validating AI outputs and providing critical feedback to advance cybersecurity...Remote workFlexible hours
DataAnnotation
Phoenix, AZ10 hours ago $40 per hour
A leading AI security firm is seeking experienced cybersecurity professionals in the United States to evaluate AI-generated security content and solve technical problems. The role offers flexible scheduling, the opportunity to work on varied projects, and hourly pay starting...Hourly payFull timePart timeRemote workFlexible hoursDataAnnotation
Kansas City, MO1 day ago$40 per hour
A cybersecurity technology company is seeking experienced professionals to evaluate AI-generated content and solve technical problems. In this remote role, candidates will work with AI systems to enhance their reasoning about real-world threats. Required qualifications...Hourly payRemote workDataAnnotation
Lansing, MI4 days ago$40 per hour
A cybersecurity company is seeking experienced cybersecurity professionals to join their team. You will evaluate AI-generated security content, solve technical problems, and provide critical feedback to enhance AI systems. This role is remote, flexible, and offers hourly...Hourly payRemote workFlexible hoursDataAnnotation
Oregon, WI4 days ago$40 per hour
A cybersecurity solutions company is seeking experienced professionals to evaluate AI-generated security content and solve technical security problems. Candidates should have over 2 years of hands-on experience in cybersecurity and coding skills, with strong writing and...Hourly payRemote workFlexible hoursDataAnnotation
Saint Paul, MN4 days ago$40 per hour
A cybersecurity-focused company is seeking experienced professionals to evaluate AI-generated security content and improve AI systems. Responsibilities include assessing threats and providing technical feedback, ideal for those with 2+ years in cybersecurity roles like...Hourly payRemote workFlexible hoursDataAnnotation
Springfield, IL1 day ago$40 per hour
A leader in AI training for cybersecurity is seeking experienced cybersecurity professionals to evaluate AI-generated content and solve technical problems. This role offers full-time or part-time remote work with the flexibility to choose projects and work hours. Candidates...Hourly payFull timePart timeRemote workDataAnnotation
Madison, WI10 hours ago$40 per hour
...cybersecurity professionals to join our team to help train AI models. In this role, you will evaluate AI-generated security content, solve technical... ...penetration testing, red teaming, incident response, detection engineering, DFIR, malware analysis, threat intelligence, or...Hourly payFull timePart timeRemote workDataAnnotation
New York, NY10 hours ago$40 per hour
A cybersecurity-focused technology firm is seeking experienced professionals to evaluate AI-generated security content and solve technical problems. This remote position offers flexible hours and the ability to choose projects, paying $40+ per hour. Candidates should have...Hourly payRemote workFlexible hoursDataAnnotation
Virginia, MN10 hours ago$40 per hour
A cybersecurity firm is seeking experienced professionals to evaluate AI-generated security content and solve technical problems. This role is flexible, allowing you to choose projects and work on your own schedule. Candidates should have over 2 years of hands-on cybersecurity...Hourly payRemote workFlexible hoursDataAnnotation
Honolulu, HI1 day ago$130k - $220k
...opportunity summary by the Joinrs AI : The selection process... ...insights company. They help engineers, enterprises, investors, media... ...is best described as an AI Evaluation Engineer / Technical Generalist. It is not a traditional software engineering role and not a pure...Full timeWorldwideAurora Jobs ApS
San Francisco, CA8 hours ago$40 per hour
A cybersecurity firm in the United States is seeking experienced professionals to evaluate AI-generated security content and solve technical problems. You'll work directly with AI models to enhance their accuracy and improve cybersecurity tools. Ideal candidates have 2+...Hourly payRemote workDataAnnotation
Charleston, WV4 days ago$40 per hour
A leading tech firm is seeking experienced cybersecurity professionals to evaluate AI-generated content and solve technical problems. In this remote role, you will work to enhance AI systems, requiring 2+ years of hands-on experience in cybersecurity and some coding knowledge...Hourly payRemote workFlexible hoursDataAnnotation
Iowa, LA4 days ago$40 per hour
A technology solutions company is seeking a Web Application Developer to improve AI models by evaluating coding outputs and performance. Candidates should be proficient in Python or JavaScript and have experience with algorithms and debugging. This remote position allows...Hourly payRemote workFlexible hoursDataAnnotation
Providence, RI4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer - AI Evaluation. Be the first to apply!
Related searches
- software sales engineer United States
- software engineer amazon United States
- oracle software engineer United States
- software engineer student United States
- agile software developer United States
- rust software engineer United States
- software developer positions United States
- senior software design engineer United States
- software developer United States
- ngo software engineer United States