Remote AI Agent Evaluation Specialist
$80 per hourMindrift
Raleigh, NC
- Remote job
A leading tech company is seeking contributors for a flexible part-time opportunity focused on quality assurance for autonomous AI agents. You will analyze complex systems, review tasks for logic, and ensure clear expected behaviors for AI. Ideal candidates possess excellent analytical and detail-oriented skills, with experience in policy evaluation or logic puzzles preferred. Compensation can reach up to $80/hour based on expertise and project needs. This role offers valuable experience in an advanced AI project and fits around your primary commitments. #J-18808-Ljbffr Mindrift
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Remote AI Agent Evaluation Specialist in Raleigh, NC vacancy
$60 per hour
...contributors for a part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and critical... ...with structured data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define expected...Remote jobPart timeFlexible hours- Mindrift connects specialists with project-based AI opportunities for testing, evaluating, and improving AI systems. You’ll design tasks within realistic simulated environments... ...multiple correct solutions. This freelance, remote role requires 5+ years in software development...Remote jobFreelance
$60 per hour
...ethically shape the future of AI. What We Do The Mindrift platform... ...thrive in ambiguity, enjoy remote asynchronous work, and want to... ...modern AI systems are tested and evaluated, we want to hear from you.... ...QA experts for autonomous AI agents in a project focused on validating...Remote workFreelanceFlexible hours- AuraOne, Inc. is seeking a remote AI data reviewer to evaluate agent evaluation outputs against a versioned rubric. You will label edge cases, provide rationales, and help calibrate review quality across long batches. Ideal candidates have prior evaluation or annotation...Remote jobFor contractorsFlexible hours
- ...experienced Spanish Linguistic Experts to join a cutting-edge AI research project. The role focuses on evaluating LLM performance in Spanish tasks and curating high-... ...efforts, collaborating with AI researchers in a distributed, remote team environment. #J-18808-Ljbffr MercorRemote work
- ...Job Description Job Description Role Title: AI Agent Evaluator Role Type: Contractor Location: United States We are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. You'll test two AI agents side by side, assess their...Remote jobFor contractorsWork at office
YO AI Labs
New York, NY10 days ago $217.57k - $271k
...roles — such as field-based sales or other remote‑by‑design positions — may have different... ...ID.me, we embrace the thoughtful use of AI tools in our daily work and there are... ...define and lead the discipline of testing AI agents, evaluating LLM behavior, and ensuring the...Remote workFull timeTemporary workWork at officeLocal areaFlexible hoursID.me
San Francisco, CA2 days ago- ...Founding AI Engineer, Agents & Evaluation New York City Full time On-site Engineering We're building the intelligence layer for all home-based... ...agent system. // The process – Intro call (45 min, remote): Daniel walks you through the role and the reactive-trap...Remote workPermanent employmentFull timeWork at officeNight shift
Zingage
New York, NY2 days ago $60 per hour
A leading AI firm in Austin is looking for QA experts to validate... ...and improve AI systems. This remote, freelance role requires... ...detail. Candidates will review AI evaluation tasks, identify inconsistencies... ...define expected behaviors for agents. Ideal applicants have experience...Remote jobFreelance- YO AI Labs seeks a former U.S. Consular Officer or Passport Specialist to evaluate AI-generated guidance on passport and citizenship matters. This remote, contractor role asks for 20-30 hours per week and leverages your State Department experience to ensure accuracy and...Remote jobFor contractors
- ...Mathematics and Statistics experts to support the training and evaluation of state-of-the-art AI systems. We need subject-matter experts who can apply... ...meaningful errors within their field. Flexible, remote contract work starting ASAP. You will work on challenging...Remote jobContract workImmediate startFlexible hours
$55 per hour
A leading AI innovation firm in Dallas is seeking QAs for autonomous AI agents to ensure the quality of complex systems and scenarios. This flexible, remote project is ideal for those with excellent analytical... ...should be adept at evaluating scenarios and documenting findings...Remote jobFlexible hours- ...connects elite creative and technical talent with leading AI research labs. The company is headquartered in San... ...a AI Safety Practitioner for a contract role with remote work options. The role focuses on evaluating AI-generated responses for safety, accuracy, and policy...Remote jobContract workSummer work
- ...Assistant contractor to support a customer project focused on evaluating tasks for an AI benchmark. You will coordinate schedules, manage... ...administrative skills and attention to detail are essential. This remote role offers fast onboarding and distributed teamwork to deliver...Remote jobFor contractors
- ...SWE Agent Evaluation Specialist is a remote engineering review track for evaluating production code, debugging traces, and developer-facing AI outputs against real-world correctness standards. Reviewers reproduce failures, write the unit test the model should have written...Remote jobHourly payFor contractors10 hours per week
- Flodesk, a remote-first email marketing company, seeks an experienced AI Systems professional to own prompts, agent instructions, quality definitions, and evaluation infrastructure. You’ll connect engineering, product, and language models to improve member experiences...Remote work
$41.6k
...Overview We are sourcing independent Search Engine Evaluation Specialists to provide their expertise for an AI benchmark evaluation project. As AI models... ...PTO do not apply. Engagement Type: Freelance / Independent Contractor Workplace Type: Remote...Remote workHourly payContract workFor contractorsFreelance$20 per hour
...Job Application for Search Engine Evaluation Specialist - Freelance AI Trainer Project at Meridial Back to jobs Search Engine Evaluation Specialist... .../ Independent Contractor Workplace Type: Remote Apply for this job * indicates a required field...Remote workHourly payPart timeFor contractorsFreelanceInvisible Technologies
San Francisco, CA9 days ago- ...Expert Codebase Evaluation Specialist - Coding Agent Review is a remote evaluation track for reviewing codebase evaluation evaluation prompts and responses against... ...team can use to retrain. Why this role matters AI data reviewers help turn codebase evaluation...Remote jobHourly payFor contractors10 hours per week
$150k - $250k
...monetize their email list. We're a remote-first company headquartered in San... ...marketers to grow. Today, our AI helps members create and edit emails... .... You will own Flodesk's prompts, agent instructions, quality definitions, evaluation criteria and continuous-improvement...Remote workImmediate startRelocationFlexible hours- Role Title: Evaluation Specialist/Recent Grad Role Type: Contractor Location: Remote Our team is engaging Evaluation Specialists/Recent Grads to contribute to a high-impact... ...your expertise to help train next-generation AI systems. Your work will shape how models learn,...Remote workFor contractors
$80 per hour
...technology company is seeking a part-time QA contributor for an AI project to validate autonomous agents. Candidates must have strong analytical thinking,... ...expected behaviors for AI agents. This flexible remote opportunity allows you to work on your own schedule while...Remote jobPart timeFlexible hours$45 - $70 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Health Expert, AI Safety and Model Evaluation Type: Contract Compensation: $45–$70/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities...Remote workContract workSummer workWeekday work$65 per hour
...and difficulty Verify the logic and test cases within provided evaluation harnesses Assess whether coding environments effectively measure... ...will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities...Remote workContract workFor contractorsTerac
San Francisco, CA21 days ago- ...Freelance · Remote · North America, LATAM, or India About Turing... ...companies, working with frontier AI labs to accelerate model... ...through high-quality training data, evaluations, and engineering talent.... ...applications, you’ll work with coding agents across real-world repositories...Remote workHourly payContract workFor contractorsFreelanceImmediate start
$260k
# Head of AI Agent SystemsWonderschoolSan FranciscoFull-timeAI tools:OpenClawClaude CodeGBrainUpload... ...for production workflows.* Build evaluation and reporting systems to track agent... ...director machine learning jobs at 18 companies.Remote work: This job is on site. 4 of the 19...Remote workPermanent employmentImmediate startWe Love AI Jobs
San Francisco, CA2 days ago- AI Agent Power User AI Agent Power User Role Type: Contractor Location: Remote (US, CA, UK, IE, AU, NZ) Scope of Work: Run complex, multi-step business scenarios using AI... ...or diverges from realistic business needs. Evaluate and score AI-generated outputs using specified...Remote workContract workFor contractors
$70k - $140k
...Program Evaluation SpecialistWe are seeking driven, thoughtful candidates with experience... ...of State (DOS). As a Program Evaluation Specialist with Censeo, you will support quantitative... ...paid vacation and sick leave, flexible and remote work opportunities, and tuition and...Remote workFull timeWork at officeWork from homeFlexible hoursNight shiftCenseo Consulting Group
Washington DC7 days ago- ...(~15 hours/week) Location: Remote Job Summary We are seeking... ...Software Engineers to support an AI training project by creating... ...learning environments that evaluate AI models on complex software... ...reference solutions. Evaluate AI agents' ability to reason through...Remote jobFor contractors
- ...Job Title: AI Skills and Agents Lead (Low/No code) Start Date: ASAP or Mid-November Duration... ...: Seattle 2-3 days per week or fully remote within the US. Allocation: 100%... ...including prompting, grounding, retrieval, evaluation, structured outputs, tool use, and...Remote workFull timeWork at officeImmediate start2 days per week3 days per week
YunoJuno
Seattle, WA1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Agent Evaluation Specialist. Be the first to apply!
