Remote: AI Agent Evaluation Scenario Designer
Mind Rift
New York, NY
- Remote job
A forward-thinking technology company is looking for a detail-oriented individual to design structured evaluation scenarios for AI agents. This entry-level part-time role allows you to contribute remotely, creating test cases for LLM-based agents. The ideal candidate will have a background in Computer Science and some experience with software testing. Flexibility is key as you engage with innovative AI projects while working at your own pace. #J-18808-Ljbffr Mind Rift
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote: AI Agent Evaluation Scenario Designer in New York, NY vacancy
- ...thinking technology company is looking for a detail-oriented individual to design structured evaluation scenarios for AI agents. This entry-level part-time role allows you to contribute remotely, creating test cases for LLM-based agents. The ideal candidate will have a...Remote jobPart time
$80 per hour
A leading AI consultancy is seeking a detail-oriented individual to design evaluation scenarios for LLM-based agents. This part-time, remote role allows you to create structured test cases while working flexibly around your commitments. The ideal candidate holds a relevant...Remote workPart time$80 per hour
...ethically shape the future of AI. What We Do The Mindrift platform... ...re looking for someone who can design realistic and structured evaluation scenarios for LLM-based agents. You'll create test cases that... ...Take part in a flexible, remote, freelance project that fits around...Remote workPart timeFreelanceFlexible hours$80 per hour
A technology firm is seeking QAs for autonomous AI agents to validate and improve task structures within a new project. Candidates... ...thinkers with strong attention to detail and experience in evaluating scenarios. This flexible, project-based role offers competitive pay...Remote jobFlexible hours$60 per hour
...shape the future of AI. What We Do The... ...ambiguity, enjoy remote asynchronous work,... ...systems are tested and evaluated, we want to hear... ...for autonomous AI agents in a project... ...evaluation tasks and scenarios for logic, completeness... ...scenario design. Background in consulting...Remote workFreelanceFlexible hours$80 per hour
...-time opportunity focused on quality assurance for autonomous AI agents. You will analyze complex systems, review tasks for logic, and... ...analytical and detail-oriented skills, with experience in policy evaluation or logic puzzles preferred. Compensation can reach up to $80/...Remote workPart timeFlexible hoursMind Rift
United States4 days ago$150k - $250k
...entrepreneurs sell online and design emails that people... ...list. We’re a remote-first company... ...to grow. Today, our AI helps members... ...Flodesk's prompts, agent instructions, quality definitions, evaluation criteria and continuous... ...datasets, behavioral scenarios, scoring rubrics,...Remote workFull timeImmediate startRelocationFlexible hours$50 per hour
...Prolific in Portland, Oregon is seeking Product Designers and UX Specialists to join their Expert Network for training AI models. Ideal candidates should hold a degree... ...relevant experience. Responsibilities include evaluating AI-generated designs, auditing accessibility,...Remote workHourly payWork from homeFlexible hoursProlific
Portland, OR1 day ago$80 - $100 per hour
...train and improve next-generation AI coding agents. In this contractor role you will... ...feature development. Critically evaluate AI agents in diverse scenarios, identifying strengths, limitations... ...type: Contractor. Location: Remote. Work will involve contributing...Remote workHourly payFor contractorsSaidGig
United States1 day ago$30 per hour
...Prolific Prolific is not just another player in the AI space – we are building the biggest pool of... ...We are seeking Professional Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image...Remote workWork from homeFlexible hoursProlific
United States4 days ago$60 per hour
...contributors for a part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and critical... ...with structured data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define expected...Remote jobPart timeFlexible hours$50 per hour
...Prolific is seeking Product Designers and UX Specialists to join their Expert Network. The role involves training and evaluating AI models using real design expertise, with opportunities... ...skills. Flexible hours and remote work are offered for this role. #J-1880...Remote workHourly payFlexible hoursProlific
Jacksonville, FL2 days ago$80 - $150 per hour
...AI Training Scenario Designer is a remote review track for evaluating AI outputs across ai training scenario designer creative review workflows. Reviewers grade craft, constraints, and tooling adherence; flag production-readiness issues; and document the corrected approach...Remote jobFor contractors10 hours per week$60 per hour
A leading AI firm in Austin is looking for QA experts to validate... ...and improve AI systems. This remote, freelance role requires... ...detail. Candidates will review AI evaluation tasks, identify inconsistencies... ...define expected behaviors for agents. Ideal applicants have experience...Remote jobFreelance$20 per hour
A leading AI technology firm is seeking a Graphic Design Expert to evaluate graphic design elements for AI model training. This part-time contract role offers remote work flexibility and requires expertise in design principles. Ideal candidates should have formal training...Remote jobContract workPart time- ...Freelance · Remote · North America, LATAM, or India... ...working with frontier AI labs to accelerate model... ...quality training data, evaluations, and engineering talent... ...you’ll work with coding agents across real-world repositories... ...data, rubric design, or post-training is a...Remote workHourly payTemporary workFor contractorsFreelanceImmediate start
Turing
Remote12 days ago - ...based in London, UK, Recraft is an AI tool for professional designers, illustrators, and marketers, setting... ...applying our model to real creative scenarios to uncover where it works, where it... ...and early-stage concept ideation Evaluate generated outputs to identify quality...Remote workFull timeContract workFreelanceWorldwideFlexible hours
ReCraft Home Remodeling
United States1 day ago - ...companies, working with frontier AI labs to accelerate model... ...through high-quality training data, evaluations, and engineering talent. We... ...used to develop and evaluate AI agents. You will be assigned to one... ...cutting-edge AI systems. Fully remote opportunity. Collaborate...Remote workFull timeContract workTemporary workFlexible hours
Turing
Remote12 days ago $217.57k - $271k
...— such as field-based sales or other remote-by-design positions — may have different work arrangements... ....me, we embrace the thoughtful use of AI tools in our daily work and there are... ...and lead the discipline of testing AI agents, evaluating LLM behavior, and ensuring the...Remote workFull timeTemporary workWork at officeLocal areaFlexible hours- ...week) Location: Remote Job Summary We... ...to support an AI training project by... ...environments that evaluate AI models on complex... ...tools. You will design reproducible... ...solutions. Evaluate AI agents' ability to reason... ...engineering scenarios. Ensure tasks accurately...Remote jobFor contractors
- Mercor is seeking a UI/UX Design Expert for a remote contract role. The candidate will design precise grading criteria for real-world digital product UX deliverables and evaluate AI-generated and human work samples against robust criteria. Responsibilities include applying...Remote jobContract work
- YO AI Labs is seeking a Creative & Design Expert to support AI training and evaluation projects remotely. You will apply your design, multimedia, and tool expertise to assess AI outputs and provide high‑quality annotations. Ideal candidates have 2+ years in graphic design...Remote job
- Prolific is seeking Product Designers and UX Specialists to join our Expert Network, contributing to the training and evaluation of cutting-edge AI models. This role requires expertise in usability, design systems, and user research, providing essential feedback on design...Remote jobWork from homeFlexible hours
- Prolific is seeking talented Product Designers and UX Specialists to join our Expert Network for training and evaluation of cutting-edge AI models. Candidates should have a relevant educational background and at least one year of experience in design-related fields. Responsibilities...Remote jobWork from homeFlexible hours
- YO AI Labs is seeking a Product Designer to support a customer project focused on design... ...This contractor role is remote, and no prior AI experience... ...design decisions, evaluate trade-offs, create design... ...principles to real-world scenarios. #J-18808-Ljbffr YO AI LabsRemote jobFor contractors
- ...AI Agent Interaction Specialist - Dialogue & Conversation Evaluation is a remote evaluation track for reviewing ai agent interaction evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind...Remote jobHourly payFor contractors10 hours per week
$197k - $232k
...looking for a Senior Product Designer, Agent Experience based in United States... ...on shaping experiences for AI-powered agents and the people... ...interactions. Working in a remote U.S. environment, you will collaborate... ...your personal data to evaluate your candidacy and share...Remote workFull time$105k - $115k
...Senior UX Designer Caseware is one of Canada's original... ...of design craft and AI-assisted making. At Caseware... ...Location: This is a remote location open to... ...everyday design practice, evaluating new tools, sharing what... ...prototypes to model scenarios rapidly and validate assumptions...Remote workPermanent employmentFull timeFlexible hoursCaseWare
United States4 days ago- ...Powered by multimodal AI and a unique practice-based... ...of Mursion simulation design. This is not a... .... Because we work as a remote, distributed team in a... ...interaction dynamics, and scenario structures that reflect... ...guidance that support valid evaluation of participant performance...Remote workFull timeLive inFlexible hours
- ...Experience Design And Research Lead We are seeking... ...the iterative design, evaluation, and refinement of user... ..., in a flexible hybrid remote. The primary work location... ...architecture models, scenario/task flows, wireframes,... ...Experience in working in AI enablement solution...Remote workFull timeContract workMonday to FridayFlexible hours
Mindlance
Boston, MA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote: AI Agent Evaluation Scenario Designer. Be the first to apply!
Related searches