AI Scenario Testing & Evaluation Lead (Project-Based)
$80 per hourMind Rift
A technology company based in New York is seeking specialists for project-based AI opportunities focusing on testing and improving AI systems. This position requires 5+ years of software development experience, primarily in Python, along with a solid background in full-stack development. Participants will handle realistic coding tasks and AI evaluation challenges, with rates up to $80/hour. The work allows for flexible hours depending on task complexity. #J-18808-Ljbffr Mind Rift
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the AI Scenario Testing & Evaluation Lead (Project-Based) in New York, NY vacancy
$80 per hour
...level professional to design evaluation scenarios for AI agents. Responsibilities include creating structured test cases and analyzing agent behaviors... ...pay rates up to $80/hour based on skills and experience. Contribute to advanced AI projects and enhance your portfolio...ProjectRemote jobHourly payPart timeFlexible hours$100k - $143k
...UnitTechnology, Data, AI and Ventures (... ..., access to leading-edge... ...experienced QA Test Engineer/Lead to... ...collaborate with project stakeholders and... ...cases.Design test scenarios and scripts and... ...quality.Continuously evaluate the... ...eligible: No Actual base salary will be...ProjectLocal area3 days per week$80 per hour
...to ethically shape the future of AI. What We Do The Mindrift platform connects specialists with AI projects from major tech innovators. Our mission... ...design realistic and structured evaluation scenarios for LLM-based agents. You'll create test cases that simulate human-...ProjectPart timeFreelanceRemote workFlexible hours$180k - $230k
...seeking an Enterprise Test & Quality Lead to provide end‑to‑end... ...leads to define test scenarios, test cases, regression... ...release confidence to project and executive stakeholders... ...platforms, especially AI‑driven technologies... ...currently expect to offer a base salary in the range of...ProjectInterim roleShift work$80 per hour
...English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What...ProjectPermanent employmentTemporary work- Toloka AI is assembling a remote, project-based team of strategy consultants to design AI-driven learning environments for enterprise... ...models. You will craft authentic project scenarios, define tasks for AI agents, and build evaluation rubrics to train grading systems. Ideal...ProjectRemote workFlexible hours
$95k - $145k
...CAPA management, and risk-based compliance activities... ...considerations for AI/ML-enabled systems and... ...will be responsible for… Lead end-to-end validation activities... ...systems and automation testing frameworks, including... ...to lead projects or workstreamsAbility to...ProjectRemote workVisa sponsorship- ...Test Lead [Brooklyn, NY] Client: NYCERS PV: Innovee... ...thoroughly and consistently, reduces project risks, and supports timely... ...strategy, test plans, test scenarios, and acceptance criteria.... ...Optimize workflows using AI-assisted testing tools and enhance...Project
Info Way Solutions
New York, NY3 days ago - YO AI Labs is seeking experienced producers across film, television... ...live events to support an AI evaluation project. You will apply production... ...to craft realistic industry scenarios, evaluate AI outputs, and... ...and this is a remote, output-based engagement. You will create budgets...ProjectRemote job
- ...Back Test Lead Other Area(s) Brooklyn , New York W-2 hourly Jul 31,... ...opportunity to work on rewarding projects, consider a career with Cloud... ...strategies, test plans, test scenarios, acceptance criteria, and end... ..., SQL, and applicable AI-assisted testing tools to organize...ProjectHourly payLocal area
Cloud and Things Inc
New York, NY1 day ago $11 - $30.65 per hour
...an Audio Specialist for a freelance AI Trainer project to evaluate and benchmark audio models for customer... ...creating realistic evaluation scenarios across various domains, contributing... ...pay range is $11 to $30.65 per hour based on experience and location. #J-18808...ProjectRemote jobHourly payFreelance- ...Labs Rise Data Labs is a leading AI training and data... ...media tasks and rubrics based on your own professional... ...Create realistic work scenarios based on your own... ...with AI model training, evaluation, or benchmark development... ...to work on innovative projects that directly shape...ProjectFlexible hours
$55 per hour
...connect domain experts with cutting-edge AI projects. We are seeking an Evaluation Scenario Writer - QA for a project focused on... ...agents. This is a flexible, project-based opportunity that blends manual scenario validation, automated test thinking, and collaboration with...ProjectPart timeFreelanceInternshipRemote workFlexible hours$85 per hour
...creative and technical talent with leading AI research labs. Headquartered... ...agents to complete and evaluate complex infrastructure... ...infrastructure engineering scenarios. Work independently and... ...asynchronously to meet sprint-based project deadlines. Qualifications...ProjectContract workSummer workRemote workMercor
New York, NY2 days ago- ...an experienced Monitoring & Testing Lead to establish and oversee an... ...framework, execute risk-based reviews, evaluate control effectiveness, manage... ...transaction monitoring coverage, scenario effectiveness, tuning... ...analytics, automation, and AI‑enabled tools to improve testing...Bank staff
$70 - $110 per hour
...technical talent with leading AI research labs. Headquartered... ...Responsibilities Evaluate AI-generated... ...-generated responses based on financial soundness... .... Review financial scenarios involving cash flow oversight... ...the remainder of the project. Application...ProjectHourly payContract workSummer workWork at officeImmediate startRemote workMercor
New York, NY6 days ago$80 per hour
...is seeking QAs for autonomous AI agents to validate complex task structures and improve evaluation frameworks. The role requires... ...detail. Applicants will review scenarios, identify inconsistencies, and... ...while contributing to advanced AI projects that positively influence...ProjectRemote workFlexible hoursMind Rift
New York, NY3 days ago- A tech company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least...ProjectContract workRemote workFlexible hours
Handshake
New York, NY21 hours ago - A tech company specializing in AI projects is seeking skilled LibreSprite users to assist in evaluating AI-generated visual content. As an independent contractor, you... ...and attention to detail. This ongoing project-based role allows you to engage in the exciting intersection...ProjectFor contractorsRemote work
Handshake
New York, NY21 hours ago - Mercor is partnering with a leading AI research organization to engage experienced UI/UX and product designers for a project focused on evaluating how well AI systems perform real-world digital... ...samples, and providing evidence-based, reproducible judgments #J-18808-Ljbffr...Project
- Mercor is seeking experienced UI/UX and product designers to lead a project evaluating how AI systems perform real-world digital product design tasks.... ...score AI-generated and human samples, apply evidence-based judgment, and iteratively improve your criteria based on...Project
- ...Software is seeking Engagement Managers to lead multiple complex, enterprise-level... ...with Sales to define SOWs, develop detailed project plans, and ensure projects deliver value on... ...products, including opportunities to apply AI-driven improvements across engagements. #J...Project
Redwood Software
New York, NY3 days ago - ...experienced Investment Banking SME to support the development, evaluation, and improvement of advanced AI models in finance. You will assess AI-generated... ...for high-quality outputs. The role is remote, project-based, and demands strong domain expertise across investment...ProjectRemote job
- Volga Partners seeks an Investment Banking SME to support AI model evaluation in finance. You will assess AI-generated analyses for accuracy... ...validate reasoning, and identify errors. This is a remote, project-based role leveraging your deep industry expertise. Ideal...ProjectRemote job
$55 per hour
A leading technology firm is seeking an Evaluation Scenario Writer - QA to ensure the quality of evaluation scenarios for AI projects. This part-time, flexible role involves reviewing tests, spotting inconsistencies, and collaborating on automation. Ideal for those with...ProjectRemote jobPart timeFlexible hours- A virtual AI evaluation firm is seeking individuals to review and evaluate AI-generated responses in therapeutic conversations. The ideal... ...detail for assessing tone and emotional nuance. This is a project-based role with immediate onboarding and a remote work environment...ProjectRemote jobImmediate start
- Mindrift is launching a remote, project-based role for consultants with direct experience at top-tier strategy firms... ...end-to-end learning environments for advanced AI systems and design structured tasks for evaluation by AI agents. No deep technical background is required...ProjectRemote jobHourly pay
- ...Cybersecurity Engineer to contribute to advanced AI evaluation projects. This fully remote hourly contract role focuses on enterprise security scenario design, incident response, and risk... ...frameworks, and hands-on experience with leading tools. #J-18808-Ljbffr SynthiresProjectRemote jobHourly payContract work
- Mindrift is offering a project-based opportunity to craft and evaluate AI coding tasks. You will build a realistic developer environment—a virtual company with... ...infrastructure, tickets, docs, and conversations—to test AI agents. You design tasks from intermediate states,...Project
- ...seeking an experienced Quantitative Finance Subject Matter Expert to support the development, evaluation, and improvement of AI models used in finance. This is a remote, project-based engagement in which you will assess AI-generated financial analyses for accuracy and...ProjectRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Scenario Testing & Evaluation Lead (Project-Based). Be the first to apply!
Related searches
- functional test lead New York, NY
- testing lead New York, NY
- test automation lead New York, NY
- implementation project manager New York, NY
- implementation project manager remote New York, NY
- wounded warrior project New York, NY
- projects work from home New York, NY
- architectural project designer New York, NY
- project finance New York, NY
- special projects New York, NY