AI Model Evaluation Specialist
Temporary
Rise Data Labs
Key Responsibilities
• Design and implement evaluation metrics for AI models
• Conduct thorough testing of model performance across different scenarios
• Analyze model outputs for bias, fairness, and accuracy
• Collaborate with ML engineers to implement improvements
• Document findings and recommendations
Ideal Candidate:
• PhD or Masters in Computer Science, ML, or related field
• 5+ years experience in AI/ML model evaluation
• Strong background in statistical analysis
• Experience with evaluation frameworks and metrics
• Excellent communication skills
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the AI Model Evaluation Specialist in New York, NY vacancy
- Cincinnatus LLC is seeking behavioral health experts to help evaluate how AI models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role focuses on neutrality, safety and balanced guidance, with 20-40 hours per week and a W-...SuggestedPart time
$110 per hour
...background and interests. Role Overview Qualified physicians may contribute to AI research projects by bringing real-world clinical expertise to model development and evaluation. Key Responsibilities Train and evaluate AI models in medicine. Create tasks and...SuggestedHourly payContract workRemote workSaidGig
New York, NY2 days ago- ...Junior Data Scientist to join a remote contract role supporting an AI lab partnership. You will apply rigorous statistics, ML methods, experimentation design, and quantitative reasoning to evaluate model outputs and drive model quality. You will assess data science...SuggestedRemote jobContract work
- AI Trainer Jobs is seeking biologists to join an AI training project. You will review AI-generated responses, evaluate reasoning quality and provide expert feedback to ensure accuracy, clarity... ...AI performance, and ensuring model integrity by fact-checking biology information...Suggested
$60 - $90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract Compensation: $60–...SuggestedContract workSummer workRemote workMercor
New York, NYa month ago$50 - $60 per hour
...is seeking an Investment Partner to join their remote team in the United States. This role involves training AI models, measuring their progress, and evaluating outputs to enhance their quality. Ideal candidates should possess expert-level financial reasoning and proficiency...Remote jobHourly payFlexible hours$45 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: Behavioral Health Expert, AI Safety and Model Evaluation Type: Contract Compensation: $45–$70/hour Location...Contract workSummer workRemote workWeekday workMercor
New York, NY15 days ago- ...ready deliverable should include Grade AI-generated deliverables (often Word... ...part in calibration sessions with other evaluators to align scoring standards Preferred Qualifications... ...skills Prior experience with AI model training, evaluation, or benchmark development...Temporary workFlexible hours
Rise Data Labs
New York, NY5 days ago $85 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms...Contract workSummer workRemote workMercor
New York, NY6 days ago- Rise Data Labs is seeking advanced Mathematics and Statistics experts to support the training and evaluation of state-of-the-art AI systems. We need subject-matter experts who can apply deep quantitative knowledge to AI evaluation problems, assess AI-generated reasoning...Remote jobContract workImmediate startFlexible hours
- micro1 is seeking a Secretary & Admin Assistant contractor to support a customer project focused on evaluating tasks for an AI benchmark. You will coordinate schedules, manage correspondence, and handle records while contributing to high-quality inputs that train AI systems...Remote jobFor contractors
$130k - $160k
...PropositionAs Lead, Operating Model Design, you will serve as an internal... ...for Global Technology’s AI-First operating model. You... ...from delivery leaders. This is a specialist individual-contributor role: your... ...incorporate field feedback, evaluate model effectiveness, and align...Full timeTemporary workWork experience placementWork at officeLocal areaRelocation package3 days per weekMetropolitan Life Insurance Company
New York, NY2 days ago$295k
...we?Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve... ...Montreal, Paris, Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling...Full timeWork at officeLocal areaRemote workHome officeCohere
New York, NY2 days ago- ...a detail-oriented contractor to review Australian voice actor evaluation prompts and responses against our quality rubric. You will label... ...severities, and provide structured feedback to help retrain the model. This remote, US-eligible role requires careful written...Remote jobFor contractors10 hours per weekFlexible hours
$147.5k - $211k
...per week Business UnitTechnology, Data, AI and Ventures (TDAV)Within the Tech, Data,... ...Corporate Vice President, Data Scientist - Model Validation and AI Governance will play a... ...independently challenging model methodologies, evaluation approaches, controls, and monitoring...Local area3 days per weekNew York Life Insurance Company
New York, NY4 days ago$255k - $300k
Who We AreNotion is the collaborative AI workspace where teams and agents think together... ...before ChatGPT launched. The job of the Model Capabilities team is to keep us there. We... ...what "better" means, build or use the evaluation to test it, and are willing to be proven...Local areaNotion Labs
New York, NY1 day ago$60 - $80 per hour
...Role Overview Selected experts may support leading AI labs and companies by helping improve AI systems in chemistry. Opportunities... ...become available. Key Responsibilities Train and evaluate AI models in chemistry. Create tasks and deliverables based on real-world...Hourly payContract workRemote workSaidGig
New York, NY2 days ago$30 - $80 per hour
...video production expertise to help train next-generation AI systems through high-quality evaluation, feedback, and curated media inputs. This remote... ...video, animation, and instructional material. Identify model weaknesses, edge cases, and opportunities to improve visual...Hourly payFor contractorsRemote workSaidGig
New York, NY1 day ago$207k - $275k
...CoreWeave is The Essential Cloud for AI™. Built for pioneers by... ...market-maker. As a Solution Specialist for AI Runtime Services, you... ...high-throughput, low-latency model serving through our Inference... ...multi-stakeholder technical evaluations without losing momentum. Why...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hoursCoreWeave
New York, NY25 days ago$100k - $140k
...to the Board and protecting the Firm from exposure to losses as a result of credit, market, liquidity, operational, model and other risks.Associate - AI Governance & ReportingPosition SummaryThe Associate will support the delivery of AI-enabled solutions across Model Risk...Temporary workMorgan Stanley
New York, NY1 day ago- ...Corporate Vice President, Model Validation & AI Governance About the Company A regulated insurer focused on model validation and responsible... ...validation of AI solutions, particularly focusing on the evaluation of agentic and multi-step AI systems, and establishing...
- AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses against a evolving quality rubric. You will compare model outputs, label edge cases, and provide structured feedback for retraining. Responsibilities include evaluating...Remote jobFor contractors
$50 per hour
...your German linguistic and literary expertise to help evaluate and improve advanced language models. You will assess model-generated German content, curate... ...quality linguistic data, and provide feedback that supports AI research and language-model development. Key...Hourly payRemote work- ...A leading construction firm is seeking a motivated BIM Specialist to join their growing team. This remote position requires 3-5 years of... ...Revit, and proficiency in CAD software. The specialist will manage model health, coordinate with project teams, and carry out quality...Immediate startRemote work
GBA
New York, NY2 days ago - We’re seeking a future team member for the role of SVP - Model Risk Management AI, Wealth and Investment to join our Model Risk team. This role is located in New York City, New York. In this role, you’ll make an impact in the following ways:Lead the Model Risk Management...WorldwideFlexible hours
The Bank of New York Mellon
New York, NY3 days ago $90.58k - $114.59k
Position InformationNY HELP: YesAgency: Mental Health, Office ofTitle: Mental Hygiene Program Evaluation Specialist 3 (NY HELPS), Kingsboro Psychiatric Center, P27987Occupational Category: Health Care, Human/Social ServicesSalary Grade: 23Bargaining Unit: PS&T - Professional...Permanent employmentFull timeWork at officeRemote workState of New York, USA
New York, NY1 day ago$197.3k - $225.1k
AI Engineer 4 (Vision model (VLM) customization experience) At Capital One, we are creating responsible and reliable AI systems, changing banking... ...-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability, etc....Full timePart timeLocal areaCapital One
New York, NY1 day ago- ...Overview We are looking for Generalists with strong analytical and critical-thinking skills to support AI and data-focused projects. The role involves reviewing, evaluating, and annotating information across different topics while maintaining high accuracy and quality....Remote jobFor contractors
YO AI Labs
New York, NY6 days ago $118.8k - $269.8k
Responsibilities About the Team JV AI & Systems Integrity is the independent function... ...Role The Technical Product Manager for AI Model Assurance owns the roadmap for how our... ...intake, approval workflow, change detection, evaluation dossier tooling, dashboards, and audit...Temporary workLocal areaShift work$65 - $75 per unit
...Apply analytical chemistry expertise to evaluate how AI systems handle technical requests... ...requests that could enable harm, ensuring models provide useful answers when appropriate... ...rationales for each judgment that non-specialists can understand. Work with scenarios involving...Remote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Model Evaluation Specialist. Be the first to apply!
