Human-Centered AI Evaluation Scientist
Contra
Contra Labs in New York City seeks a researcher to transform ambiguous client questions into rigorous projects across evaluation, post-training data, benchmarking, and human-centered research. You will own end-to-end engagements, design studies, run analyses in Python/SQL, and generate client-ready insights, reports, and publications while balancing rigor with startup speed and pragmatism. #J-18808-Ljbffr Contra
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Human-Centered AI Evaluation Scientist in New York, NY vacancy
$229.9k - $262.4k
AI Engineer 5 (Gen AI SDK: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible... ...AI & ML are bringing humanity and simplicity to banking... ...(IFX) team is at the center of bringing our vision... ...engineers, research scientists, technical program managers...SuggestedFull timePart timeLocal area$197.3k - $225.1k
AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability) At Capital One, we are creating... ...AI & ML are bringing humanity and simplicity to... ...Experiences (IFX) team is at the center of bringing our vision... ...of engineers, research scientists, technical program...SuggestedFull timePart timeLocal area- ...Position Title: Senior Computer Programmer – AI Evaluation Specialist Location: Remote (... ...will partner with product managers, data scientists, AI engineers, and security and privacy... ..., model-as-a-judge approaches, human-in-the-loop evaluation, red teaming, or...SuggestedPermanent employmentFull timeRemote work
- ...About WRITERWRITER is the AI platform built for the world... ...world. As an AI research scientist, you'll be at the center of that work. You'll drive... ...through model training, evaluation, and production deploymentDesign... ...learning from human feedback (RLHF), RLAIF, DPO...SuggestedFull timeLocal area
- Ophthalmology AI Evaluation Specialist — Clinical Reasoning Review is a remote evaluation track for reviewing ophthalmology ai evaluation... ...auditable labels, rationales, and regression cases for AuraOne Human Data. Responsibilities Evaluate ophthalmology ai evaluation...SuggestedHourly payFor contractorsRemote work10 hours per week
- AuraOne, Inc. is seeking a Remote Human Resources Expert, Spanish (Spain) to review and label evaluation outputs against our quality rubric. You will compare paired responses, assign severities, and document edge cases for retraining. Ideal candidates have prior annotation...Remote jobFor contractors10 hours per week
- AuraOne, Inc. seeks a Consulting Expert for Mandarin Chinese evaluation on a remote, contractor basis. You will review model outputs against... ...with mandarin evaluation are required; flexibility for asynchronous collaboration is essential. #J-18808-Ljbffr AuraOne Human DataRemote jobFor contractors
- AuraOne, Inc. is seeking a remote evaluation specialist to review agentic task completion model outputs, apply a structured rubric, and provide... ...async availability for at least 10 hours per week. Compensation discussed after interviews. #J-18808-Ljbffr AuraOne Human DataRemote jobFor contractors10 hours per week
$229.9k - $262.4k
Sr. Manager, AI Engineer (Gen AI Platform Services... ...AI, Guardrails, Evaluation) At Capital One, we are... ...AI & ML are bringing humanity and simplicity to banking... ...(IFX) team is at the center of bringing our vision... ...of engineers, research scientists, technical program managers...Full timePart timeLocal area$75 per hour
OpenTrain AI, Inc. is seeking a Human Resources Professional to evaluate AI outputs and write prompts for HR-related tasks. The role is remote, part-time, and contract-based, with compensation up to $75 per hour and a requirement of at least four years of HR experience...Remote jobHourly payContract workPart time$30 - $50 per hour
A tech company is seeking an AI Researcher to support end-to-end research for modern AI systems. This remote role involves designing experiments, defining evaluation protocols, and improving evaluation rigor for large language models. Key responsibilities include developing...Remote jobHourly pay- Human Resource Specialist - Freelance AI Trainer Project United States of America Are you a human resource expert eager to shape the future of AI? Large... ...suggest improvements to our prompt engineering and evaluation metrics. A bachelor's or master’s degree in human...Hourly payContract workFor contractorsFreelanceImmediate startRemote work
$30 - $50 per hour
Review AI-generated guidance on federal student loans, repayment, forgiveness, servicing... ..., and servicer-related challenges. Evaluate AI-generated answers for inaccurate, missing... ...About AI training work AI training is the human work behind systems that answer questions...Hourly payContract workTemporary workPart timeFor contractorsRemote work- LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects. You will assess prompts, review text, analyze multimedia content, and contribute voice recordings, applying local cultural insight and brand guidelines. As an independent contractor...For contractorsLocal area
$8 - $65 per hour
Human Pathogens and Clinic Specialist - Freelance AI Trainer Project United States of America Are you a master of human pathogens and clinical microbiology... ...suggest improvements to our prompt engineering and evaluation metrics. A PhD in microbiology, clinical...Hourly payContract workFor contractorsFreelanceRemote work- LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects. You will work across diverse task types, including evaluating prompts and AI-generated outputs, reviewing and correcting text, analyzing multimedia content, contributing voice...For contractors
$125k - $155k
Company DescriptionPosition: Senior Data Center ConsultantEYP Mission Critical Facilities, part of Ramboll Americas Engineering, is an... ...expertise in interpreting electrical one-line diagrams and evaluating mission-critical electrical distribution systems, including utility...Work at officeLocal areaHome officeFlexible hours- ...A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities...Immediate start
$110k - $135k
...12Accelerate possibilities. The WSP Data Center CoE (Center of Excellence) is growing, and... ...centers—supporting the world’s leading AI and cloud infrastructure companies. You’ll... ...construction administration tasks Survey and evaluate existing conditions Who You AreThe ideal...For contractorsWork at officeLocal areaFlexible hours$97k - $135k
...OpportunityAccelerate possibilities. The WSP Data Center CoE (Center of Excellence) is growing,... ...centers—supporting the world’s leading AI and cloud infrastructure companies. You’... ...Computational Fluid Dynamic (CFD) evaluations for existing and new facilities, both internal...For contractorsLocal areaFlexible hours- LILT is seeking detail-oriented human reviewers to support AI training across diverse tasks. You will evaluate prompts and outputs, review text, analyze multimedia, and contribute voice recordings as needed. Success comes from strong judgment, local cultural insight, and...For contractorsLocal area
- ...trading power in real time, using AI across the business, and... ...customers. As an AI Research Scientist, you will do the pre-training... ...between them that a human analyst would never think to... ...improves results Build rigorous evaluations and honest backtests, with calibrated...Work at office
$150k - $200k
...investment and technology development firm is seeking an AI and DevOps Engineer – Human Capital to join its New York office. The firm seeks... ...first-line investigation of feature issues. Run AI model evaluations, carry out migrations as models are updated or retired,...Work at office- LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects from anywhere. You will assess prompts and outputs, review text, analyze multimedia content, and contribute voice recordings, applying local cultural context and market expectations...Local area
- OpenTrain AI, Inc. is seeking an AI Question and Answer Evaluation Specialist to create research-backed questions and answers, test AI model responses, and improve evaluation content remotely for an hourly-equivalent, task-based compensation model. This entry-level, part...Remote jobHourly payPart time
- Zekai is seeking Evaluation Specialists/Recent Grads for a remote, contractor role contributing to a high-impact AI project. You will develop QA pairs, conduct source-triangulated research, and craft multi-step questions to probe model understanding. No prior AI experience...Remote jobFor contractorsWorldwide
- AuraOne is seeking contractors to review multimodal physics evaluation prompts and responses. You will compare paired outputs, label edge cases, and provide structured feedback to help retrain models. As a remote role with US-eligibility, you will judge model behavior...Remote jobFor contractorsFlexible hours
$60 per hour
Prolific is seeking Computer Science Specialists to join its Expert Network to help train and evaluate AI models. If you have a CS research or technical analysis background, you’ll complete a brief 10-15 minute test; successful candidates will be invited to join Prolific...Remote jobWork from homeFlexible hours- micro1 is a data lab that helps AI labs and enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM capabilities, as well as contextual evaluations to monitor and improve AI agents in enterprise...Remote jobFor contractors
- ...remote contract work. You will craft prompts, run side-by-side evaluations using ChatGPT and Claude, and provide thorough verbal feedback... ...emphasizes consistent evaluation standards and collaboration with AI model improvement efforts. Ideal candidates have strong English...Remote jobContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Human-Centered AI Evaluation Scientist. Be the first to apply!




