Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Human-Centered AI Evaluation Scientist

Contra

Contra Labs in New York City seeks a researcher to transform ambiguous client questions into rigorous projects across evaluation, post-training data, benchmarking, and human-centered research. You will own end-to-end engagements, design studies, run analyses in Python/SQL, and generate client-ready insights, reports, and publications while balancing rigor with startup speed and pragmatism. #J-18808-Ljbffr Contra

Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Human-Centered AI Evaluation Scientist in New York, NY vacancy
  • $229.9k - $262.4k

    AI Engineer 5 (Gen AI SDK: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible...  ...AI & ML are bringing humanity and simplicity to banking...  ...(IFX) team is at the center of bringing our vision...  ...engineers, research scientists, technical program managers... 
    Suggested
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 days ago
  • $197.3k - $225.1k

    AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability) At Capital One, we are creating...  ...AI & ML are bringing humanity and simplicity to...  ...Experiences (IFX) team is at the center of bringing our vision...  ...of engineers, research scientists, technical program... 
    Suggested
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 days ago
  •  ...Position Title: Senior Computer Programmer – AI Evaluation Specialist Location: Remote (...  ...will partner with product managers, data scientists, AI engineers, and security and privacy...  ..., model-as-a-judge approaches, human-in-the-loop evaluation, red teaming, or... 
    Suggested
    Permanent employment
    Full time
    Remote work

    LinkedIn

    New York, NY
    5 days ago
  •  ...About WRITERWRITER is the AI platform built for the world...  ...world. As an AI research scientist, you'll be at the center of that work. You'll drive...  ...through model training, evaluation, and production deploymentDesign...  ...learning from human feedback (RLHF), RLAIF, DPO... 
    Suggested
    Full time
    Local area

    Writer Corporation

    New York, NY
    4 days ago
  • Ophthalmology AI Evaluation Specialist — Clinical Reasoning Review is a remote evaluation track for reviewing ophthalmology ai evaluation...  ...auditable labels, rationales, and regression cases for AuraOne Human Data. Responsibilities Evaluate ophthalmology ai evaluation... 
    Suggested
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne, Inc.

    New York, NY
    4 days ago
  • AuraOne, Inc. is seeking a Remote Human Resources Expert, Spanish (Spain) to review and label evaluation outputs against our quality rubric. You will compare paired responses, assign severities, and document edge cases for retraining. Ideal candidates have prior annotation... 
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    2 days ago
  • AuraOne, Inc. seeks a Consulting Expert for Mandarin Chinese evaluation on a remote, contractor basis. You will review model outputs against...  ...with mandarin evaluation are required; flexibility for asynchronous collaboration is essential. #J-18808-Ljbffr AuraOne Human Data
    Remote job
    For contractors

    AuraOne Human Data

    New York, NY
    4 days ago
  • AuraOne, Inc. is seeking a remote evaluation specialist to review agentic task completion model outputs, apply a structured rubric, and provide...  ...async availability for at least 10 hours per week. Compensation discussed after interviews. #J-18808-Ljbffr AuraOne Human Data
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    5 days ago
  • $229.9k - $262.4k

    Sr. Manager, AI Engineer (Gen AI Platform Services...  ...AI, Guardrails, Evaluation) At Capital One, we are...  ...AI & ML are bringing humanity and simplicity to banking...  ...(IFX) team is at the center of bringing our vision...  ...of engineers, research scientists, technical program managers... 
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 days ago
  • $75 per hour

    OpenTrain AI, Inc. is seeking a Human Resources Professional to evaluate AI outputs and write prompts for HR-related tasks. The role is remote, part-time, and contract-based, with compensation up to $75 per hour and a requirement of at least four years of HR experience... 
    Remote job
    Hourly pay
    Contract work
    Part time

    OpenTrain AI, Inc.

    New York, NY
    5 days ago
  • $30 - $50 per hour

    A tech company is seeking an AI Researcher to support end-to-end research for modern AI systems. This remote role involves designing experiments, defining evaluation protocols, and improving evaluation rigor for large language models. Key responsibilities include developing... 
    Remote job
    Hourly pay

    Rex.zone

    New York, NY
    4 days ago
  • Human Resource Specialist - Freelance AI Trainer Project United States of America Are you a human resource expert eager to shape the future of AI? Large...  ...suggest improvements to our prompt engineering and evaluation metrics. A bachelor's or master’s degree in human... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start
    Remote work

    Meridial

    New York, NY
    2 days ago
  • $30 - $50 per hour

    Review AI-generated guidance on federal student loans, repayment, forgiveness, servicing...  ..., and servicer-related challenges. Evaluate AI-generated answers for inaccurate, missing...  ...About AI training work AI training is the human work behind systems that answer questions... 
    Hourly pay
    Contract work
    Temporary work
    Part time
    For contractors
    Remote work

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects. You will assess prompts, review text, analyze multimedia content, and contribute voice recordings, applying local cultural insight and brand guidelines. As an independent contractor... 
    For contractors
    Local area

    LILT

    New York, NY
    5 days ago
  • $8 - $65 per hour

    Human Pathogens and Clinic Specialist - Freelance AI Trainer Project United States of America Are you a master of human pathogens and clinical microbiology...  ...suggest improvements to our prompt engineering and evaluation metrics. A PhD in microbiology, clinical... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    Meridial

    New York, NY
    5 days ago
  • LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects. You will work across diverse task types, including evaluating prompts and AI-generated outputs, reviewing and correcting text, analyzing multimedia content, contributing voice... 
    For contractors

    LILT

    New York, NY
    5 days ago
  • $125k - $155k

    Company DescriptionPosition: Senior Data Center ConsultantEYP Mission Critical Facilities, part of Ramboll Americas Engineering, is an...  ...expertise in interpreting electrical one-line diagrams and evaluating mission-critical electrical distribution systems, including utility... 
    Work at office
    Local area
    Home office
    Flexible hours

    Ramboll Group

    New York, NY
    1 day ago
  •  ...A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities... 
    Immediate start

    Crossing Hurdles

    New York, NY
    4 days ago
  • $110k - $135k

     ...12Accelerate possibilities. The WSP Data Center CoE (Center of Excellence) is growing, and...  ...centers—supporting the world’s leading AI and cloud infrastructure companies. You’ll...  ...construction administration tasks Survey and evaluate existing conditions Who You AreThe ideal... 
    For contractors
    Work at office
    Local area
    Flexible hours

    WSP Group

    New York, NY
    15 hours ago
  • $97k - $135k

     ...OpportunityAccelerate possibilities. The WSP Data Center CoE (Center of Excellence) is growing,...  ...centers—supporting the world’s leading AI and cloud infrastructure companies. You’...  ...Computational Fluid Dynamic (CFD) evaluations for existing and new facilities, both internal... 
    For contractors
    Local area
    Flexible hours

    WSP Group

    New York, NY
    1 day ago
  • LILT is seeking detail-oriented human reviewers to support AI training across diverse tasks. You will evaluate prompts and outputs, review text, analyze multimedia, and contribute voice recordings as needed. Success comes from strong judgment, local cultural insight, and... 
    For contractors
    Local area

    LILT

    New York, NY
    5 days ago
  •  ...trading power in real time, using AI across the business, and...  ...customers. As an AI Research Scientist, you will do the pre-training...  ...between them that a human analyst would never think to...  ...improves results Build rigorous evaluations and honest backtests, with calibrated... 
    Work at office

    Fuse Energy LLC

    New York, NY
    4 days ago
  • $150k - $200k

     ...investment and technology development firm is seeking an AI and DevOps Engineer – Human Capital to join its New York office. The firm seeks...  ...first-line investigation of feature issues. Run AI model evaluations, carry out migrations as models are updated or retired,... 
    Work at office
    New York, NY
    12 days ago
  • LILT is seeking detail-oriented human reviewers to support AI training and evaluation projects from anywhere. You will assess prompts and outputs, review text, analyze multimedia content, and contribute voice recordings, applying local cultural context and market expectations... 
    Local area

    LILT

    New York, NY
    5 days ago
  • OpenTrain AI, Inc. is seeking an AI Question and Answer Evaluation Specialist to create research-backed questions and answers, test AI model responses, and improve evaluation content remotely for an hourly-equivalent, task-based compensation model. This entry-level, part... 
    Remote job
    Hourly pay
    Part time

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • Zekai is seeking Evaluation Specialists/Recent Grads for a remote, contractor role contributing to a high-impact AI project. You will develop QA pairs, conduct source-triangulated research, and craft multi-step questions to probe model understanding. No prior AI experience... 
    Remote job
    For contractors
    Worldwide

    Confidential

    New York, NY
    4 days ago
  • AuraOne is seeking contractors to review multimodal physics evaluation prompts and responses. You will compare paired outputs, label edge cases, and provide structured feedback to help retrain models. As a remote role with US-eligibility, you will judge model behavior... 
    Remote job
    For contractors
    Flexible hours

    Auraone

    New York, NY
    4 days ago
  • $60 per hour

    Prolific is seeking Computer Science Specialists to join its Expert Network to help train and evaluate AI models. If you have a CS research or technical analysis background, you’ll complete a brief 10-15 minute test; successful candidates will be invited to join Prolific... 
    Remote job
    Work from home
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    6 days ago
  • micro1 is a data lab that helps AI labs and enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM capabilities, as well as contextual evaluations to monitor and improve AI agents in enterprise... 
    Remote job
    For contractors

    micro1

    New York, NY
    2 days ago
  •  ...remote contract work. You will craft prompts, run side-by-side evaluations using ChatGPT and Claude, and provide thorough verbal feedback...  ...emphasizes consistent evaluation standards and collaboration with AI model improvement efforts. Ideal candidates have strong English... 
    Remote job
    Contract work

    Crossing Hurdles

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Human-Centered AI Evaluation Scientist. Be the first to apply!