Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Research Scientist, Agent Flywheel — AI Evaluation

FDE Times

Databricks is seeking a Sr. Research Scientist for the Agent Flywheel team in New York. You will build context layers for recursive self-improvement, turning raw production traces into signals that guide training and agent enhancement. You will calibrate LLM judges, create datasets from multi-turn trajectories, and design benchmarks to probe long-horizon, tool-using agent behavior, driving state-of-the-art evaluation. #J-18808-Ljbffr FDE Times

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Research Scientist, Agent Flywheel — AI Evaluation in New York, NY vacancy
  • Databricks is seeking a Sr. Research Scientist for the Agent Flywheel team to build the context layer for recursive self-improvement and autofix loops for data agents. You will turn raw production traces into online signals for optimization and contribute to models and... 
    Senior

    Neon

    New York, NY
    10 hours ago
  • $164k - $235k

    Sr. Research Scientist - Agent Flywheel Company Description At Databricks, we are obsessed with enabling data...  ...running the world's best data and AI platform so our customers can focus...  ...the loop between production signals, evaluation, and agent improvement. Design benchmarks... 
    Senior
    Local area
    Worldwide

    FDE Times

    New York, NY
    2 days ago
  • $123k - $215.25k

     ...the market.We are looking for a Senior Technical Product Manager to...  ...that hlep teams test and improve agent-enabled customer experiences....  ...and adoption of shared evaluation tools and workflows. Working with...  ...analytics, quality tooling, or AI products.Experience owning evaluation... 
    Senior

    American Express

    New York, NY
    2 days ago
  • $259.2k - $324k

    About ScaleAt Scale AI, our mission is to accelerate the development...  ...building upon our prior model evaluation work with enterprise customers...  ....About the ACE team The Agent Capabilities & Environments (ACE) team, part of Scale’s Research organization, brings together... 
    Senior
    Full time

    Scale AI

    New York, NY
    10 hours ago
  •  ...Description: As a Senior AI Research Scientist, you will bridge cutting-edge AI research with production...  ...Large Language Models (LLMs), AI Agents, and Multimodal AI to solve real-world...  ...through implementation, deployment, evaluation, and continuous iteration. Build AI... 
    Senior
    Full time

    CTW

    New York, NY
    2 days ago
  • Mecka AI in New York seeks an experienced Research Scientist to advance large-scale vision-language and multimodal foundation models. You will work on model...  ...will collaborate across data, training strategies, evaluation, and optimization, exploring emerging approaches in... 
    Senior

    Mecka

    New York, NY
    10 hours ago
  • $295k

     ...is the leading security-first enterprise AI company. We build cutting-edge...  ...for our customers. Cohere is a team of researchers, engineers, designers, and more, who are...  ...Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling... 
    Senior
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    10 hours ago
  • $200k - $250k

     ...data removal from broker sites, and an AI voice agent that screens calls before they ring through...  ...About the Role This role is a Senior Product Manager on the Agents and Automation...  ..., so they trust it with more Build evaluations that ensure the quality, safety, and... 
    Senior
    Full time
    Work at office
    Home office

    cloaked

    New York, NY
    3 days ago
  •  ...Senior Software Engineer, Agent Platform New York, NY A recently-minted unicorn — valued at $1.5B and...  ...growth investors — is building the AI infrastructure powering human-led creator...  ...how autonomous agents get built, evaluated, and trusted in production, inventing... 
    Senior
    Full time

    Integra Tech Advisors

    New York, NY
    4 days ago
  • $216k - $270k

     ...is to develop reliable AI systems for the world’s...  ...most, combining rigorous evaluation with full-stack...  ...paired with applied ML research, design, and evaluation...  ...a Software Engineer on Agent Oversight, you will build...  ...mentoring engineers at a senior/staff levelCompensation... 
    Senior
    Full time

    Scale AI

    New York, NY
    10 hours ago
  • $200k - $240k

    Traba is building the AI operating system for the industrial supply...  ...'ve expanded our suite of AI agents to help logistics providers,...  ...re seeking an entrepreneurial Senior AI Agent Engineer to join as a...  ...a deadline, you scope, build, evaluate, and ship without waiting for... 
    Senior
    Temporary work
    Local area
    Day shift

    Traba

    New York, NY
    1 day ago
  • Senior Full Stack Engineer Kiddom is transforming K-12 education by...  ...Engineer to join our Content Agents team, building intelligent systems...  ...frontend, backend, data, and AI, developing intuitive product...  ...systems. Familiarity with LLM evaluation, retrieval techniques, and... 
    Senior
    Local area
    Flexible hours

    Kiddom

    New York, NY
    3 days ago
  • $61 - $71 per hour

     ...Senior AI Agent Developer – Conversational AI, Python & GCP (Hybrid) Job Title: Senior AI Agent Developer – Conversational AI, Python...  ...engineering, and intent classification . Experience building evaluation frameworks to measure conversational AI accuracy,... 
    Senior
    Long term contract
    Full time
    Contract work
    Remote work
    Monday to Friday
    3 days per week

    Seneca Resources

    New York, NY
    2 days ago
  • $55 - $75 per hour

     ...About the opportunity Help evaluate AI-generated answers on cybersecurity tasks. You’ll read code, apply the assigned policy to each case, and write concise notes that explain your labels. You’ll switch between policy sets as assignments change, flag issues in AI output... 
    Senior
    Hourly pay
    Full time
    For contractors
    Remote work

    Braintrust

    New York, NY
    2 days ago
  • $290.4k - $363k

    Scale AI is the data foundation for AI, helping organizations...  ....About the General Agents TeamThe General Agents...  ....About the RoleAs a Senior/Staff Machine Learning...  ...and system design to evaluation, deployment, and...  ...problem spaces, balancing research-driven approaches with... 
    Senior
    Full time

    Scale AI

    New York, NY
    10 hours ago
  • $262.5k - $299.6k

    Applied Researcher 5 (AI Foundations: Evaluation and Agentic Workflows) At Capital One, we are creating trustworthy and reliable AI systems, changing...  ...ll Do: Partner with a cross-functional team of data scientists, software engineers, machine learning engineers and... 
    Full time
    Part time
    Work at office
    Local area
    Flexible hours

    Capital One

    New York, NY
    2 days ago
  • $167.4k - $310.8k

     ...us Roche.Advances in AI, data, and computational...  ...development. Roche’s Research and Early Development...  ...Intelligence (AI) to assist our scientists in both pRED and gRED...  .... We are seeking a Senior or Principal Machine...  ...strategies, and evaluation methodologies.Model Capability... 
    Senior
    Full time
    Local area
    Worldwide
    Relocation package

    Genentech

    New York, NY
    10 hours ago
  • Senior UI/UX Designer - AI We are looking for a Senior UI/UX Designer - AI to help...  ...AI/ML teams, Engineering, Research, and other designers throughout...  ...and quantitative data to evaluate experiences and recommend...  ...loops. Familiarity with AI agents and emerging patterns for designing... 
    Senior
    Immediate start

    Product Service

    New York, NY
    3 days ago
  • $145k - $225k

     ...About the RoleWe're looking for a Senior Software Engineer to join our Director...  ...of Engineering building ShopMy's agent orchestration platform: the system our AI agents are built, run, managed,...  ...console agent managers use to create, evaluate, and improve agents (React)Make... 
    Senior
    Work experience placement
    Work at office
    Immediate start
    Flexible hours

    ShopMy

    New York, NY
    10 hours ago
  •  ...We are looking for a Senior Applied AI Engineer to join the Agent Runtime team behind Alwin , our new Agentic Automation Platform for audit and finance...  ...prototype with our audit domain experts, through evaluation, to production on Alwin Define and build automatic... 
    Senior
    Full time
    Work at office
    Flexible hours

    datasnipper

    New York, NY
    9 days ago
  • $167.1k - $226.1k

     ...generation tools to enable custom AI/ML solutions for customers.AWS,...  ...problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate...  ...multimodal), agentic AI, agentic evaluations, MTRL, ML gyms.Basic qualifications... 
    Senior
    Immediate start
    Flexible hours

    Amazon

    New York, NY
    4 days ago
  • Overview Senior ServiceNow Developer (Virtual Agent / Now Assist) St Louis, MO Are you ready...  ..., and future NLU/AI readiness in a customized...  ...common troubleshooting needs. Evaluate existing customizations,...  ...confident. To help you with your research, we suggest you review... 
    Senior
    Contract work
    For contractors
    Work at office
    Remote work
    Work from home
    Home office

    Chenega MIOS

    New York, NY
    3 days ago
  • $99.6k - $174k

    Senior Full Stack Engineer, AI Platform & AgentsBuild the GenAI platform that powers...  ..., and deployment of AI agents. We also drive the development...  ...Expert AI—a medical research and clinical reasoning agent...  ...production, including safety evaluations, iterative testing (e.g.,... 
    Senior
    Full time
    Work at office
    Remote work
    2 days per week

    Wolters Kluwer

    New York, NY
    10 hours ago
  • $167.1k - $226.1k

     ...curation, post-training, evaluation, and production...  ...customized models? As a Senior Applied Scientist in Amazon Web...  ...deep relationships with AI service and science teams...  ...security/governance/agent-ops, and DevOps....  ...6+ years of applied research experience - Experience... 
    Senior
    Full time
    Local area
    Flexible hours
    Day shift

    Amazon

    New York, NY
    9 days ago
  • $189k - $260k

    Secure Every Identity, from AI to HumanIdentity is the key to unlocking the...  .... If you are too, let's talk.As a Senior Product Manager, Okta for AI Agents, you will drive the product...  ...Ability to work cross-functionally with research, design, engineering, and customer... 
    Senior
    Local area
    Worldwide
    Flexible hours

    Okta

    New York, NY
    3 days ago
  •  ...Senior / Staff Member of Technical Staff Employment Type: Full...  ...Experience: 4+ years Focus: AI Product Engineering, Full-...  ...Distributed Systems, APIs, Customer Research Platforms About Our...  ...business decisions. Their AI agents automate the entire research... 
    Senior
    Full time

    Nxt Level

    New York, NY
    4 days ago
  •  ...About the Role We're looking for a Senior Software Engineer (AI Agents) to build the core systems behind an intelligent automation platform used in highly complex, operations-heavy industries. You’ll design and ship production-grade LLM systems, contribute across the... 
    Senior
    Full time

    arcadia

    New York, NY
    2 days ago
  • $127.4k - $236.6k

     ...requiring fiduciary-grade AI. The Thomson LLM family...  ...latest model release.The research team trains and evaluates the Thomson models to perform...  ...efforts.We are hiring scientists at Senior and Lead level across three...  ...tasks inside real agent harnesses), multi-turn RL... 
    Senior
    Full time
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    2 days ago
  •  ...enterprises who are building AI systems to power...  ...search, RAG, and agents. We believe that our...  ...Cohere is a team of researchers, engineers, designers...  ...future! Why this role?Evaluation is critical to making...  ...LLM progress. As a Senior Research Scientist, Model Evaluation, you... 
    Senior
    Full time
    Work at office
    Remote work
    Flexible hours

    cohere

    New York, NY
    3 days ago
  • $147.6k - $274k

     ...makes us Roche.Advances in AI, data, and computational...  ...development. Roche’s Research and Early Development organisations...  ...(AI) to assist our scientists in both pRED and gRED to...  ...Learning Scientist / Senior Machine Learning Scientist building agents for applied small-... 
    Senior
    Full time
    Local area
    Worldwide
    Relocation package

    Genentech

    New York, NY
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Research Scientist, Agent Flywheel — AI Evaluation. Be the first to apply!