Senior Research Scientist, Agent Flywheel — AI Evaluation
FDE Times
Databricks is seeking a Sr. Research Scientist for the Agent Flywheel team in New York. You will build context layers for recursive self-improvement, turning raw production traces into signals that guide training and agent enhancement. You will calibrate LLM judges, create datasets from multi-turn trajectories, and design benchmarks to probe long-horizon, tool-using agent behavior, driving state-of-the-art evaluation. #J-18808-Ljbffr FDE Times
- Databricks is seeking a Sr. Research Scientist for the Agent Flywheel team to build the context layer for recursive self-improvement and autofix loops for data agents. You will turn raw production traces into online signals for optimization and contribute to models and...Senior
$164k - $235k
Sr. Research Scientist - Agent Flywheel Company Description At Databricks, we are obsessed with enabling data... ...running the world's best data and AI platform so our customers can focus... ...the loop between production signals, evaluation, and agent improvement. Design benchmarks...SeniorLocal areaWorldwide$123k - $215.25k
...the market.We are looking for a Senior Technical Product Manager to... ...that hlep teams test and improve agent-enabled customer experiences.... ...and adoption of shared evaluation tools and workflows. Working with... ...analytics, quality tooling, or AI products.Experience owning evaluation...Senior$259.2k - $324k
About ScaleAt Scale AI, our mission is to accelerate the development... ...building upon our prior model evaluation work with enterprise customers... ....About the ACE team The Agent Capabilities & Environments (ACE) team, part of Scale’s Research organization, brings together...SeniorFull time- ...Description: As a Senior AI Research Scientist, you will bridge cutting-edge AI research with production... ...Large Language Models (LLMs), AI Agents, and Multimodal AI to solve real-world... ...through implementation, deployment, evaluation, and continuous iteration. Build AI...SeniorFull time
- Mecka AI in New York seeks an experienced Research Scientist to advance large-scale vision-language and multimodal foundation models. You will work on model... ...will collaborate across data, training strategies, evaluation, and optimization, exploring emerging approaches in...Senior
$295k
...is the leading security-first enterprise AI company. We build cutting-edge... ...for our customers. Cohere is a team of researchers, engineers, designers, and more, who are... ...Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling...SeniorFull timeWork at officeLocal areaRemote workHome office$200k - $250k
...data removal from broker sites, and an AI voice agent that screens calls before they ring through... ...About the Role This role is a Senior Product Manager on the Agents and Automation... ..., so they trust it with more Build evaluations that ensure the quality, safety, and...SeniorFull timeWork at officeHome office- ...Senior Software Engineer, Agent Platform New York, NY A recently-minted unicorn — valued at $1.5B and... ...growth investors — is building the AI infrastructure powering human-led creator... ...how autonomous agents get built, evaluated, and trusted in production, inventing...SeniorFull time
$216k - $270k
...is to develop reliable AI systems for the world’s... ...most, combining rigorous evaluation with full-stack... ...paired with applied ML research, design, and evaluation... ...a Software Engineer on Agent Oversight, you will build... ...mentoring engineers at a senior/staff levelCompensation...SeniorFull time$200k - $240k
Traba is building the AI operating system for the industrial supply... ...'ve expanded our suite of AI agents to help logistics providers,... ...re seeking an entrepreneurial Senior AI Agent Engineer to join as a... ...a deadline, you scope, build, evaluate, and ship without waiting for...SeniorTemporary workLocal areaDay shift- Senior Full Stack Engineer Kiddom is transforming K-12 education by... ...Engineer to join our Content Agents team, building intelligent systems... ...frontend, backend, data, and AI, developing intuitive product... ...systems. Familiarity with LLM evaluation, retrieval techniques, and...SeniorLocal areaFlexible hours
$61 - $71 per hour
...Senior AI Agent Developer – Conversational AI, Python & GCP (Hybrid) Job Title: Senior AI Agent Developer – Conversational AI, Python... ...engineering, and intent classification . Experience building evaluation frameworks to measure conversational AI accuracy,...SeniorLong term contractFull timeContract workRemote workMonday to Friday3 days per week$55 - $75 per hour
...About the opportunity Help evaluate AI-generated answers on cybersecurity tasks. You’ll read code, apply the assigned policy to each case, and write concise notes that explain your labels. You’ll switch between policy sets as assignments change, flag issues in AI output...SeniorHourly payFull timeFor contractorsRemote work$290.4k - $363k
Scale AI is the data foundation for AI, helping organizations... ....About the General Agents TeamThe General Agents... ....About the RoleAs a Senior/Staff Machine Learning... ...and system design to evaluation, deployment, and... ...problem spaces, balancing research-driven approaches with...SeniorFull time$262.5k - $299.6k
Applied Researcher 5 (AI Foundations: Evaluation and Agentic Workflows) At Capital One, we are creating trustworthy and reliable AI systems, changing... ...ll Do: Partner with a cross-functional team of data scientists, software engineers, machine learning engineers and...Full timePart timeWork at officeLocal areaFlexible hours$167.4k - $310.8k
...us Roche.Advances in AI, data, and computational... ...development. Roche’s Research and Early Development... ...Intelligence (AI) to assist our scientists in both pRED and gRED... .... We are seeking a Senior or Principal Machine... ...strategies, and evaluation methodologies.Model Capability...SeniorFull timeLocal areaWorldwideRelocation package- Senior UI/UX Designer - AI We are looking for a Senior UI/UX Designer - AI to help... ...AI/ML teams, Engineering, Research, and other designers throughout... ...and quantitative data to evaluate experiences and recommend... ...loops. Familiarity with AI agents and emerging patterns for designing...SeniorImmediate start
$145k - $225k
...About the RoleWe're looking for a Senior Software Engineer to join our Director... ...of Engineering building ShopMy's agent orchestration platform: the system our AI agents are built, run, managed,... ...console agent managers use to create, evaluate, and improve agents (React)Make...SeniorWork experience placementWork at officeImmediate startFlexible hours- ...We are looking for a Senior Applied AI Engineer to join the Agent Runtime team behind Alwin , our new Agentic Automation Platform for audit and finance... ...prototype with our audit domain experts, through evaluation, to production on Alwin Define and build automatic...SeniorFull timeWork at officeFlexible hours
$167.1k - $226.1k
...generation tools to enable custom AI/ML solutions for customers.AWS,... ...problems that will give Applied Scientists like you endless opportunities to see your research have a positive and immediate... ...multimodal), agentic AI, agentic evaluations, MTRL, ML gyms.Basic qualifications...SeniorImmediate startFlexible hours- Overview Senior ServiceNow Developer (Virtual Agent / Now Assist) St Louis, MO Are you ready... ..., and future NLU/AI readiness in a customized... ...common troubleshooting needs. Evaluate existing customizations,... ...confident. To help you with your research, we suggest you review...SeniorContract workFor contractorsWork at officeRemote workWork from homeHome office
$99.6k - $174k
Senior Full Stack Engineer, AI Platform & AgentsBuild the GenAI platform that powers... ..., and deployment of AI agents. We also drive the development... ...Expert AI—a medical research and clinical reasoning agent... ...production, including safety evaluations, iterative testing (e.g.,...SeniorFull timeWork at officeRemote work2 days per week$167.1k - $226.1k
...curation, post-training, evaluation, and production... ...customized models? As a Senior Applied Scientist in Amazon Web... ...deep relationships with AI service and science teams... ...security/governance/agent-ops, and DevOps.... ...6+ years of applied research experience - Experience...SeniorFull timeLocal areaFlexible hoursDay shift$189k - $260k
Secure Every Identity, from AI to HumanIdentity is the key to unlocking the... .... If you are too, let's talk.As a Senior Product Manager, Okta for AI Agents, you will drive the product... ...Ability to work cross-functionally with research, design, engineering, and customer...SeniorLocal areaWorldwideFlexible hours- ...Senior / Staff Member of Technical Staff Employment Type: Full... ...Experience: 4+ years Focus: AI Product Engineering, Full-... ...Distributed Systems, APIs, Customer Research Platforms About Our... ...business decisions. Their AI agents automate the entire research...SeniorFull time
- ...About the Role We're looking for a Senior Software Engineer (AI Agents) to build the core systems behind an intelligent automation platform used in highly complex, operations-heavy industries. You’ll design and ship production-grade LLM systems, contribute across the...SeniorFull time
$127.4k - $236.6k
...requiring fiduciary-grade AI. The Thomson LLM family... ...latest model release.The research team trains and evaluates the Thomson models to perform... ...efforts.We are hiring scientists at Senior and Lead level across three... ...tasks inside real agent harnesses), multi-turn RL...SeniorFull timeLocal areaFlexible hours- ...enterprises who are building AI systems to power... ...search, RAG, and agents. We believe that our... ...Cohere is a team of researchers, engineers, designers... ...future! Why this role?Evaluation is critical to making... ...LLM progress. As a Senior Research Scientist, Model Evaluation, you...SeniorFull timeWork at officeRemote workFlexible hours
$147.6k - $274k
...makes us Roche.Advances in AI, data, and computational... ...development. Roche’s Research and Early Development organisations... ...(AI) to assist our scientists in both pRED and gRED to... ...Learning Scientist / Senior Machine Learning Scientist building agents for applied small-...SeniorFull timeLocal areaWorldwideRelocation package
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Research Scientist, Agent Flywheel — AI Evaluation. Be the first to apply!
- scientist 1 New York, NY
- research scientist machine learning deep learning New York, NY
- scientist ii New York, NY
- junior research scientist New York, NY
- scientist New York, NY
- quality control scientist New York, NY
- entry level research scientist New York, NY
- molecular biology scientist New York, NY
- pharmaceutical scientist New York, NY
- scientist antibody discovery New York, NY





