Staff Applied Scientist — AI Agent Interfaces & Evaluation
Datadog
Datadog is seeking a Staff Applied Scientist in New York to own evaluation strategies for AI agent integrations. The role involves defining metrics, building datasets, and collaborating with AI engineers to enhance tool selection and retrieval relevance. The ideal candidate will have 10+ years of experience, a high level of expertise in machine learning, and a proven track record in product-driven environments. Competitive salary and benefits package offered. #J-18808-Ljbffr Datadog
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Staff Applied Scientist — AI Agent Interfaces & Evaluation in New York, NY vacancy
$276k - $345k
Team descriptionAt Datadog, AI agents are becoming first-class... ...Bits Dev Agent. The Agentic Interfaces team owns the platform... ...into, and — critically — the evaluation systems that tell us... ...last piece. We're hiring a Staff Applied Scientist to define what "good" means...Suggested$218.4k - $273k
About ScaleAt Scale AI, our mission is to accelerate... ...upon our prior model evaluation work with enterprise... ...About the ACE team The Agent Capabilities & Environments... ...facing Researchers and Applied AI Engineers. Our core... ...code repositories, GUI interfaces, browsers, and more....SuggestedFull time$260k - $382k
Garner Health is seeking a Staff Applied Scientist to build algorithmic systems that evaluate healthcare providers and optimize cost and quality. The position, located in New York City, requires 3+ years of experience in data-driven algorithms, collaboration with engineering...SuggestedFlexible hours$252k - $315k
About Scale AIScale AI is the data foundation for AI, helping... ...to build production AI agents that automate complex... ...engineering challenges.As a Staff Frontier Agent Engineer (Applied AI), you'll bridge the gap... ...enterprise customers to design, evaluate, and deploy intelligent...SuggestedFull time$80 per hour
...specialists with project-based AI opportunities for leading... ...companies, focused on testing, evaluating, and improving AI systems. Participation... ...on building React‑based interfaces and robust Back‑end systems... ...- B2 How It Works Apply → Pass qualification(s) → Join...SuggestedPermanent employmentTemporary work- Afresh Technologies, Inc. is seeking a Staff Applied Scientist to lead R&D on AI models for replenishment technology. Your work will tackle complex inventory problems, aimed at reducing food waste and enhancing produce freshness globally. With a focus on operations research...
- Afresh Technologies, Inc. is seeking a Staff Applied Scientist to lead AI and ML model development for fresh food inventory management. This role focuses on complex decision-making problems impacting food waste and supply chains. Ideal candidates will have an MS or PhD...Remote job
- Zealot is deploying AI systems at the center of western cyber operations... ...live with close U.S. allies abroad. APPLIED AI VR RE FUZZING EXPLOIT DEV... ..., and exploit primitives into tool interfaces for agents. + Build the evaluation and benchmarking infrastructure that...
- ...We’re hiring an Applied AI Engineer to push the boundaries of our Cofounder agent. You’ll own core backend systems and applied LLM work: advancing agent reliability and autonomy, building evaluation pipelines, and shipping techniques that measurably improve agent performance...Full timeImmediate startFlexible hours
The General Intelligence Company Of New York
New York, NY1 day ago $147.6k - $274k
...That’s what makes us Roche.Advances in AI, data, and computational sciences are transforming... ...Intelligence (AI) to assist our scientists in both pRED and gRED to deliver more... ...Machine Learning Scientist building agents for applied small-molecule drug design. You will develop...Full timeLocal areaWorldwideRelocation package$250k
...is establishing a new, unified firmwide AI organization under a newly created... ...competes in the AI era. The Head of Applied AI & Agent Factory is the leader who turns Citi's... ...industrialized capability for designing, building, evaluating, and certifying enterprise AI agents -...Hobbsnews
New York, NY3 days ago$180k - $210k
...TrabaTraba is building the AI operating layer for the... ...them, we are building applied AI that drives real... ...founding member of the Agents team and help us build... ...workDefine product behavior, evaluation systems, and quality... ..., copilots, chat interfaces, wrappers, or lightweight...Temporary workLocal areaFlexible hoursShift work$80 per hour
...intelligence to ethically shape the future of AI. What We Do The Mindrift platform... ...can design realistic and structured evaluation scenarios for LLM-based agents. You'll create test cases that... ...reusable How To Get Started Simply apply to this post, qualify, and get the...Part timeFreelanceRemote workFlexible hours- ...customer experiences with AI. We are primarily an in... ...growing industries—our agents already reach over 50%... ...roles where you interfaced with customers or led technical... .... We encourage you to apply even if your experience... .... We strive to evaluate all applicants consistently...Full timeFlexible hours
Sierra
New York, NY1 day ago $262.5k - $299.6k
...creating trustworthy and reliable AI systems, changing banking for... ...to building world‑class applied science and engineering teams... ...cross‑functional team of data scientists, software engineers, machine‑... ...design through training, evaluation, validation, and implementation...Full timePart timeLocal areaCapital One
New York, NY1 day ago$300k - $390k
...redesign how healthcare works: applying 550+ proprietary clinical... ...in the U.S. — and using AI to scale that impact further... ...are seeking an exceptional Staff Applied Scientist to join our Applied Science... ...our website, including the evaluation harness, guardrails, and ongoing...Work at officeFlexible hours3 days per weekGarner Health
New York, NY5 days ago- ...deployed to a user base of over 80 million customers.As an AI Agents Applied Research/Engineering Lead in our The Digital Team, you will... ...LLM systems through caching, batching, prompt governance, and evaluation frameworks.Implement privacy, safety, and security controls...
$262.5k - $299.6k
Applied Researcher II (AI Foundations, LLM Core and Agentic AI) Overview: At Capital One, we are creating... ...with a cross-functional team of data scientists, software engineers, machine learning... ..., from design through training, evaluation, validation, and implementation....Full timePart timeLocal areaFlexible hoursCapital One
New York, NY5 days ago$262.5k - $299.6k
Applied Researcher II (AI Foundations) Overview: At Capital One, we are creating trustworthy and reliable... ...with a cross-functional team of data scientists, software engineers, machine learning... ..., from design through training, evaluation, validation, and implementation....Full timePart timeLocal areaFlexible hoursCapital One
New York, NY5 days ago$170k - $199k
...are building an elite team, applying frontier technologies to the... ...predictive insights and tactical AI into our workforce systems.... ...organizational needs.As a Staff Data Scientist, you will serve as the team'... ...owner for internal data interfaces—collaborating directly with...Work at officeFlexible hoursShift work3 days per week$172.5k - $260.1k
...experience, please consider applying for a maximum of 3 roles... ...is the #1 AI CRM, where humans with agents drive customer success together... ...looking for an experienced Staff Data Scientist to support Slack's Go-to... ...understand customer behavior, evaluate strategic initiatives,...Full time$195.5k - $218.5k
...are about finding new ways to apply a blend of market design,... ...advertising.OpenX is seeking a Staff Data Scientist to join our Data Science... ..., deep learning, and modern AI techniques delivered through... ...across adjacent areasLead the evaluation and adoption of new AI/ML modeling...Full timeWork experience placementLocal area- ...Intern will work within the SearsKAIros Applied AI Division at TransformCo, partnering directly... ...prototype, deploy, and improve AI agents that drive measurable operational and financial... ...through production deployment Develops evaluations, guardrails, monitoring, and rollback...Internship
- ...Applied AI/ML Researcher As an Applied AI/ML Researcher within the AI4Tech Team at JPMorgan Chase, you will lead technology research... ...observability, and operational resilience. - Design, prototype, and evaluate AI-driven tools and frameworks that streamline engineering...
Chase
New York, NY5 days ago - Primitive Labs is building AI systems that are human-aligned, hiring for potential and rewarding impactful research. The role sits in... ...design experiments, and build simulations for behavioral intelligence agents. Expect to publish work, contribute to open-source projects, and...Full time
$260k - $382k
...impact on healthcare at scale. About the role We are seeking an exceptional Staff Applied Scientist to build the algorithmic systems that power Garner. These systems determine how we evaluate providers, make recommendations, and optimize for outcomes across cost, quality...Work at officeWork visaFlexible hours3 days per week- ...creating trustworthy and reliable AI systems, changing banking for... ...We are building world-class applied science and engineering teams... ...-functional team of data scientists, software engineers, machine... ...from design through training, evaluation, validation, and implementation...Full timePart time
$205.6k - $257k
Scale has been the leading AI data foundry, helping fuel the most... ...vertical within our Agents Data & Reinforcement Learning... ...workflows that labs use to train and evaluate agents) and the “data as a product... ..., clinicians & clinical staff using EHRs daily, revenue cycle...$150k - $250k
About Distyl AI Distyl is an applied AI technology company partnering with the world’s most ambitious institutions to rearchitect critical operations... ...and alignment requirements. Researchers develop and evaluate techniques such as supervised fine-tuning, preference optimization...Work at officeFlexible hours3 days per week- ...Hobbsnews seeks a Head of Applied AI & Agent Factory to lead AI transformation and turn AI capabilities into measurable business outcomes. This executive role will manage the Agent Factory, overseeing the design and certification of enterprise AI agents while ensuring...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Applied Scientist — AI Agent Interfaces & Evaluation. Be the first to apply!
Related searches