Senior AI Scientist Agent Evaluation & Experimentation
Amazon Locker
Amazon is seeking a Sr. Applied Scientist in DTx Science to lead agent evaluation, experimentation, and impact measurement across the advertiser experience.
You will apply ML and AI methods, mentor junior scientists, and drive collaborations with cross‑functional teams to deliver scalable, science‑driven improvements. The role emphasizes state‑of‑the‑art techniques, from generative AI to reinforcement learning and causal inference, with strong communication to business stakeholders and a focus
#J-18808-LjbffrVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior AI Scientist Agent Evaluation & Experimentation in Seattle, WA vacancy
$150k - $250k
...expert marketers to grow. Today, our AI helps members create and edit... ...You will own Flodesk's prompts, agent instructions, quality definitions, evaluation criteria and continuous-improvement... ...evaluation, observability and experimental design are areas where you have direct...SeniorImmediate startRemote workRelocationFlexible hours- Flodesk, a remote-first email marketing company, seeks an experienced AI Systems professional to own prompts, agent instructions, quality definitions, and evaluation infrastructure. You’ll connect engineering, product, and language models to improve member experiences and...SeniorRemote work
- Apple Inc. in Seattle, WA is seeking a Senior Applied Scientist to advance AI evaluation and quality systems. You will design scalable ground-truth pipelines and real-time monitors, calibrate evaluators against human gold sets, and surface root-cause analysis to improve...Senior
- Perplexity in Seattle is hiring a senior software/ML engineer to build enterprise evaluation environments and automated graders for agents, with the goal of improving reliability, latency, and cost across complex workflows. You will design robust backend systems, run experiments...Senior
$138k - $172.1k
...expertise, and Knowledge and AI are our strategic advantage. We... ...ecosystem and autonomous service agents that together accelerate... ...support operations.We are hiring a Senior Manager, Knowledge Engineering... ...systems.Establish agent evaluation and benchmarking practices covering...SeniorFull timeTemporary workWork at officeImmediate startVisa sponsorshipWork visaSonos
Seattle, WA1 hour ago$124k - $280k
...Description & SummaryThe OpportunityAs an AI & GenAI Data Scientist - Senior Manager you will leverage advanced... ..., predictive modeling, and rapid experimentation to generate actionable insights... ...scalable solution delivery- Evaluating model performance, testing assumptions...SeniorFull timeH1bPwC
Seattle, WA1 day ago- Systems Planning and Analysis, Inc. seeks a Testing and Evaluation AI Subject Matter Expert to provide hybrid support for DHS customers in Washington, DC. The role focuses on integrating AI T&E into DHS acquisition programs, developing robust evaluation methods, and mentoring...SeniorTemporary work
$184.7k - $324.8k
...States Machine Learning and AI Do you get excited by driving... ...impact via measurement and evaluation, for products and services... .... Work with evaluation/experimentation engineering teams to get your... ...topics) to everyone from data scientists to engineers to business partners...SeniorWork experience placementRelocationApple Inc.
Seattle, WA4 days ago- ...Fuel Talent LLC is seeking a Senior Software Engineer to design and build the foundational architecture for AI agent systems in a hybrid Seattle, WA environment. You will shape frameworks, SDKs, APIs, and the execution infrastructure that enables agents, tools, and services...Senior
Fuel Talent
Seattle, WA1 day ago - ...Amazon is seeking a Sr Software Development Engineer for AI Agent Identity & Governance within the AWS Applied AI Solutions Core Services team. You will help build foundational AI building blocks used across AWS applications, owning an area such as agent identity and...Senior
Amazon
Seattle, WA1 day ago - ...and post-training experiments, optimize models for edge and cloud targets, and ship telemetry-driven improvements. You'll design robust evaluation suites, scale data pipelines, and collaborate across engineering to align research with product needs. #J-18808-Ljbffr...Senior
Noösphere
Seattle, WA22 hours ago $77k - $202k
...LevelSenior AssociateJob Description & SummaryThe OpportunityAs an AI & GenAI Data Scientist-Senior Associate you will leverage advanced analytics,... ...proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and...SeniorFull timeH1bShift workPwC
Seattle, WA2 days ago$180k - $225k
...Remitly, Inc. in Seattle is seeking a Senior AI Native Software Engineer to orchestrate AI agents in building complex production-grade systems. You'll lead efforts in rebuilding legacy systems and guide teams on effective AI tools. The ideal candidate has proficiency in...SeniorRemitly
Seattle, WA22 hours ago- ...P2S|CMTA is seeking a Commissioning Agent in Seattle to evaluate and optimize building system performance. The role requires hands-on testing, design reviews, and commissioning planning across MEP systems. Qualified candidates have 5+ years in mechanical design for commercial...SeniorWork at officeRemote work
P2S
Seattle, WA2 days ago $159.75k - $255.6k
...matter.Your ImpactWe are seeking a skilled and innovative Senior AI Research Scientist to join a new team focusing on agentic video and... ...definition and data strategy through model development, evaluation, deployment, and iteration in production environments.Experience...SeniorWork experience placementWork at officeRemote workAxon
Seattle, WA2 days ago- ...Smartsheet is hiring Sr. Software Engineer II - AI Engineering (Remote Eligible). You will architect and deliver production-grade AI agents and workflows, partnering with Engineering and AI Platform teams across cloud environments. You will mentor engineers, define best...SeniorRemote work
Jobleads-US
Bellevue, WA3 days ago $146.88k - $220.32k
...team at the Allen Institute for AI, part of AI for the Planet.... ...infrastructure, security, evaluation, user-interfaces, and monitoring... ...that want to run AI agents against their own systems. They... ...Experience building evaluation or experimentation systems. Small or growth...SeniorContract workWork at officeFlexible hoursWeekend workThe Allen Institute for Artificial Intelligence
Seattle, WA1 day ago- ...At Atlassian is seeking a Senior Machine Learning Engineer to join the... ...organization. You will build AI-native experiences, agentic systems, models, evaluation frameworks, and data platforms... ...definition and data development through experimentation, deployment, and continuous...Senior
Atlassian
Seattle, WA3 days ago - Senior Software Engineer, Agent FrameworksLocation: Seattle, WA | HybridCompensation: Competitive base salary... ...architecture behind sophisticated AI agent systems. You’ll design the frameworks... ...AI, developer infrastructure, evaluation, and production systems.What You’ll DoDesign...Senior
Fuel Talent
Seattle, WA3 days ago - ...engineering, model alignment, and intelligent agent systems. Beyond model serving, we operate... ...in Large Language Models and AI Agent systems to join our dynamic team.We... ...graduation date clearly in your resume. Design evaluation systems for LLM-based agents, covering...Internship
ByteDance
Seattle, WA2 days ago $264k - $379.5k
...usher in this new era, we seek AI-native thinkers across every... ...and move with an experimental mindset — who rapidly test emerging... ...Snowflake product is developed, evaluated and supported. The job is no... ...codebases are built upon.Act as an agent of clarity in a space that is...SeniorSnowflake
Bellevue, WA4 days ago$110k - $220k
...summaryAs a Principal Data Scientist at Walmart, you will define... ...data science roadmap for the experimentation platform that powers... ...large-scale data systems, and AI evaluation. You will own the scientific... ...platform investment.Serve as the senior technical advisor to...Full timeTemporary workPart timeWalmart
Bellevue, WA2 days ago- ...Description About the Role \n We are looking for a Senior Data Scientist - Experimentation & Causal Inference to serve as the statistical brain... ...engineers can implement your methods in production. \n AI-Native Workflow: Strong AI Sense — extensive hands-on...Senior
ByLabs
Seattle, WA11 days ago - ...Senior Principal Technical Program Manager - Agent InfrastructureEngineering | Seattle, United States | Remote, RemoteAtlassians... ...of our core Conversational AI & Agent Platforms.In this role,... ...ecosystems).Familiarity with offline LLM evaluation harnesses, synthetic benchmarking,...SeniorWork at officeRemote work
Trello
Seattle, WA1 day ago $191k - $253k
...by Lattice OS, an AI-powered operating... ...are looking for a Senior AI Infrastructure... ...required to train, evaluate, host, and serve complex... ...vision, and RL agents) across cloud... ...with AI Research Scientists and Platform Engineers... ...orchestration, and experimentation infrastructure to...SeniorFull timeWork experience placementImmediate startAnduril Industries
Seattle, WA1 day ago$266.72k - $350.07k
...The Enterprise Data & AI organization is a strategic... ...Applied AI/ML Scientist, you define and drive... ...portfolio. You guide senior leaders on where and how... ...ROIEstablish enterprise experimentation and validation standards... ...AI systems, evaluation frameworks, and responsible...Permanent employmentFull timePart timeWork visaLululemon Athletica
Seattle, WA4 days ago$302.4k - $378k
...Senior / Staff Machine Learning Research Scientist, Agents About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale... ...(AGI), and building upon our prior model evaluation work with enterprise customers and governments...SeniorFull timeScale
Seattle, WA3 days ago$293.5k
...Information Job Title Expert Senior Manager, AI Engineering Job ID 10... ...— from research and experimentation to production deployment... ...AI systems, including multi-agent workflows and production-grade... ...model experimentation, and evaluation design to production system...SeniorPermanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per weekBain & Company
Seattle, WA4 days ago- ...seeking a seasoned Flight Test Engineer to plan and conduct ground, flight, and maintenance test flights for experimental and transport aircraft. You will evaluate performance, document findings, and contribute to flight manuals and reports. The role requires military...SeniorRelocation
$176.76k - $232k
...The Enterprise Data & AI team is a strategic and... ...responsibilities As a Senior AI/ML Engineer, you... ...practices, implement rigorous evaluation frameworks, and design... ...partner with applied scientists, software engineers,... ..., model training and experimentation infrastructure,...SeniorPermanent employmentFull timeContract workPart timeWork visaLululemon Athletica
Seattle, WA6 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Scientist Agent Evaluation & Experimentation. Be the first to apply!
Related searches