Senior AI Systems Engineer, Evaluation & Orchestration
ServiceNow
ServiceNow seeks a senior backend/infrastructure engineer to build and operate evaluation and observability tooling for its enterprise AI agent platform. You will design runtimes for multi-turn agent scenarios, implement calibration against human labels, and drive scalable, reproducible experiments across production-scale datasets. The role emphasizes Python/Go expertise, OpenTelemetry tracing, and building non-deterministic systems that yield meaningful evaluation signals. #J-18808-Ljbffr
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior AI Systems Engineer, Evaluation & Orchestration in Mountain View, CA vacancy
$153.2k - $234.1k
..., implementing, and maintaining robust processes for evaluation and validation of our AVs. To achieve this objective,... ...Stack. In this position, you will work closely with AI/ML Engineers, Simulation Engineers, Systems Engineers, Product Managers to identify, analyze, monitor...SeniorFull timeLocal areaWork from homeFlexible hoursGeneral Motors
Sunnyvale, CA4 days ago$174k - $252k
...machines, tool-use orchestration, and reflection... ...into intent-driven AI topologies.... ...with product and engineering teams via Forward... ...advance autonomous AI systems.Collaborate... ...model deployment, evaluation, data processing)... ...advanced models. As a Senior AI/ML Research...SeniorGoogle
Mountain View, CA6 days ago$174k - $252k
...state machines, tool-use orchestration, and reflection loops,... ...into intent-driven AI topologies.Partner with product and engineering teams via Forward Deployed... ...advance autonomous AI systems.Collaborate across Google... ..., model deployment, evaluation, data processing).Experience...SeniorGoogle
Mountain View, CA5 days ago- ...General Motors is seeking a Senior Engineer for Embodied AI Scaling Foundations to measure and visualize AV model performance at scale. You will design and iterate evaluation tools across the stack, collaborating with modeling and data teams to connect signals to data...Senior
General Motors
Mountain View, CA13 hours ago $188.5k - $282.7k
...accelerates the world’s AI transformation. As AI... ...the role:We want senior engineers who have already shipped... ...’ll own your agent systems end to end: understanding... ...pipelines, automated evaluations, and defensive... ..., including tool and orchestration flows, context management...SeniorRubrik
Palo Alto, CA6 days ago- ...A leader in AI technology in Palo Alto is seeking a Senior AI Systems Performance Engineer to optimize the latest foundation models on their innovative platform. This role involves collaborating with cross-functional teams to push the performance limits of AI systems....Senior
SambaNova
Palo Alto, CA17 hours ago - ...Micron Technology, Inc. in San Jose seeks a Systems Engineer to join the DRAM Systems Engineering team. You will develop, evaluate and optimize next‑generation memory systems for AI accelerators, focusing on hardware architecture, OS interactions, and performance analysis...Senior
Micron Technology
San Jose, CA3 days ago $143k - $286k
...A leading technology company in Sunnyvale is seeking a Senior Software Development Engineer to design and develop AI systems that transform customer operations. You will leverage cutting-edge technology to create personalized customer experiences that drive efficiency...SenioreGain
Sunnyvale, CA4 days ago- ...Databricks is the Data and AI company behind the Databricks Data + AI Platform. We run the world... ...training for enterprise data. Staff Software Engineers will tackle hard problems across AI training infrastructure and guide system-level decisions that scale with growing...Senior
Databricks
Mountain View, CA14 hours ago $150.4k - $277.6k
...AI Engineer – Algorithm Evaluation & Agentic Systems How do we ensure Apple's next-generation AI products are robust, safe, and truly intelligent? Join the DAQ team to help answer that. We are seeking an AI Engineer specializing in algorithm evaluation and agentic...RelocationApple Inc.
Sunnyvale, CA4 days ago- ...Micron Technology, Inc. seeks a senior engineer for the DRAM Systems Engineering team to develop, evaluate, and optimize memory systems for AI workloads. You will explore hardware architecture, operating systems, and performance analysis for agentic AI inference. The...Senior
1000 Micron Technology, Inc.
San Jose, CA3 days ago $159.2k - $301.6k
The Opportunity We are looking for a Senior AI Systems Engineer with deep C++ expertise to help... ...infrastructure, service integration layers, evaluation systems, and reliability mechanisms... ...between product capabilities, AI orchestration systems, and backend services. Ensure...SeniorFull timeTemporary workLocal areaRemote workWorldwideAdobe Systems
San Jose, CA4 days ago- ...GEICO in Palo Alto is seeking a Senior Engineer to design, develop, and deploy large-scale distributed AI applications powering multi-channel self-service for customers. You will work with cross-functional teams to build scalable AI/ML solutions, leverage Python/Java...Senior
GEICO
Palo Alto, CA20 hours ago - ServiceNow is seeking engineers to scale and optimize a cutting-edge Generative AI product for enterprise users. You will work on the Conversation Engine, applying... ...to a probabilistic, generative conversational system at the core of Moveworks AI solutions. You will collaborate...Senior
- ...NVIDIA Corporation is seeking a senior AI tools engineer to evaluate AI tools, models, and agent harnesses within engineering workflows. You will design reference workflows, benchmark performance, and provide practical guidance for tool adoption across teams. The role...Senior
NVIDIA
Santa Clara, CA18 hours ago - Scorpion Therapeutics is seeking a Senior Software Engineer focused on developing AI systems that leverage genetic datasets to generate personalized health insights... ...RAG architectures using vector stores, and build evaluation frameworks for model quality. Responsibilities...Senior
- Intuitive Surgical, Inc. in Sunnyvale, CA is seeking a Senior Systems Research Engineer to join a forward‑looking team exploring embodied AI at the intersection of AI/ML and robotics. You will architect, develop, and integrate embodied AI applications to de-risk and demonstrate...Senior
- ...NVIDIA AI in Santa Clara is seeking a senior engineer to design and build a scalable enterprise AI platform for safe deployment and orchestration of autonomous agents. You will develop runtime safety... ...with 12+ years in distributed systems, and hands-on experience with...Senior
NVIDIA AI
Santa Clara, CA3 days ago - Google LLC is seeking a Senior Software Engineer for Infrastructure, AI and Infrastructure to help build scalable systems and texture the next generation of Google's platform. You will work across AI, infrastructure, and core software, with opportunities to influence design...Senior
$201.3k - $352.3k
...It all started when engineer Fred Luddy wrote code... ...Today, ServiceNow is the AI control tower for business... ...Otto capable: agent orchestration, including harness... ...policy design, and quality evaluation across languages; the... ...reliability of the systems behind ServiceNow Otto...SeniorWork at officeImmediate startRemote workFlexible hoursServiceNow
Santa Clara, CA3 days ago$169k - $338k
...OfficeAI/ML & Agentic Systems Technical Leadership:Architect... ...advanced agentic AI systems that can... ...handle complex reliability engineering workflows, predictive... ...implement multi-agent orchestration platforms that... ...Proven track record as a senior individual contributor...Full timeTemporary workPart timeWalmart
Sunnyvale, CA5 days ago$152k - $241.5k
...models are foundational to American AI leadership and cybersecurity, and that... ...scrutiny. Our AI Safety & Security Engineering team builds and evaluates AI-powered tooling that helps find,... ...We are looking for an Evaluation/ML-Systems Engineer to own how we measure the program...Full timeRemote workNvidia
Santa Clara, CA6 days ago$178k - $321k
...an early but working AI-native capability: a multi... ...agentic runtime and evaluation harnesses that make... ...This is a two-person engineering team: you deploy, debug... ...build production-grade systems end to end on what exists... ...runtime and harness: orchestration, multi-model routing...SeniorOKX
San Jose, CA5 days ago$137.2k - $206.6k
...Overview We're looking for a Senior AI Engineer to help bring pragmatic,... ...grade AI capabilities into the systems that power Joby's... ...model gateways, prompt/agent orchestration patterns, feature/embedding... ...across internal systems Develop evaluation strategies for LLM and ML...SeniorTemporary work$231k - $378k
...Role Is As a Principal AI Engineer on LinkedIn Search,... ...execution, tool/API orchestration, and conversational search... ...architect the AI systems that turn static search... ...: Build rigorous evaluation harnesses and guardrails... ...AI Engineer and senior-level member of technical...For contractorsWork at officeFlexible hoursShift workLinkedIn
Mountain View, CA2 days ago$185k - $215k
...Job Description In this role, you will operate as a Staff AI Systems Engineer, setting technical direction across multiple teams,... ...this role, you will guide the design, implementation, and evaluation of our Advanced Driver Assistance Systems (ADAS) and Autonomous...Work experience placementLocal areaWorldwideBosch Group
Sunnyvale, CA2 days ago- ...Capital One is seeking an AI Engineer 4 to join the Intelligent Foundations and Experiences team in San Jose, CA. You will contribute to AI systems that power real-time banking experiences, partnering with engineers, researchers, and product managers to build scalable...Senior
Capital One
San Jose, CA13 hours ago - ...Stanford University’s Enterprise Technology team seeks an experienced AI/GenAI engineer to design, implement, and support AI-driven solutions across university use cases. You will prototype, productionize, and mentor while coordinating with project managers and architects...Senior
SLAC Inc
Redwood City, CA15 hours ago - ...Advanced Micro Devices (AMD) is seeking a senior Software Systems Design Engineer to lead HPC and AI solutions on AMD EPYC platforms. You will optimize software paths and enable scalable deployments across clusters, collaborating with product, marketing, and silicon teams...Senior
Advanced Micro Devices , Inc.
Santa Clara, CA17 hours ago - ...ServiceNow is hiring for Moveworks' AI agent evaluation platform team in Mountain View. You will design... ...8+ years building production backend systems, strong Python or Go, and experience with distributed systems, orchestration, and OpenTelemetry observability. #J-1880...
ServiceNow
Mountain View, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Systems Engineer, Evaluation & Orchestration. Be the first to apply!
Related searches
- machine learning ai engineer Mountain View, CA
- ai developer Mountain View, CA
- ai ml engineer Mountain View, CA
- ai prompt engineer Mountain View, CA
- senior ai engineer Mountain View, CA
- ai research engineer Mountain View, CA
- ai engineer Mountain View, CA
- senior windows systems engineer Mountain View, CA
- ground systems engineer Mountain View, CA
- system performance engineer Mountain View, CA