Senior AI Systems Engineer, Evaluation & Orchestration
ServiceNow
ServiceNow seeks a senior backend/infrastructure engineer to build and operate evaluation and observability tooling for its enterprise AI agent platform. You will design runtimes for multi-turn agent scenarios, implement calibration against human labels, and drive scalable, reproducible experiments across production-scale datasets. The role emphasizes Python/Go expertise, OpenTelemetry tracing, and building non-deterministic systems that yield meaningful evaluation signals. #J-18808-Ljbffr Servicenow
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Senior AI Systems Engineer, Evaluation & Orchestration in Mountain View, CA vacancy
- ...MetAntz, a fast-moving AI startup in Palo Alto, seeks experienced software engineers who thrive in ambiguity and want to own end-to-end services from design to production... ...frontends, backends, and cloud-native distributed systems, delivering reliable APIs and scalable...Senior
MetAntz
Palo Alto, CA4 days ago - A leader in AI technology in Palo Alto is seeking a Senior AI Systems Performance Engineer to optimize the latest foundation models on their innovative platform. This role involves collaborating with cross-functional teams to push the performance limits of AI systems....Senior
- CoreWeave is looking for a Senior Software Engineer II to enhance our Kubernetes-native platform for AI workloads. You will design and manage Kubernetes services while mentoring junior engineers. Ideal candidates will have 5-8 years of experience in software engineering...SeniorRemote workFlexible hours
$152k - $241.5k
...models are foundational to American AI leadership and cybersecurity, and that... ...scrutiny. Our AI Safety & Security Engineering team builds and evaluates AI-powered tooling that helps find,... ...We are looking for an Evaluation/ML-Systems Engineer to own how we measure the program...SuggestedFull timeRemote work$178k - $321k
...an early but working AI-native capability: a multi... ...agentic runtime and evaluation harnesses that make... ...This is a two-person engineering team: you deploy, debug... ...build production-grade systems end to end on what exists... ...runtime and harness: orchestration, multi-model routing...Senior- ...advance their careers. Job Title: Senior Applied AI Engineer Agentic Systems Location: Mountain View, CA... ...workflow agents, multi-step task orchestration, tool-calling loops, and human-in... ...software demands Design evaluation harnesses to continuously test...Senior
NMK GLOBAL, INC
Mountain View, CA1 day ago - ...JPMorgan Chase & Co. is seeking a Senior Director of Software Engineering within the Commercial and Investment... ...technical areas and align across Data and AI domains. You will drive adoption... ..., and deliver scalable, resilient systems across complex trading and financial...SeniorBank staff
- Google is seeking a Senior Hardware Engineer in the Board and System Design team to develop ML/AI hardware for data centers. You will collaborate with ASIC, FPGA, software, and verification teams to ensure designs meet requirements and scale to volume manufacturing. You...Senior
- NVIDIA in Santa Clara, CA is seeking a Senior Software Engineer for Agent Simulation & Evaluation to advance AI workloads across GPUs and LPUs. You will build agentic systems, develop evaluation tools, and collaborate with inference, hardware, and product teams. Required...Senior
- ...leader in enterprise orchestration, Workato helps businesses... ...and experiences. Its AI-powered platform... ...exceptional Search/AI Engineer with experience in Search... ...of intelligent search systems that leverage machine... ...Own the design of evaluation frameworks for offline...Full timeRemote workFlexible hours
Workato
Palo Alto, CA13 hours ago $200k - $400k
...machines at scale. At Scout AI, we’re developing Fury,... ...We're looking for a Senior or Staff AI Engineer to join the Fury Orchestration Team with a deep... ...directly influence how our system reasons, plans, remembers... ...: model training and evaluation, reinforcement learning...SeniorFull timeRelocation packageScout Ai
Sunnyvale, CA13 hours ago$160k - $198k
...and celebrates all of our team members.What You’ll DoAs a Senior AI Systems Engineer, you will architect, deploy, and manage the critical... ...raw data and high-performance AI models.Infrastructure & Orchestration: Deploy, scale, and manage resilient infrastructure services...SeniorLocal area$144k - $236k
...needs of the team. The Video AI team sits at the heart of... ...intelligent content recommendation systems, positioning our company as... ...one billion members. As a Senior AI Software Engineer, you will own end-to-end... ...-art models, designing and evaluating online experiments, to...SeniorFor contractorsWork at officeFlexible hours$227.5k - $300k
...the transformation to AI-enabled software-... ...We are seeking a Senior Staff AI Engineer with a combination of... ...scalability of the resulting system. Responsibilities:... ...of agentic orchestration by designing and implementing... ...engineering, and model evaluation techniques. ~...SeniorFull timeWork at officeWorldwideFlexible hoursShift workSonatus
Sunnyvale, CA13 hours ago$190.9k - $334.1k
...DescriptionIt all started when engineer Fred Luddy wrote code... ..., ServiceNow is the AI control tower for... ...-scale search systems that power Now Assist,... ...multi-agent systems — orchestration, tool use, planning loops... ...search, re-ranking, and evaluation — as a critical dependency...SeniorWork experience placementWork at officeImmediate startRemote workFlexible hours$185k - $215k
...this role, you will operate as a Staff AI Systems Engineer, setting technical direction across... ...guide the design, implementation, and evaluation of our Advanced Driver Assistance... ...distributed systems, cloud infrastructure, or orchestration tools (e.g., AWS, Azure, GCP, Airflow...Odd jobWork experience placementInternshipLocal areaWorldwide$139k - $257.55k
...consistency, reusable design systems, and collaboration... ...next generation of AI-native creative systems... ...for forward-thinking engineers who are excited to explore... ...experience.Develop evaluation and quality frameworks... ..., structured outputs, orchestration frameworks, multimodal...SeniorFull timeTemporary workLocal areaWorldwide$163.2k - $220.8k
...growth. Wilson Sonsini is looking for a Senior AI Security Engineer to join the Security Operations team.... ...law firms builds and deploys AI systems with security and trust at their core... ...function: designing secure AI pipelines, evaluating MCP server integrations and agentic...SeniorFull timeWork experience placementRemote workWorldwideShift work$168k - $264.5k
...architecture, silicon, systems, and manufacturing. The... ...performance loss.We're hiring a Senior Manufacturing & System Co-Design Workflow Engineer to lead the methodology... ...-loop checkpoints, and evaluation-based CI gates running... ...that makes AI genuinely usable in a rigorous...SeniorFull timeImmediate start$226.15k - $252k
...audio logic within the orchestration framework.Guarantee... ...production by triaging system issues, debugging code... ...in Computer Science, Engineering, Computer Information... ...for model deployment, evaluation, and optimization; Root... ..., we are a pioneering AI lab with exceptional interdisciplinary...SeniorFull timeTemporary workWork at office$129.4k - $198.4k
.... About the Organization:The Evaluation team builds and evolves the evaluation... ...with Autonomy, Simulation, Systems, and Safety teams, we act as... ...interpretable insights to engineering teams and leadership,... ...testing best practices. Leverage AI-assisted development tools and...SeniorFull timeLocal areaRemote workWork from homeFlexible hours$144.7k - $221.4k
.... About the Organization The Evaluation team builds and evolves the evaluation... ...with Autonomy, Simulation, Systems, and Safety teams, we act as... ...into clear feedback for engineering and leadership, and help... ...systems. Experience leveraging AI-assisted development and analytics...SeniorFull timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours$182.4k - $250.6k
...breakthrough hardware and battery systems to intuitive design,... ...? Join the Embodied AI team at General Motors. Our... ...real-world scenarios.As a Senior AI/ML Future Sensing Engineer in the Embodied AI organization... ..., you will develop and evaluate machine learning solutions...SeniorFull timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours$190k - $260k
...an artificial intelligence (AI) powered technology stack purpose... .... We are looking for engineers who make model training fast:... ...throughput data loading and streaming systems for multimodal sensor data (... ...applications. As part of the evaluation process, we provide Endorsed...SeniorTemporary workWork at officeVisa sponsorship- ...new era in enterprise AI — one defined not by model... ..., memory systems, and routing intelligence... ...research and production engineering — investigating the foundational... ...stalls.You Are:As a Senior Advanced AI Research... ...benchmarking and evaluation methodologies scoped to...SeniorFull timeWork experience placementLive inWork at officeLocal areaRelocation
- SpaceXAI is seeking a hands-on Engineer to ensure engineering teams get high-quality data for training and evaluation. You will craft data projects, partner with model teams, and drive data strategy at the intersection of engineering and data operations. This role focuses...
$204k - $259k
...Waymo Planner Evaluation Team Software EngineerWaymo is an autonomous driving technology company... ...for experienced data-minded software engineers and data scientists to help us improve... ...areaStrong self-motivation to navigate complex systems and pursue open-ended problems to...SeniorFull timeRemote workLatent Logic
Mountain View, CA1 day ago- ...technologies—like the da Vinci surgical system and Ion—have transformed how... ...worldwide.We’re a team of engineers, clinicians, and innovators... ...robotic platforms. As a Senior AI/ML Research Engineer (Computer... ...: designing, training, and evaluating models that extract anatomy,...SeniorLocal areaWorldwideFlexible hours
- Google in Mountain View, CA is seeking a software engineer to work on next‑generation technologies powering billions of users.... ...information retrieval, distributed computing, large‑scale system design, and AI across multiple Google Ads teams, with opportunities to switch...Senior
$184k - $287.5k
...tapping into the unlimited potential of AI to define the next era of computing. An era... .... We are looking for talented Software Engineers to work on developing and deploying an AI... ...of experience building AI-driven software systems, ideally applied to complex engineering workflows...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Systems Engineer, Evaluation & Orchestration. Be the first to apply!
Related searches
- ai ml engineer Mountain View, CA
- machine learning ai engineer Mountain View, CA
- ai developer Mountain View, CA
- ai engineer Mountain View, CA
- ai prompt engineer Mountain View, CA
- senior ai engineer Mountain View, CA
- senior windows systems engineer Mountain View, CA
- software system engineer Mountain View, CA
- mission system engineer Mountain View, CA
- healthcare systems engineer Mountain View, CA