Principal AI System Architect

Principal AI Systems Architect – Agentic AI & Engineering Automation

Location: San Jose, CA (remote is okay)

A leading global technology company is investing heavily in AI-driven engineering automation and enterprise intelligence. The team is building a reusable AI platform that enables specialized agents to reason, use tools, execute complex workflows, collaborate with other agents, and operate reliably in production.

The Role

We are seeking a Principal AI Systems Architect to define and build the underlying agentic AI platform and execution infrastructure. This is a hands-on architecture role combining AI agents, distributed systems, platform engineering, AI infrastructure, and production software.

You will establish the common architecture that enables multiple engineering and business teams to develop and deploy agentic applications without rebuilding core orchestration, integration, security, and observability infrastructure for each use case.

Key Responsibilities

• Architect and implement a reusable agentic AI harness supporting planning, routing, tool use, delegation, iteration, and multi-agent coordination.
• Design durable workflow execution including state management, checkpointing, retries, concurrency, failure recovery, and asynchronous processing.
• Establish secure interfaces for agents to interact with APIs, databases, enterprise applications, engineering tools, and external services using MCP, function calling, or equivalent technologies.
• Develop model abstraction and routing across commercial, open-source, and privately hosted LLMs.
• Architect AI inference environments across Linux, containers, GPU infrastructure, private cloud, and on-premises environments.
• Define architectures for context, memory, retrieval, knowledge access, provenance, and human-in-the-loop workflows.
• Establish evaluation, guardrail, observability, and monitoring frameworks covering agent decisions, tool execution, reliability, latency, and resource utilization.
• Own security architecture spanning identity, authorization, secrets, data isolation, auditability, and controlled system access.
• Establish CI/CD, testing, versioning, deployment, rollback, and lifecycle-management standards for AI agents and applications.
• Build reusable APIs, SDKs, tool registries, templates, and development frameworks for engineering teams.
• Evaluate emerging agent frameworks and AI infrastructure technologies, making pragmatic build-vs-buy decisions.
• Remain hands-on across architecture, prototyping, implementation, debugging, and production deployment.

Requirements

• 8+ years in software engineering, distributed systems, platform engineering, or systems architecture.
• Strong hands-on Python and production software development experience.
• Proven experience architecting and deploying production AI/agentic systems.
• Deep understanding of tool-using LLMs, multi-step workflows, agent orchestration, and modern LLM architectures.
• Strong distributed-systems fundamentals: state, concurrency, messaging, idempotency, fault tolerance, and recovery.
• Experience with APIs, service integration, MCP/function calling, structured outputs, and agent-to-tool interfaces.
• Strong experience with Linux, Docker, cloud/on-prem infrastructure, and GPU-based AI workloads.
• Experience with production observability, security, CI/CD, and enterprise software deployment.
• Ability to translate architectural concepts into working, production-grade systems.

Preferred

• Experience building an internal AI, ML, developer, or agent platform used by multiple teams.
• Experience with Temporal, Airflow, Dagster, Prefect, LangGraph, LlamaIndex, Semantic Kernel, OpenAI Agents SDK, or equivalent technologies.
• Experience deploying open-weight models using vLLM, NVIDIA NIM, Triton, TensorRT-LLM, or similar inference stacks.
• Kubernetes and large-scale workload orchestration.
• Experience with AI evaluation, guardrails, vector/graph databases, knowledge graphs, or enterprise knowledge systems.
• Background in engineering software, EDA/CAD/CAE, robotics, industrial automation, or other complex technical domains.

Candidate Profile

The ideal candidate sits at the intersection of:

Agentic AI + Distributed Systems + Platform Engineering + Production Infrastructure

This is a builder-architect role. We are looking for someone capable of defining the architecture, writing the initial systems, and establishing the platform that other engineering teams will build upon.