Principal AI Engineer

Salesforce is an AI CRM that values humans working with technology to drive customer success, innovation, and ethical leadership. The AI Platform Engineer will build the next‑generation ML/AI platform that supports autonomous AI agents at enterprise scale. The role sits at the intersection of platform infrastructure, agent systems design, and evaluation tooling. What You’ll Do Agent Harness & Flywheel Engineering Design and build agent harness infrastructure that wraps LLM calls, manages tool use, handles retries, enforces policy, and feeds results back into iterative improvement loops. Implement agentic loop patterns with multi‑turn reasoning, tool orchestration, memory management, and structured output handling as reusable platform primitives. Build the agent flywheel: automated pipelines that collect agent traces, surface regressions, route failures to evaluation, and close the loop from production signal back to prompt/model improvement. Own the end‑to‑end lifecycle from agent experiment to production deployment, including versioning, rollout controls, and rollback mechanisms. Sandbox & Safe Execution Build sandboxed execution environments for agent tools with isolated code execution, API calls, and file system access so agents can act without unconstrained blast radius. Design tiered autonomy models, define which actions agents can take automatically, which require human approval, and which are off‑limits and enforced at the infrastructure layer. Implement replay and dry‑run capabilities so new agent versions can be tested against real traces before going live. Agent Evaluation, Observability & Optimization Implement evaluation frameworks for agent behavior using a combination of vendor, open‑source or in‑house built tools covering task success, tool selection accuracy, trajectory evaluation, hallucination rates, latency, and cost. Build and maintain eval datasets, golden trace libraries, and regression test suites that run automatically on every agent code change. Instrument agent traces end‑to‑end (LLM calls, tool invocations, intermediate reasoning, final outputs) — surfaced in Grafana or equivalent observability tooling. Define and track agent quality metrics over time; own the signal that tells the team whether agents are getting better or worse. Drive continuous quality, latency, and cost improvements across deployed agents by closing the loop between production traces, evaluations, and agent design. Validate every optimization through A/B tests, shadow deployments, and replay against golden traces, with the eval suite gating rollout so wins are real and regressions are caught before they reach users. CI/CD & Workflow Automation Build and optimize CI/CD pipelines (GitHub Actions, ArgoCD) that cover not just code deployment but agent evaluation gates — no agent ships without passing its eval suite. Automate Docker and package builds, security scanning, and agent integration tests as first‑class pipeline steps. Design self‑healing CI patterns where agent‑based automation can diagnose and fix common pipeline failures. Tooling, Developer Experience & Architecture Build internal tools and developer self‑service interfaces that let ML engineers and data scientists iterate on agents without platform team involvement. Maintain a comprehensive view of how all platform components — infrastructure, agent harnesses, evaluation pipelines, observability — work together. Create architecture diagrams and drive long‑term platform vision; own the “how does this scale to 10×” conversation. Monitoring, Security & Reliability Establish alerting (Grafana, PagerDuty) for both traditional platform health and agent‑specific signals (error rates, tool call failures, eval score drift). Ensure all agent infrastructure adheres to security best practices: sandboxed execution, auditable traces, access controls on every tool. Participate in security reviews; own compliance for agent workloads. Qualifications 9+ years as a Platform Engineer, ML Infrastructure Engineer, or Software Engineer. Demonstrated experience building agent harness infrastructure using agentic loops, tool orchestration, structured output handling, multi‑turn conversation management. Hands‑on experience with agent evaluation frameworks like Braintrust, LangSmith, or equivalent. Strong understanding of sandboxing and safe agent execution (isolation patterns, tiered autonomy, blast radius controls). Experience with context engineering related to agent orchestration. Strong Python engineering skills for building scalable tools, automation, and platform components. Extensive experience with AWS and infrastructure‑as‑code (Terraform). Experience with CI/CD tooling, especially GitHub Actions and ArgoCD. Experience with containerization (Docker) and orchestration (Kubernetes). Experience with AgentOps concepts and production multi‑agent systems. Strong problem‑solving skills and the ability to manage multiple priorities across a complex platform. Preferred Qualifications (Bonus Points) Experience with Salesforce Ecosystem including Agentforce and Data360. Experience with unstructured databases (vector or graph databases) and RAG pipelines. Experience working with modern data platforms and real‑time processing frameworks, including cloud data warehouses (e.g., Snowflake), streaming technologies (Kafka, Flink). Benefits and Compensation Salary: Typical base salary range is $197,300 – $313,700 annually. In the SanFrancisco and NewYorkCity metropolitan areas, the range is $237,700 – $344,700 annually. Benefits include: time‑off programs, medical, dental, vision, mental health support, paid parental leave, life and disability insurance, 401(k), and an employee stock purchasing program. Additional details are available at Accommodations If you need a reasonable accommodation during the application or the recruiting process, please submit a request via the Accommodations Request Form. Equal Opportunity & Non‑Discrimination Statement Salesforce is an equal‑opportunity employer and maintains a policy of non‑discrimination with all employees and applicants for employment. Compensation offered will be based on location, job level, job‑related knowledge, and experience. Qualified applicants with arrest and conviction records are considered in accordance with the SanFrancisco Fair Chance Ordinance and the LosAngeles Fair Chance Initiative for Hiring. Recruitment and AI Tools Salesforce uses artificial intelligence (AI) tools to help recruiters assess and evaluate candidates’ resumes and qualifications throughout the recruiting process. Human selection and hiring decisions are ultimately made by Salesforce employees. #J-18808-Ljbffr 100 Salesforce, Inc.