Principal Software Engineer

Principal Engineer, AI/ML & Cloud

Principal Engineer, AI/ML & Cloud, to serve as the top individual-contributor technical authority for our customer-facing AI capabilities and the cloud platform that powers them. This is a strategy-and-architecture role: you will define the reference architecture for Generative and Agentic AI on the platform, establish the engineering standards other teams build against, and personally deliver the hardest, highest-leverage pieces of the system. You will operate as a force multiplier across the AI/data science and platform engineering teams — shaping direction through architecture, design reviews, and hands-on mentorship rather than through people management.

The ideal candidate brings 13+ years of experience in Data Engineering and ML (including large-scale distributed systems and architecture), 3+ years of hands-on experience with Generative/Agentic AI in production, and a track record of designing and deploying production-grade AI solutions across a hybrid estate — on-prem (Oracle) and AWS Cloud. You set technical vision, make the build/buy and platform decisions that others depend on, and are trusted to be right on the questions that matter most.

Responsibilities

The position's purpose includes, but is not limited to:

• Own the end-to-end technical architecture for the AI platform — from data foundations through model serving, retrieval, and agent orchestration — and drive it from vision to production.
• Define reference architectures, engineering standards, and paved-path patterns that AI and platform teams build against.
• Make and document the high-stakes build/buy, model-selection, and platform decisions (frameworks, vector stores, inference strategy, cloud services), informing senior leadership of alternatives and trade-offs.
• Lead the analysis and design of net-new and enhanced AI capabilities for the marketing platform as per business strategy and requirements.
• Set the standard for the development environment, tooling, and AI-assisted engineering workflow across teams.
• Partner with QA, product, data science and production support to bring new or enhanced AI features to market and resolve the most complex issues.
• Establish monitoring, evaluation, cost-governance and reliability practices for production AI systems.
• Guide and de-risk platform, security, and version/patch upgrade strategy for the AI and cloud estate.

Duties and responsibilities:

• Act as the principal technical authority on AI and cloud architecture, driving alignment across developers, project managers, business analysts, data scientists and business users on conceptualizing, estimating and delivering new applications and enhancements.
• Own the definition of technical strategy and multi-quarter roadmaps for the AI platform, including scoping, sequencing, and realistic estimates for complex programs of work.
• Lead the definition, development, and documentation of the platform's technical objectives, architecture, and specifications in collaboration with internal users and departments.
• Author reference architectures and technical design documentation that define system components, the development environment, and implementation strategy for teams to execute against.
• Provide technical direction, review designs and code, and raise the engineering bar for Principal, Staff and Senior engineers — acting as a force multiplier rather than a people manager.
• Partner with the QA organization to define test strategy, quality metrics, and evaluation frameworks for AI systems, and to resolve questions about results.
• Recommend and drive architecture and business-process improvements, and clearly communicate problem/resolution processes to leadership.
• Develop and implement solutions in accordance with policies, procedures, and security/compliance requirements, and shape those policies where they need to evolve.
• Facilitate the design and implementation of new products and platform initiatives, informing senior executives of technical alternatives, risks, and trade-offs.
• Set the standard for project and architecture documentation across the practice.
• Lead root cause analysis (RCA) on the most complex, cross-cutting incidents.
• Work with cross-functional teams during critical incidents to assess, analyze, and resolve complex problems that span multiple systems and teams.
• Identify where existing engineering policies, standards and procedures require improvement and drive the process-improvement effort.
• Champion knowledge sharing — create and improve Knowledge Base articles, reference designs, and case studies, and grow the technical maturity of the organization.

Qualifications

Skill Summary

Experience & leadership

• 13+ years of experience across ML engineering, data engineering, and large-scale distributed systems, including deep architecture experience.
• 3+ years of hands-on experience designing and shipping Generative/Agentic AI systems in production.
• Demonstrated ability to operate as a Principal-level individual contributor — setting technical direction, owning architecture, and influencing multiple teams without direct people-management authority.
• Track record of being the technical decision-maker on high-stakes platform and AI initiatives.

Generative & Agentic AI

• Hands-on experience building and deploying production AI applications using LLMs at scale.
• Deep experience with RAG architectures, vector databases, embeddings, and retrieval systems in production — including chunking, indexing, and retrieval-quality optimization.
• Experience designing agentic AI systems — tool-using agents, orchestration, and multi-step reasoning workflows — with appropriate guardrails and evaluation.
• Expertise in optimizing LLM usage across a platform: model selection, prompt engineering, context strategy, fine-tuning where warranted, and inference/deployment optimization for cost and latency.
• Ability to establish LLMOps/MLOps practice: monitoring, observability, and evaluation frameworks for AI systems — performance, cost management, and output-quality/guardrail metrics.
• Active, expert use of AI coding assistants (Cursor, Claude Code, GitHub Copilot) and spec-driven AI-assisted development in daily workflow, and ability to set that standard for others.

Data & platform engineering

• Strong data engineering skills: pipeline design, data transformation, and working with large-scale datasets that feed AI systems — integrating with existing infrastructure while building new pipelines where needed.
• Depth with distributed data processing (Spark/PySpark, Databricks, Hadoop/Hive) and workflow orchestration (Airflow); experience with MLflow or equivalent for the ML lifecycle.
• Ability to design and deploy RAG systems, embeddings pipelines, and LLM-powered features on native Python infrastructure and/or Scikit-learn, XGBoost, PyTorch, TensorFlow, and FastAPI.
• Ability to define and lead migration strategies from legacy systems to modern AI and cloud architecture.

Cloud & software engineering

• Hands-on experience designing and deploying production-grade solutions on AWS — including compute and data services (e.g., EC2, S3, EMR, Glue, Lambda, Redshift) — as well as on-prem (Oracle) in a hybrid estate.
• Experience with Infrastructure-as-Code (Terraform/CloudFormation), containers and orchestration (Docker, Kubernetes), and secure, multi-account cloud design (IAM, network, and security boundaries).
• Strong software engineering fundamentals: version control, testing, CI/CD, and code review — and the ability to set these standards for teams.
• Proficiency in Python and in Linux/Shell/bash scripting to automate processes.
• Strong understanding of Disaster Recovery and Business Continuity solutions, performance tuning, and application monitoring/support of production applications with distributed teams.
• Experience with scheduling applications with complex interdependencies, and with reporting/report development on any reporting tool.

Ways of working

• Ability to work independently and set direction while integrating with, and elevating, a team.
• Proven analytical and problem-solving abilities, with the ability to anticipate and avoid problems before they occur.
• Experience mentoring and technically guiding senior engineers; comfortable being the person others escalate the hardest problems to.
• Excellent experience working with geographically and culturally diverse teams.
• Ability to write detailed technical specifications for developers, and to design, validate and oversee software test strategy.
• Deep familiarity with the full Application Development Life Cycle, able to independently lead each phase.

Education

• Bachelor's degree in Computer Science or equivalent; advanced degree or equivalent depth of experience preferred.

Preferred Knowledge Areas

Technical

• Prior experience in automotive marketing and/or CRM applications strongly preferred.
• Experience with ETL tooling (e.g., Informatica or equivalent).