Senior AI Application Engineer

At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying. Want to learn more about life at Klaviyo? Visit klaviyo.com/careers to see how we empower creators to own their own destiny.At Klaviyo, the ARIA (Applications, Resources, Intelligence, and Automations) team is building the AI systems that transform how our employees work. We are looking for a Senior IT App Engineer to design, build, and scale production-grade LLM integrations and AI-powered automation tools from the ground up. This role sits at the intersection of applied AI research and full-stack software engineering. You will own everything from LLM integration architecture and RAG system design to CI/CD pipeline management and production observability. If you want to build AI that real people use at scale every day, this is the role for you. How you’ll make a difference: Design and build LLM-powered applications: Architect and develop production-grade systems leveraging large language models (GPT-4, Claude, Mistral, Llama) for internal automation, knowledge retrieval, and intelligent workflow assistance. Build and maintain AI integration pipelines: Design robust, scalable API integrations between AI models and internal systems such as HRIS, ticketing, CRM, and knowledge bases, ensuring reliability, low latency, and high availability. Develop full-stack AI features end-to-end: Own the complete development of AI features across the stack from backend model orchestration and API layers to frontend interfaces that make AI accessible to non-technical employees. Architect and manage CI/CD pipelines for AI systems: Build and maintain automated deployment pipelines for AI models and services, including model evaluation frameworks, A/B testing infrastructure, and safe rollout strategies. Implement RAG and retrieval systems: Design and build retrieval-augmented generation systems grounded in Klaviyo internal knowledge, ensuring accuracy, relevance, and up-to-date responses at scale. Evaluate and fine-tune AI models: Lead benchmarking, evaluation, and fine-tuning of foundation models for Klaviyo use cases, including prompt engineering and parameter-efficient fine-tuning techniques. Establish AI engineering best practices: Define coding standards, testing frameworks, and architectural patterns for AI development at Klaviyo, enabling the team to ship reliable AI features consistently. Build AI observability and monitoring tooling: Instrument AI systems with latency monitoring, output quality evaluation, hallucination detection, cost dashboards, and production alerting. Collaborate with Product and Data Science: Work closely with PMs, data scientists, and ML engineers to translate requirements into technical implementations, maintaining a tight feedback loop between experimentation and production. Drive security and compliance in AI systems: Ensure AI integrations follow