Lead AI Engineer (LLM Inference)

Salary: $193,400 - 240,800 per year Requirements:

• Bachelors degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or a related discipline plus a minimum of 4 years of experience in developing AI and ML algorithms or technologies, or a Masters degree with at least 2 years of relevant experience
• Minimum of 4 years of programming experience in Python, Go, Scala, or Java
• 6 years of experience in deploying scalable and responsible AI solutions on cloud platforms (e.g., AWS, Google Cloud, Azure, or similar private cloud)
• Experience in designing, developing, delivering, and supporting AI services
• Knowledge of developing AI and ML algorithms or technologies (e.g., LLM Inference, Similarity Search, VectorDBs, Guardrails, Memory) using Python, C++, C#, Java, or Golang
• Familiarity with developing and applying advanced techniques for optimizing training and inference software to enhance hardware utilization, latency, throughput, and cost
• Passion for keeping updated with the latest AI research and applying innovative techniques in practical scenarios

• Collaborate with a multidisciplinary team including engineers, research scientists, technical program managers, and product managers to deliver AI-driven products that transform associate workflows and customer interactions with Capital One
• Design, develop, test, deploy, and provide support for AI software components, including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability
• Utilize a diverse stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and others
• Innovate and implement cutting-edge LLM optimization methods to enhance the performance - scalability, cost, latency, throughput - of large-scale production AI systems
• Contribute to shaping the technical vision and long-term roadmap for foundational AI systems at Capital One

• AI
• AWS
• Azure
• C#
• Cloud
• Golang
• Hardware
• Support
• Java
• LLM
• Machine Learning
• PyTorch
• Python
• Scala

More:

At Capital One, we are pioneers in developing dependable AI systems that are transforming the banking experience for the better. Our commitment to leveraging machine learning facilitates real-time, personalized interactions for our customers. With a robust technological foundation and a talented team, we aim to lead the industry in AI applications. Our Intelligent Foundations and Experiences (IFX) team plays a crucial role in implementing our AI vision, working closely with partners to advance scientific and engineering capabilities that deliver meaningful value to our customers. We invite you to join us in redefining how we serve our clients through innovative AI and machine learning solutions.

last updated 33 week of 2026