Senior Technical Program Manager (Engineering) - AI Tooling & Systems

Role Description

Deepgram is seeking a Senior Technical Program Manager (AI Tooling & Systems) to drive execution of large-scale ML infrastructure and AI tooling initiatives. In this role, you'll own the end-to-end delivery of programs that span model serving infrastructure, ML pipelines, internal AI tooling, and real-time inference systems—working closely with our ML engineers, research teams, and product to unlock capability at scale.

You'll thrive here if you enjoy:

• Creating clarity around complex ML system tradeoffs.
• Building tools and processes that accelerate model development and deployment.
• Partnering across research, engineering, and product to align on technical strategy and execution.

What You'll Do

• Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring.
• Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms).
• Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines.
• Drive cost and latency optimization for real-time inference workloads at scale.
• Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure).
• Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows.
• Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems.

You'll Love This Role If You

• Are passionate about building ML systems and infrastructure that powers frontier AI applications.
• Enjoy optimizing inference cost, latency, and throughput for LLM and multimodal workloads at scale.
• Love solving hard problems at the intersection of ML research and production systems (e.g., distillation, quantization, batching strategies).
• Are excited about frontier model serving technologies, vector search, and real-time ML inference.
• Want to directly enable ML researchers and engineers to iterate faster and ship better models.

Qualifications

• 5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent).
• Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer.
• Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring).
• Proven ability to translate ML/research requirements into robust, scalable infrastructure.
• Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost).
• Excellent communication with both technical and non-technical stakeholders.
• Familiarity with high-growth or startup environments.

It Would Be Great If You Had

• Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar).
• Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies).
• Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar).
• Background in feature stores, vector databases, or real-time ML systems.
• Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure.
• Experience with multi-region model serving or edge deployment.
• Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML).