Principal Software Engineer

Job Overview DataRobot delivers AI that maximizes impact and minimizes business risk. As a Principal Software Engineer, you will be responsible for technical leadership and vision, leading by example as a technical contributor, shaping architecture, mentoring engineers, and influencing cross‑team roadmaps. You will build resilient, observable, and low‑intervention infrastructure software across our control plane systems. What You’ll Do Help design, develop, and optimize the inference engine that powers DataRobot’s agentic infrastructure API, ensuring large language model (LLM) serving systems are fast, scalable, and efficient. Contribute to the design and implementation of the inference engine and collaborate on a model‑serving stack optimized for large‑scale LLM inference. Partner with NVIDIA and other vendors to bring new model architectures or features (sparsity, activation compression, mixture‑of‑experts) into the engine. Optimize for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators. Build and maintain instrumentation, profiling, and tracing tooling to uncover bottlenecks and guide optimizations. Develop scalable routing, batching, scheduling, memory management, and dynamic loading mechanisms for inference workloads. Integrate with federated, distributed inference infrastructure—orchestrate across nodes, balance load, and handle communication overhead. Collaborate cross‑functionally with platform engineers, cloud infrastructure, and security/compliance teams. Document and share learnings, contributing to internal best practices and open‑source efforts when possible. What We’re Looking For 10+ years of engineering experience, with at least 5+ in infrastructure, platform, or backend systems roles. Deep expertise in Kubernetes internals and operations, including networking, scheduling, scaling, and controller patterns. Proven ability to design and build systems from scratch, making pragmatic trade‑offs along the way. Strong proficiency in modern programming languages such as Python or Go. Experience building production‑quality, reliable, and observable systems that are used across engineering organizations. Growth‑oriented mindset—driven to teach, learn, and improve systems as well as people. Experience operating across multiple cloud providers (AWS, GCP, Azure) and/or hybrid environments. Strong experience with Helm, container orchestration patterns, and CI/CD automation. Comfortable working with IaC (Terraform, Pulumi) and GitOps workflows. Ability to influence without authority and align diverse stakeholders around technical decisions. Nice to Have Familiarity with Cilium, Kyverno, KEDA, Gateway API, OPA, or similar technologies. Experience building and running multi‑tenant SaaS platforms. Exposure to on‑prem delivery models or regulated environments. Experience with performance tuning for large‑scale data or compute workloads. Past success driving infrastructure transformation or decomposing legacy systems. Experience working with GPU infrastructure for training and inference. Benefits Medical, Dental & Vision Insurance. Flexible Time Off Program. Paid Holidays. Paid Parental Leave. Global Employee Assistance Program (EAP) and more, subject to local legal requirements. Equal Employment Opportunity & EEO Statement DataRobot is a proud Equal Employment Opportunity and affirmative action employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity or expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. DataRobot is committed to working with and providing reasonable accommodations to applicants with physical and mental disabilities. All applicant data submitted is handled in accordance with our Applicant Privacy Policy. You may view the United States Department of Labor’s EEO poster and EEO poster supplement for additional information. #J-18808-Ljbffr