Staff Software Engineer, Slurm

Crusoe's mission is to accelerate the abundance of energy and intelligence. We’re crafting the engine that powers a world where people can create ambitiously with AI — without sacrificing scale, speed, or sustainability. Be a part of the AI revolution with sustainable technology at Crusoe. Here, you'll drive meaningful innovation, make a tangible impact, and join a team that’s setting the pace for responsible, transformative cloud infrastructure. About the Role: We are actively seeking an exceptional Staff Software Engineer to join our cloud software team, focusing specifically on building and operating Slurm as a fully managed cloud service within Crusoe Cloud. This role is crucial for delivering next-generation orchestration capabilities to power GPU-accelerated and high-performance computing (HPC) at scale. Your expertise will be instrumental in designing and scaling our carbon-reducing operating model, and advancing our AI training clusters to lead the industry in reliability and performance. You will shape the technical direction of systems that allow customers to run advanced workloads across CPUs, NVIDIA and AMD GPUs, and high-performance networking environments. You will be involved in writing and reviewing code, contributing to proposals, and drafting architecture documents. You will evaluate tools and frameworks, considering their impact on reliability, scalability, operational costs, and ease of adoption. What You'll Be Working On: Lead the development and engineering of our managed Slurm offering, providing a seamless experience for AI/ML and HPC customers who rely on robust Slurm job scheduling. Contribute to the development of scalable and robust software solutions, closely aligning with the strategic objectives outlined in the Crusoe Cloud roadmap. Design, build, and maintain Kubernetes operators and controllers dedicated to managing the lifecycle, configuration, and state of large-scale Slurm clusters. Drive the integration of GPU acceleration in the Slurm environment, including device plugin architecture, GPU operators, accelerator-aware scheduling, and resource allocation. Ensure that high-performance networking technologies, such as InfiniBand and RoCE, are correctly leveraged for distributed GPU workloads running through Slurm. Implement and manage features such as multi-tenancy, cluster lifecycle management, auto-scaling, and high availability for the managed Slurm control plane services. Develop scalable systems to compete with leading managed services. Support the development of your peers by sharing knowledge and providing guidance in technical discussions. What You'll Bring to the Team: You have 7+ years of experience working in software engineering, with strong experience in Systems Engineering. Experience in distributed systems, cloud, or HPC environments is a must You possess 2+ years of programming experience in GoLang . Strong proficiency in other systems languages (Rust, C++, Python for HPC tooling) is also beneficial. You have extensive experience with Kubernetes and Linux Engineering and debugging . You possess deep knowledge of Slurm (Simple Linux Utility for Resource Management) administration and the architecture required for managing compute jobs in high-performance environments. You are skilled in infrastructure as code and familiar with systems-level challenges, ideally with experience utilizing Terraform . You understand Argo, CI/CD, and Automated Testing pipelines . You can design system architecture, taking ownership of system architecture, including CI/CD pipelines, while ensuring adherence to security standards. Strong knowledge of container networking (CNI plugins, service meshes) and Linux networking fundamentals. Familiarity with GPU integration in Kubernetes, including device plugins and GPU operators. You have excellent communication skills, both verbal and written. Benefits: Industry competitive pay Restricted Stock Units in a fast growing, well-funded technology company Health insurance package options that include HDHP and PPO, vision, and dental for you and your dependents Employer contributions to HSA accounts Paid Parental Leave Paid life insurance, short-term and long-term disability Teladoc 401(k) with a 100% match up to 4% of salary Generous paid time off and holiday schedule Cell phone reimbursement Tuition reimbursement Subscription to the Calm app MetLife Legal Company paid commuter benefit; $300 per month Compensation Range Compensation will be paid in the range of $185,000 - $224,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicants knowledge, education, and abilities, as well as internal equity and alignment with market data. Crusoe is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, disability, genetic information, pregnancy, citizenship, marital status, sex/gender, sexual preference/ orientation, gender identity, age, veteran status, national origin, or any other status protected by law or regulation. #J-18808-Ljbffr