Principal Site Reliability Engineer (Intelligent Automation)

Role Overview Within the CS CoE, the Solutions team develops interconnected computational and data ecosystems. As a Site Reliability Engineer in the Solutions Engineering capability, you will design, implement, and maintain scalable, resilient, and supportable cloud‑based platform solutions that enable research applications, machine‑learning (ML) workloads, and high‑performance computing (HPC) environments. Responsibilities Infrastructure as Code (IaC) Design and Implementation Architect and implement IaC solutions using Terraform, Spacelift, or CloudFormation to provision and manage cloud infrastructure for ML and HPC workloads. Automate the deployment of scalable ML pipelines, HPC clusters, and supporting services across global regions. Global Availability and Resiliency Build resilient, highly available solutions for ML and HPC workloads leveraging cloud‑native practices such as auto‑scaling, load balancing, and failover mechanisms. Implement disaster recovery (DR) and business continuity plans for critical systems to ensure global operational integrity. Conduct chaos engineering experiments to validate system reliability and identify potential weaknesses. Automation and Observability Develop automation scripts and workflows to streamline infrastructure management, deployment, and scaling for ML and HPC use cases. Implement robust monitoring, logging, and alerting frameworks using Prometheus, Grafana, Datadog, or ELK Stack to provide deep insights into system health and performance. Apply AIOps incident management processes and tooling. Collaboration and Leadership Provide technical leadership to a team of engineers, fostering a culture of collaboration, innovation, and continuous improvement. Partner with cross‑functional teams to align infrastructure solutions with business objectives and ML/HPC workload requirements. Mentor and train junior engineers in IaC practices, ML, and HPC infrastructure design. Cost Optimization and Governance Monitor and optimize cloud infrastructure usage and costs for ML and HPC workloads. Ensure compliance with organizational security, governance, and regulatory policies in all IaC and cloud implementations. Qualifications Bachelor’s or Master’s degree in Computer Science or a related technical field, or equivalent experience. 7+ years of experience in software engineering Site Reliability Engineering (SRE). Proven expertise in supporting and deploying IaC solutions in cloud environments (AWS, Azure, or GCP) for ML and HPC workloads. Background in MLOps pipelines, including model versioning, CI/CD for ML, and feature store integration, with experience using managed ML services such as AWS SageMaker, Google AI Platform, or Azure ML. Deep understanding of cloud‑native architectures, including autoscaling, serverless, and multi‑region deployments. Technical Skills Advanced proficiency with IaC tools: Terraform, Pulumi, or CloudFormation. Expertise in scripting and automation: Python, Bash, or Go. Strong understanding of GPU‑accelerated computing (e.g., NVIDIA CUDA, TensorFlow) and HPC workload scaling. Knowledge of distributed systems, storage solutions, and data pipelines. Familiarity with monitoring and observability tools: Prometheus, Grafana, Datadog, or similar. Soft Skills Strong problem‑solving skills with a methodical approach to troubleshooting. Excellent communication, leadership, and mentoring abilities. Ability to work collaboratively across teams in a fast‑paced, dynamic environment. Preferred Qualifications Certifications in cloud platforms (e.g., AWS Certified Solutions Architect, GCP Professional Cloud Architect, or Azure Solutions Architect). Experience with distributed ML frameworks and data engineering pipelines such as Horovod, TensorFlow Distributed, Apache Airflow, and Apache Spark. Experience with compliance frameworks (e.g., GDPR, SOC2, ISO27001). Location & Work Onsite presence in South San Francisco is expected for at least 3 days a week. Relocation benefits are not available for this job posting. Salary Based on the primary location of California: $162,600–$302,000. Actual pay will be determined by experience, qualifications, geographic location, and other job‑related factors permitted by law. Benefits This position qualifies for benefits detailed at the link provided below. EEO Statement Genentech is an equal opportunity employer. We employ, promote, and otherwise treat all employees and applicants on the basis of merit, qualifications, and competence. The company's policy prohibits unlawful discrimination, including but not limited to discrimination on the basis of protected veteran status, disability status, and consistent with all federal, state, or local laws. If you have a disability and need an accommodation in the online application process, please contact us by completing the Accommodations for Applicants form. #J-18808-Ljbffr F. Hoffmann-La Roche AG