Principal Software Engineer - Spark

Business Area: Engineering Seniority Level: Director At Cloudera, we empower people to transform complex data into clear and actionable insights. We provide cloud‑native data engineering services that enable customers to run large‑scale workflows across on‑premises and public cloud environments, leveraging technologies such as Apache Spark, Apache Airflow, and Apache Iceberg. This role is not eligible for immigration sponsorship or relocation. Responsibilities Drive the multi‑year technical roadmap and architectural vision for Cloudera Data Engineering. Gain deep technical knowledge across the data services stack, with a focus on Spark, Airflow, and Iceberg, and apply this expertise in daily work. Foster engineering excellence through technical mentorship, design reviews, and architectural guidance. Collaborate with product, engineering, and cross‑functional partners, leading the delivery of large, critical features in Cloudera’s data engineering experience. Work on large‑scale distributed systems, ranging from hundreds to thousands of nodes in production clusters. Bring passion for programming, clean coding practices, attention to detail, and a strong focus on quality. Lead the Cloudera Data Engineering experience for all customers using Spark, Airflow, and Lakehouse. Qualifications Relevant studies – BS or MS in Computer Science or related field. 10+ years of experience as a Software Engineer in the data infrastructure space. Strong understanding of at least one of the following languages: Java, Scala, C++, Python, GoLang, with a willingness to learn additional languages used by Cloudera. Deep expertise in distributed data processing systems and cloud‑native architectures. Excellent communication and collaboration skills. Experience with containerization (Kubernetes, Docker). Experience developing or using Apache Spark, Airflow, or related technologies. Experience with public cloud (AWS, Azure, GCP) and/or private cloud (OpenShift, Rancher). Open‑minded attitude, desire to learn new things, and build great products. Additional Experience (Nice to Have) Contribution to open‑source projects. Strong understanding of modern Lakehouse architectures, open table formats, and metadata/catalog services. Experience with large‑scale distributed systems design and development, including scaling, performance, and scheduling. Solid experience with at least one cloud service (AWS, Azure, GCP, OpenShift). Salary The expected base salary range for this role in California is $270,000 – $320,000. Salary will vary based on job‑related skills, experience, and location. Benefits Generous PTO policy Support for work‑life balance with Unplugged Days Flexible WFH policy Mental & Physical Wellness programs Phone and Internet Reimbursement program Access to Continued Career Development Comprehensive benefits and competitive packages Paid Volunteer Time Employee Resource Groups EEO/VEVRAA #J-18808-Ljbffr