Senior Principal - Databricks Platform Engineering Lead (Python)

Senior Principal Software Engineer

Join JPMorgan Chase as a Senior Principal Software Engineer where you will own the Databricks platform architecture on AWS, build Terraform/Python automation, mentor teams, and drive high-impact data innovation.

The Chief Data & Analytics Office (CDAO) at JPMorgan Chase is responsible for accelerating the firm's data and analytics journey. This includes ensuring the quality, integrity, and security of the company's data, as well as leveraging this data to generate insights and drive decision-making. The CDAO is also responsible for developing and implementing solutions that support the firm's commercial goals by harnessing artificial intelligence and machine learning technologies to develop new products, improve productivity, and enhance risk management effectively and responsibly.

As a Senior Principal Software Engineer at JPMorganChase in Corporate – AIML Data Platforms and the Chief Data & Analytics team, you will bring your deep expertise in Databricks and large-scale data engineering. In this role you will drive the architecture, design, and implementation of advanced data solutions, leveraging Databricks and related technologies to enable business insights and innovation

Job Responsibilities

• Leads the design, build, and troubleshooting of a scalable Databricks/Spark data platform and analytics workloads across cloud environments.
• Develops and maintains secure, production-grade Python code; perform code reviews, debugging, and engineering quality improvements across the team.
• Owns delivery of scalable data pipelines and analytics solutions on Databricks, applying strong distributed computing and performance troubleshooting practices.
• Designs and develops infrastructure automation for the Databricks control plane, including workspace provisioning, account configuration, onboarding/offboarding, and Delta Sharing using Terraform, AWS, and Python Lambdas.
• Develops secure, high-quality production code across Terraform modules, Python Lambda functions, and control plane services while reviewing and debugging code across the team.
• Leads the architecture of scalable Databricks platform infrastructure, including workspace automation, AWS networking, and control plane APIs enabling self-service for downstream teams.
• Define and implement best practices for data engineering, data lake architecture, and distributed computing
• Solves the companies most challenging cloud data platform problems by building innovative technical solutions around Data Lake Tools.
• Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems
• Adds to team culture of diversity, opportunity, inclusion, and respect
• Mentor and guide technical teams, fostering a culture of continuous learning and excellence in software engineering practices

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 10+ years applied experience
• Expert-level proficiency in Databricks platform administration, AWS networking, and infrastructure-as-code with Terraform
• Hands-on experience with Python application program development with use of automated unit testing
• Hands-on practical experience delivering system design, application development, testing, and operational stability
• Hands-on practical experience with terraform development and understanding of terraform enterprise
• Hands-on experience with GitHub / Bitbucket code versioning tool, Jenkins build tool and pypi / maven artifactory integrations
• Experience in managing product release lifecycle at enterprise level.

Preferred qualifications, capabilities, and skills

• Strong knowledge of distributed compute frameworks like Spark and how platform-level configurations impact workload performance and reliability
• Experience with Agile development processes, as needed (SCRUM/KANBAN) using JIRA.
• Experience in Data pipelines using Spark

About Us

JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world's most prominent corporate, institutional and government clients under the J.P. Morgan and Chase brands. Our history spans over 200 years and today we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing and asset management.

We offer a competitive total rewards package including base salary determined based on the role, experience, skill set and location. Those in eligible roles may receive commission-based pay and/or discretionary incentive compensation, paid in the form of cash and/or forfeitable equity, awarded in recognition of individual achievements and contributions. We also offer a range of benefits and programs to meet employee needs, based on eligibility. These benefits include comprehensive health care coverage, on-site health and wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, financial coaching and more. Additional details about total compensation and benefits will be provided during the hiring process.

We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.

JPMorgan Chase & Co. is an Equal Opportunity Employer, including Disability/Veterans

About the Team

Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that we're setting our businesses, clients, customers and employees up for success.