Lead Software Engineer - Big Data

Salary: $61,000 - 101,000 per year Requirements:

• We require formal training or certification in software engineering concepts, plus at least 5 years of practical experience.
• We are looking for advanced expertise in one or more languages and frameworks such as Python, Java, Scala, Apache Spark, Databricks, Grafana, Prometheus, Elasticsearch, CloudWatch, Spring Boot APIs, and containers.
• We need experience implementing low-latency, scalable data operations and supporting real-time lookups, updates, and analytics with Apache HBase and Apache Cassandra.
• We expect experience designing and building scalable ETL pipelines for structured and semi-structured data, including partitioning within Hadoop-based architectures.
• We need experience managing large-scale Iceberg data lake tables and applying automation and continuous delivery methods for real-time and batch ingestion, cleansing, transformation, and feature extraction on Spark.
• We require experience implementing ACID-compliant data operations and enabling schema evolution with Delta table structures.
• We are looking for demonstrated experience leading the use of approved AI-assisted software development tools, with the ability to set expectations for validating outputs for correctness, performance, and security.
• We need a strong understanding of responsible AI use in engineering workflows, including data sensitivity, secure handling of inputs and outputs, and compliance with resiliency and security expectations.
• We expect experience configuring and maintaining Grafana dashboards integrated with Prometheus, Elasticsearch, or CloudWatch to monitor pipeline performance, API services, and system health in real time.
• We require advanced understanding of agile practices such as CI/CD, application resiliency, and security, along with documentation of data workflows, Spring Boot API specifications, CI/CD processes, Grafana configurations, and cloud architecture using Confluence.
• We are looking for demonstrated proficiency in software applications and technical processes within a technical discipline, and broad capability across the software development life cycle.
• Preferred: in-depth knowledge of the financial services industry and its IT systems.
• Preferred: practical cloud-native experience.
• Preferred: experience creating and deploying RESTful APIs with Spring Boot in Docker containers.
• Preferred: experience managing source code workflows, version control, and collaboration using Git, GitHub, and Bitbucket.
• Preferred: experience building, deploying, and managing scalable data engineering pipelines and analytics infrastructure using Azure Data Factory, Databricks, or AWS services such as EC2, S3, EMR, Lambda, Glue, IAM, or CloudWatch.

• We design, develop, and troubleshoot creative software solutions, using non-standard approaches where needed to solve complex technical problems.
• We review, optimize, automate, and convert existing one-off data transformation pipelines into scalable discrete tasks.
• We plan, design, implement, and operate data transformation pipelines in a production environment.
• We identify and remove recurring issues through automation to improve operational stability of software applications and systems.
• We drive adoption of enterprise-approved AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes.
• We use SDLC tools, including approved AI-assisted development and automation, while partnering with internal clients and service delivery engineers to identify data needs, workflows, and solutions.
• We gather, analyze, and document detailed technical requirements and share guidance with other engineers.
• We contribute code to infrastructure, software development kits, and platforms that support bespoke data transformation pipelines and large-scale predictive models.
• We identify engineering opportunities to reduce operational effort and lower running costs for the data platform.
• We lead evaluation sessions with external vendors, startups, and internal teams to assess architectural designs, technical fit, and system compatibility.
• We lead communities of practice across software engineering to promote awareness of emerging technologies and mentor junior engineers in day-to-day development work.
• We help foster a team culture centered on diversity, opportunity, inclusion, and respect.

• AI
• API
• AWS
• Lambda
• Azure
• BitBucket
• CI/CD
• Cloud
• CloudWatch
• Cassandra
• Confluence
• Databricks
• Docker
• EC2
• ETL
• ElasticSearch
• Git
• GitHub
• Grafana
• Hadoop
• IAM
• Support
• Java
• Prometheus
• Python
• Scala
• Security
• Spark
• Spring
• Spring Boot
• Machine Learning
• Mobile
• Network

More:

We are JPMorganChase, one of the worlds oldest financial institutions, delivering innovative financial solutions to consumers, small businesses, and major corporate, institutional, and government clients through the J.P. Morgan and Chase brands. With more than 200 years of history, we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing, and asset management. We offer a competitive total rewards package that may include base salary, discretionary incentives, and other forms of compensation depending on the role, experience, skills, and location. Eligible employees may also receive benefits such as comprehensive health coverage, on-site health and wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching. We value diversity and inclusion and are committed to equal opportunity and reasonable accommodations. Our Corporate Technology team works with smart, driven people to build applications and provide technology support across corporate functions, including Global Finance, Corporate Treasury, Risk Management, Human Resources, Compliance, Legal, and the Corporate Administrative Office.

last updated 41 week of 2026