Lead Software Engineer - Data Engineer, Regulatory Reporting

Salary: $61,000 - 101,000 per year Requirements:

• Formal training or certification in software engineering concepts, with 5+ years of applied experience
• 5+ years building production data engineering and/or software engineering solutions across design, development, testing, and operations
• Hands-on experience designing, developing, testing, and operating large-scale data pipelines, including maintaining their operational stability
• Advanced programming ability, including strong proficiency in Python and hands-on PySpark experience
• Advanced Spark SQL skills and strong SQL fundamentals, including data modeling, query optimization, and execution plan analysis
• Experience leading the effective use of approved AI-assisted software development tools for coding, code review, testing, or troubleshooting, and setting standards to validate outputs for correctness, performance, and security
• Understanding of responsible AI use in engineering, including data sensitivity, secure handling of inputs and outputs, and resilience and security expectations; experience coaching engineers on safe, compliant adoption
• Experience with AWS data management patterns, including S3 and AWS Glue Data Catalog for metadata governance, schema hygiene, and discoverability; experience with another cloud data platform may also be considered
• Experience delivering and operating Spark workloads on EMR and/or Databricks, including tuning, troubleshooting, monitoring, and cost and performance optimization
• Production experience with Apache Iceberg, including table design and operations such as partitioning, file layout optimization, schema evolution, compaction, small-file mitigation, snapshot retention, metadata maintenance, safe backfills and rewrites, and reprocessing
• Proficiency in CI/CD, automated testing, and repeatable deployment methods for data pipelines
• Familiarity with Kafka, including topic design, producer and consumer patterns, schema compatibility, and operational considerations, is a plus
• Experience with Delta Lake concepts and trade-offs compared with Iceberg
• Experience with Spark Structured Streaming and streaming ETL patterns
• Working knowledge of Java for interoperability or use of existing JVM-based components
• Experience with AI-assisted engineering tools and workflows, such as GitHub Copilot or Claude, including spec-driven development, prompt-assisted refactoring, and code review using enterprise-safe practices

• Design, develop, and troubleshoot creative software solutions, focusing on data engineering and Spark-based ETL/ELT
• Build secure, high-quality production code in Python, PySpark, and Spark SQL, and review and debug others code and data workflows end to end
• Advance the teams use of enterprise-authorized AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes; establish validation standards for secure coding, peer review, and automated testing, and encourage reuse of effective approaches
• Apply software development lifecycle tools, including approved AI-assisted development and automation capabilities, to increase the value of automation
• Find opportunities to prevent or automate recurring issue remediation, improving application and system stability, data pipeline reliability, and lakehouse maintenance
• Lead evaluations with vendors, startups, and internal teams, assessing architecture, technical capabilities, and suitability for existing systems and information architecture, including EMR or Databricks, lakehouse formats, and catalog and governance approaches
• Lead software engineering communities of practice to promote new technologies and practices, particularly Spark performance, Iceberg, and data platform operations
• Contribute to a team culture grounded in diversity, opportunity, inclusion, and respect

• AI
• AWS
• AWS Glue
• CI/CD
• Cloud
• Copilot
• Databricks
• ETL
• GitHub
• Support
• JVM
• Java
• Kafka
• Python
• PySpark
• SQL
• Security
• Spark

More:

We are hiring a Lead Software Engineer for JPMorganChases Commercial & Investment Bank Regulatory Reporting Team. You will be a core technical contributor in an agile team building and improving secure, stable, scalable technology products in support of the firms business objectives. JPMorganChase serves consumers, small businesses, and corporate, institutional, and government clients, and operates across investment banking, banking, payments, transaction processing, and asset management. Our Commercial & Investment Bank serves clients in more than 100 countries across banking, markets, securities services, and payments. We offer a competitive total rewards package, with compensation based on role, experience, skills, and location; eligible roles may also include commission or discretionary incentives. Benefits, subject to eligibility, include health coverage, wellness centers, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We value diversity and inclusion, provide reasonable accommodations, and are an equal opportunity employer.

last updated 42 week of 2026