Senior Data Engineer - Databricks/Spark/AWS

Salary: $61,000 - 101,000 per year Requirements:

• Formal software engineering training or certification, plus at least 3 years of applied data engineering experience spanning design, application development, testing, and operational stability.
• Advanced hands-on Apache Spark (PySpark) expertise for large-scale distributed processing, including building and operating production pipelines on Databricks with Delta Lake and lakehouse patterns.
• Strong knowledge of AWS data services, including S3, EMR, Glue, Lambda, Athena, and AWS storage and compute services; familiarity with data formats such as Parquet and Iceberg.
• Strong Python programming skills for data processing and application development; Java or Scala experience is a plus.
• Experience with automation and continuous delivery, including CI/CD, version control, and tools such as Git, Bitbucket, Jenkins, or Spinnaker.
• Practical experience with system design, application development, testing, and operational stability, plus advanced knowledge of agile methods, resiliency, and security.
• In-depth knowledge of financial services and related IT systems.
• Solid SQL and data modeling skills; experience with relational databases such as Oracle is a plus.
• Experience using scheduling tools such as Airflow or Autosys.
• Databricks certifications, such as Data Engineer Associate or Professional, are preferred.
• Familiarity with Generative AI and Agentic AI frameworks, including AI coding assistants such as Claude or GitHub Copilot, is preferred.
• Broader expertise in the AWS cloud platform and its services is preferred.

• Design, build, maintain, review, and debug secure, high-quality production data pipelines on Databricks using PySpark.
• Tune PySpark jobs and Databricks clusters for performance, scalability, and cost efficiency, including through partitioning, caching, and resource management.
• Develop scalable frameworks for end-to-end workforce analytics pipelines using medallion lakehouse patterns.
• Apply Delta Lake and Delta Live Tables expectations to implement data quality checks and validation.
• Establish monitoring and alerting with Databricks and AWS CloudWatch to detect ingestion issues and improve performance and throughput.
• Identify recurring operational issues and automate their remediation using Databricks Workflows and AWS-native tools.
• Use AI and Agentic AI solutions, along with AI-assisted engineering tools, to accelerate pipeline development and improve productivity and code quality.
• Deliver curated, reliable datasets to BI partners using tools such as Sigma, Tableau, and Alteryx.
• Partner with business stakeholders to understand needs, shape solutions, and create architecture and design artifacts for complex applications.
• Contribute to engineering communities of practice exploring emerging technologies and support a culture of diversity, inclusion, opportunity, and respect.

• Agentic AI
• AI
• Airflow
• AWS
• Lambda
• Alteryx
• BitBucket
• CI/CD
• Cloud
• CloudWatch
• Copilot
• Databricks
• Git
• GitHub
• Support
• Java
• Jenkins
• Oracle
• Python
• PySpark
• SQL
• Scala
• Security
• Spark
• Tableau
• Marketing

More:

We are hiring a Software Engineer III for our Employee and Experience Technology team, where you will help advance our modernization to a Databricks-on-AWS lakehouse. You will work as an experienced member of an agile team delivering secure, stable, and scalable technology in support of our business objectives. At JPMorganChase, we provide financial services to consumers, small businesses, and corporate, institutional, and government clients, and our Corporate Functions teams help our businesses, clients, customers, and employees succeed. We offer a competitive total rewards package, with compensation based on role, experience, skills, and location; eligible roles may also receive incentive compensation. Benefits, subject to eligibility, include health care, wellness resources, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We are an equal opportunity employer and value diversity and inclusion.

last updated 41 week of 2026