Lead Software Engineer - Databricks/AI

Salary: $61,000 - 101,000 per year Requirements:

• Formal training or certification in software engineering, with at least 5 years of applied experience
• Expert Python and PySpark programming skills, supported by a portfolio of production-grade code
• Extensive hands-on experience with Databricks and AWS services, including Glue, S3, SQS/SNS, and Lambda
• Deep expertise in Spark and SQL
• Hands-on experience with Lakehouse and Delta Lake architecture, application development, testing, and operational stability; experience with Snowflake, Terraform, and LLMs; and knowledge of data observability, data quality, query optimization, and cost optimization
• In-depth knowledge of enterprise-scale big data and data warehousing
• Extensive experience with CI/CD processes and automated testing frameworks
• Understanding of agile methodologies, DevOps practices, application resiliency, and security measures
• Understanding of agentic AI concepts, including how autonomous agents plan, reason, use tools, and execute multi-step workflows, and of the data infrastructure that supports them
• Experience building APIs, data services, and retrieval systems connecting AI agents with enterprise data
• Ability to lead through hands-on engineering, mentor colleagues, and drive team delivery
• Preferred: Experience with agentic AI frameworks such as LangGraph, AutoGen, CrewAI, or OpenAI Assistants API
• Preferred: Familiarity with tool-use and function-calling patterns for LLM-based agents, including building APIs and data endpoints for autonomous use
• Preferred: Experience with vector databases such as Pinecone, FAISS, or Chroma, and embedding workflows for agent memory, semantic search, and retrieval-augmented generation
• Preferred: Exposure to agent memory and state-management patterns, including short- and long-term memory and conversation or task history
• Preferred: Familiarity with guardrails and safety frameworks, including input/output validation, action approvals, and human-in-the-loop controls
• Preferred: Understanding of agentic-system observability and monitoring, including tracing decisions, logging tool use, and debugging multi-step workflows
• Preferred: Understanding of responsible AI principles, including autonomous decision-making, data provenance, and auditability

• Build and optimize data pipelines and workflows that provide agentic AI systems with reliable, timely access to high-quality structured and unstructured data
• Develop data retrieval and indexing layers so AI agents can search, query, and synthesize information across multiple sources
• Build and maintain APIs, data services, and function endpoints that agents use to execute tasks, retrieve data, and interact with enterprise systems
• Apply data management best practices and uphold quality, security, compliance, and governance for data used or generated by autonomous agents
• Write secure, high-quality production code following AWS best practices, and deploy through CI/CD pipelines
• Build orchestration and state-management layers for multi-step agent workflows, including memory, context persistence, and task chaining
• Write and review code regularly, conduct thorough code reviews, and raise the teams technical standards
• Mentor and guide junior and mid-level engineers through pairing, code reviews, and technical coaching
• Work with product owners, data scientists, and business stakeholders to turn business needs into production-ready agentic AI solutions
• Evaluate and adopt emerging agentic AI frameworks, tools, and data engineering practices to improve the teams capabilities

• Agentic AI
• AI
• AI Agents
• API
• AWS
• Lambda
• Big Data
• CI/CD
• Databricks
• DevOps
• Support
• LLM
• Marketing
• Python
• PySpark
• SQL
• Security
• Snowflake
• Spark
• Terraform
• Cloud
• AWS Glue
• Backbone
• RAG
• Windows

More:

We are collaborating with hackajob to connect exceptional professionals with this Lead Software Engineer opportunity at JPMorganChase, within our Corporate Sector–Global Finance team. This is a senior, hands-on technical leadership role on an agile team building data products and infrastructure for agentic AI systems. Our professionals in Corporate Functions work across areas such as finance, risk, human resources, and marketing to support our businesses, clients, customers, and employees. We are one of the worlds oldest financial institutions, serving consumers, small businesses, and corporate, institutional, and government clients across banking, transaction processing, and asset management. We offer competitive total rewards and, for eligible roles, incentive compensation, along with benefits that may include health care, wellness centers, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We value diversity and inclusion, are an equal opportunity employer, and provide reasonable accommodations for religious practices and disability-related needs.

last updated 41 week of 2026