Lead Software Engineer - Machine Learning (AI/ML)

Salary: $61,000 - 101,000 per year Requirements:

• Formal software engineering training or certification, with at least 5 years of applied experience.
• Experience running machine learning training in cloud environments and debugging infrastructure and code issues.
• Strong Python skills and sound engineering practices, including testing, code reviews, modular design, and dependency management.
• Experience building automation and CI workflows for ML codebases, including build, test, release, deployment, and promotion processes.
• Hands-on experience with deep learning training workflows and a major framework such as PyTorch or TensorFlow.
• Understanding of training performance and stability, including data-loading bottlenecks, mixed precision, checkpointing, reproducibility, and evaluation methods.
• Experience with distributed training concepts such as DDP, FSDP, or DeepSpeed, collective communication, scaling, and bottleneck analysis.
• Ability to profile and optimize training systems across CPU/GPU utilization, memory, I/O, networking, and scheduling.
• Experience with Kubernetes fundamentals for compute-intensive workloads and AWS services such as EKS, ECR, S3, IAM, VPC, CloudWatch, and EC2.
• Experience leading the effective use of approved AI-assisted software development tools and setting expectations for validating their outputs for correctness, performance, and security.
• Understanding of responsible AI use in engineering, including data sensitivity, secure handling of inputs and outputs, and security and resiliency expectations; experience coaching engineers on safe, compliant adoption.
• Preferred: Experience running training workloads across multiple cloud platforms and managing portability, performance, and governance.
• Preferred: Familiarity with cloud-native networking and storage patterns for high-throughput training and artifact management.
• Preferred: Experience optimizing training input pipelines and working with large datasets.
• Preferred: Familiarity with distributed compute frameworks such as Spark, Ray, or Dask for feature and dataset generation.
• Preferred: Familiarity with workflow orchestration tools and model registry concepts.
• Preferred: Experience optimizing training cost and performance through right-sizing, scheduling, interruptible capacity strategies, budget guardrails, and quota planning.
• Preferred: Strong observability practices for training systems, including metrics, logs, traces, GPU telemetry, dashboards, and alert tuning.
• This position is subject to Section 19 of the Federal Deposit Insurance Act; an offer is contingent on our review of criminal conviction history, including pretrial diversions or program entries.

• Design, build, and maintain an end-to-end machine learning training platform.
• Run and optimize single-node and distributed GPU training workloads to improve throughput, utilization, and reproducibility.
• Build and operate training infrastructure on Kubernetes, including resource management and workload troubleshooting.
• Enable generative AI and LLM training and fine-tuning workflows, including evaluation, artifact and version governance, and scalable GPU execution aligned with enterprise controls.
• Establish observability for training systems through metrics, logs, dashboards, alerts, and operational runbooks.
• Work with data engineering and platform teams to define interfaces, standards, and guardrails for security, access, and cost controls.
• Improve the developer experience for training with standardized containers, CI/CD, templates, documentation, and self-service workflows.
• Encourage team adoption of enterprise-authorized AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes, while maintaining validation standards such as secure coding, peer review, and automated testing.
• Use software development lifecycle tools, including authorized AI-assisted development and automation capabilities, to increase the value of automation.

• AI
• AWS
• CI/CD
• Cloud
• CloudWatch
• EC2
• Fine-tuning
• IAM
• Support
• Kubernetes
• LLM
• Machine Learning
• PyTorch
• Python
• Security
• Spark
• TensorFlow
• NodeJS
• Airflow
• Marketing

More:

We are collaborating with hackajob to connect exceptional professionals with J.P. Morgan. As a Lead Software Engineer in our AI/ML Data Platforms group, you will join an agile team building and operating scalable, reliable ML training systems and pipelines on AWS and other cloud platforms, productionizing training workloads and improving performance and cost efficiency. JPMorganChase is a longstanding financial institution serving consumers, businesses, and institutional and government clients across banking, investment banking, transaction processing, and asset management. Our Corporate Functions teams support the success of our businesses, clients, customers, and employees. We offer competitive total rewards and, for eligible roles, incentive compensation, along with benefits that may include health care, wellness services, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We are an equal opportunity employer and value diversity and inclusion; reasonable accommodations are available for religious practices and beliefs and for mental or physical disability needs.

last updated 42 week of 2026