Senior Software Engineer, Machine Learning Platform New San Francisco, CA, USA

Senior Software Engineer, Machine Learning Platform

San Francisco, CA, USA

About the role

Chime's Machine Learning Platform (MLP) team builds and operates the infrastructure, tooling, and developer experience that powers machine learning across the company. We enable data scientists and ML engineers to develop, train, deploy, and monitor models reliably and efficiently.

As a Senior Software Engineer on the Machine Learning Platform team, you will design and build scalable systems spanning traditional machine learning and emerging AI workloads, including model training, feature computation, real-time inference, foundation-model access, evaluation, and agentic orchestration. You'll work at the intersection of distributed systems, cloud infrastructure, applied machine learning, and AI product engineering.

This role focuses on creating secure, reliable, and reusable platform capabilities that help teams choose the right approach, from conventional predictive models to LLM-powered and multi-step agentic systems, while maintaining strong standards for evaluation, observability, governance, privacy, and cost efficiency.

In this role, you can expect to

• Design, build, and operate scalable ML and AI infrastructure on AWS.
• Design and operate shared platform capabilities for LLM and agentic workloads, including model access, prompt and configuration lifecycle, retrieval, tool integration, state management, and workflow orchestration.
• Build evaluation frameworks for non-deterministic AI systems, including offline benchmarks, regression testing, online quality signals, human feedback, and failure analysis.
• Establish observability, reliability, and governance for models and agents, covering traces, model and prompt versions, tool calls, latency, token usage, quality, safety, privacy, and cost.
• Help teams make principled architecture decisions across traditional ML, LLM-powered applications, and agentic workflows, and contribute to the platform’s technical roadmap.
• Build distributed training, batch inference, and large-scale processing systems using frameworks such as Ray or Spark.
• Build and maintain infrastructure as code using Terraform.
• Support and evolve the feature store and feature pipelines.
• Develop data ingestion and streaming systems using technologies such as Kinesis, Kafka, Flink, or Spark.
• Improve CI/CD workflows for ML models, AI applications, and platform components.
• Partner closely with Data Science and ML Engineering teams to improve developer experience.
• Participate in on-call rotations to support production systems.

To thrive in this role, you have

• Knowledge of the machine learning development lifecycle, including data preprocessing, model training, evaluation, deployment, and monitoring.
• Experience designing distributed systems and large-scale data or compute platforms on AWS using frameworks such as Spark or Ray.
• 5+ years of experience in ML or AI infrastructure, platform engineering, distributed systems, or production ML systems.
• Working knowledge of LLM application patterns such as retrieval-augmented generation, structured outputs, tool calling, agent orchestration, and evaluation of non-deterministic systems.
• Experience designing production systems that integrate ML or foundation models through reliable APIs, workflows, and data contracts.
• Hands‑on experience with CI/CD pipelines, DevOps practices, and infrastructure as code.
• Experience with containerization and orchestration technologies such as Docker and Kubernetes.
• Strong programming skills in Python, Go, Scala, Java, or similar languages.
• Solid understanding of software engineering fundamentals, including testing, version control, code review, and observability.

Nice‑to-have

• Experience shipping LLM-powered or agentic systems to production.
• Experience with one or more of the following: model gateways, prompt lifecycle management, retrieval or vector search, tool execution, and agent orchestration frameworks.
• Experience building evaluation, tracing, and observability capabilities for non-deterministic AI systems.
• Familiarity with managed or self-hosted foundation model infrastructure, such as Amazon Bedrock, SageMaker, or equivalent platforms.
• Experience operating GPU-based workloads and optimizing training or inference performance and cost; CUDA experience is a plus.

#LI-Onsite #LI-WW1

Below is the base salary offered for this role and level of experience. Full-time employees may also be eligible for bonus(es), competitive equity, and benefits. For commissioned roles, the base salary listed in this job description does not include incentive/variable pay. The actual base salary offered may be higher, depending on your location, skills, qualifications, and experience.

Wage Notice

$187,000 - $259,000 USD

A little about us

At Chime, we believe that everyone can achieve financial progress. We created Chime - a financial technology company, not a bank* - on the premise that core banking services should be helpful, easy, and free. Through our user-friendly tools and intuitive platforms, we empower our members to take control of their finances and work towards their goals. Whether it's starting a savings account, purchasing a first car or home, launching a business, or pursuing higher education, we're proud to have helped millions unlock their financial potential.

We're a team of problem solvers, dreamers, and builders with one shared obsession: our members. From day one, Chimers have worked tirelessly to out-hustle and out-execute competitors to bring our mission to life. Their grit and determination inspire us to work harder every day to deliver the very best experience possible. We each bring an owner's mindset to our work, refusing to be outdone and holding ourselves accountable to meet and exceed the highest bars for our teams, our company, and our members.

We believe in being bold, dreaming big, and taking risks, while also working together, embracing our diverse perspectives, and giving each other honest feedback. Our culture remains deeply entrepreneurial, encouraging every Chimer to see themselves as stewards of our mission to help everyday Americans unlock their financial progress.

We know that to achieve our mission, we must earn and keep people's trust - so we hold ourselves to the highest standards of integrity in everything we do. These aren't just words on a wall - our values are embedded in every aspect of our business, serving as a north star that guides us as we work to help millions achieve their financial potential.

Because if we don't - who will?

*Chime is a financial technology company, not a bank. Banking services provided by The Bancorp Bank, N.A. or Stride Bank, N.A., Members FDIC.

What we offer for our full-time, regular employees

• Our in-office work policy is designed to keep you connected - with four days a week in the office and Fridays from home for those near one of our offices, plus team and company-wide events depending on location. Whether you're coming in regularly or are part of our fully remote program, you'll stay engaged with your work and teammates.
• Benefits that support your work and life, including backup child, elder, and pet care and subsidized commuter benefits for eligible employees.
• Comprehensive health, financial, and wellbeing benefits designed to support you at every stage of life.
• Generous vacation policy and company-wide paid days off
• 1% of your time off to support local community organizations of your choice
• Annual wellness stipend to use towards eligible wellness related expenses
• Up to 22 weeks of paid parental leave for birthing parents and 12 weeks of paid parental leave for non-birthing parents
• Access to family planning reimbursement
• A challenging and fulfilling opportunity to join one of the most experienced teams in FinTech and help millions unlock financial progress

We know that great work