Senior Principal Software Engineer - LLM Engineering

Salary: $61,000 - 101,000 per year Requirements:

• Formal training or certification in software engineering concepts, with at least 10 years of applied experience.
• At least 8 years of AI/ML engineering experience, including substantial expertise with LLMs, GNNs, and architectures such as GPT, Llama, Falcon, and Mistral.
• Proven experience architecting and deploying LLM and GNN solutions at enterprise scale on AWS services such as SageMaker, Bedrock, or EKS; experience with Azure ML or GCP Vertex AI is also valuable.
• Experience building large-scale LLM and GNN serving platforms in environments comparable to those at major technology companies.
• Hands-on experience developing LLM inference engines with Triton Inference Server and vLLM, including autoscaling, caching, and throughput optimization.
• Advanced Python skills and experience applying optimization techniques with deep learning frameworks such as PyTorch, TensorFlow, and Hugging Face Transformers.
• Deep knowledge of LLMOps and MLOps tools and practices, such as MLflow, SageMaker Pipelines, or Kubeflow, with a record of applying them at scale.
• Experience designing and scaling agentic AI-enabled development practices across teams, including governance for human review, traceability, auditability, and secure handling of sensitive information.
• Strong understanding of responsible AI adoption and controls at scale, including security, resiliency, data sensitivity, and risk-based governance; ability to advise senior leaders on safe adoption, reuse, and measurable outcomes.
• Expertise in inference optimization and distributed systems for large models, including system design, testing, and operational stability for high-throughput, low-latency enterprise AI platforms.
• Excellent communication and collaboration skills, including experience working with SRE teams on observability, incident response, and SLIs/SLOs for LLM services, and influencing technical and nontechnical stakeholders.
• Preferred: Masters or PhD in Computer Science, Engineering, or a related discipline, or equivalent experience.
• Preferred: Practical cloud-native experience with Docker, Kubernetes, and infrastructure-as-code tools such as Terraform or CloudFormation.
• Preferred: Expertise in AI/ML security, compliance, and governance in regulated environments.
• Preferred: Experience in trust and safety or fraud prevention; payments-platform familiarity is a plus.
• Preferred: Contributions to open-source LLM projects or peer-reviewed research, experience presenting at industry conferences, or leadership in technical communities.
• Preferred: Familiarity with GPU and TPU acceleration and specialized inference runtimes.
• Preferred: Experience building Java-based applications.

• Advise and lead strategy, architecture, and development of model-serving solutions for different model architectures, including LLMs and GNNs, across cloud and on-premises environments, aligning work with business outcomes.
• Set and implement MLOps and LLMOps approaches for the full model lifecycle, including training, versioning, deployment, monitoring, and governance.
• Improve inference performance across model architectures to achieve high throughput and low latency, using methods such as quantization, model parallelism, intelligent batching, and hardware acceleration.
• Establish strategy and operating standards for agentic AI-enabled engineering across a portfolio, using enterprise-authorized tools to improve delivery speed, reliability, and code quality. Set guardrails for validation, security, resiliency, and reuse across teams and functions.
• Use software development lifecycle tools, including authorized AI-assisted development and automation capabilities, to increase the value of automation at scale.
• Create durable, reusable software and platform frameworks that standardize ML engineering services and support adoption across teams and functions.
• Define best practices for automation, CI/CD, and infrastructure as code using containerization and orchestration technologies.
• Partner with data science, platform engineering, and SRE teams to put models into production on AWS, with a focus on observability, reliability, and cost efficiency.
• Lead deployment and optimization of model inference servers, including Triton Inference Server and vLLM, for high-throughput, low-latency serving at scale.
• Oversee production operations for AI workloads, including monitoring, incident response, security, compliance, and continuous improvement.
• Turn complex technical concepts and emerging trends into practical strategies; influence senior stakeholders and cross-functional partners to prioritize and deliver AI/ML capabilities with measurable business impact, while fostering diversity, opportunity, inclusion, and respect.

• Agentic AI
• AI
• AWS
• Azure
• CI/CD
• Cloud
• Docker
• GCP
• Hardware
• Support
• Java
• Kubeflow
• Kubernetes
• LLM
• MLflow
• MLOps
• PyTorch
• Python
• Security
• TensorFlow
• Terraform
• vLLM
• Model Serving

More:

We are seeking a Senior Principal Software Engineer for our Commercial & Investment Bank Trust & Safety Fraud Prevention team. In this senior technology leadership role, you will bring deep engineering expertise to agile teams and help us build and deliver secure, stable, scalable technology products while advancing our product and technology strategy. JPMorganChase serves consumers, small businesses, and corporate, institutional, and government clients, and operates across investment banking, consumer and small business banking, commercial banking, payments processing, and asset management. Our Commercial & Investment Bank serves clients in more than 100 countries. We offer competitive total rewards, with compensation based on role, experience, skills, and location; eligible roles may also receive incentives. Benefits may include healthcare, wellness services, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching, subject to eligibility. We are an equal opportunity employer and value diversity and inclusion, with reasonable accommodations available for applicants and employees. This role is subject to Section 19 of the Federal Deposit Insurance Act, and an offer is contingent on our review of criminal conviction history, including pretrial diversions or program entries.

last updated 42 week of 2026