Senior/Lead DevOps Engineer (On-Prem Infrastructure) - Remote (New York)

Our client builds and operates hybrid infrastructure, spanning on-premises GPU clusters and multi-cloud environments, for enterprise banking clients. Youll join a delivery team managing large-scale Kubernetes and cloud infrastructure for demanding, security-critical AI/ML and business workloads.

This is a US-based role (New York City, Jersey City, Lake Mary, FL, or Pittsburgh) with an anticipated base salary range of $113,120-$141,400 annually.

Responsibilities

• Manage and operate large-scale on-premises Kubernetes clusters, with hands-on ownership of CNI and CSI configuration, networking, and storage integration
• Manage operating systems across GPU nodes using NVIDIA BCM
• Optimize GPU resource allocation and model performance tuning using NVIDIA Run: AI and GPU/model parameter configuration
• Set up and manage CI/CD pipelines using GitLab CI/CD for infrastructure and application delivery
• Configure networking, firewalls, and load balancing across on-prem environments
• Ensure infrastructure meets banking-grade compliance and security standards
• Collaborate directly with client stakeholders and lead small delivery teams

Requirements

• 5+ years of experience in DevOps/Infrastructure Engineering, with deep hands-on expertise in on-premises Kubernetes
• Strong experience managing GPU compute infrastructure (NVIDIA BCM, Run: AI, or equivalent)
• Proficiency with GitLab CI/CD for pipeline automation
• Solid networking fundamentals (firewalls, load balancing, VPNs, peering)
• Experience working directly with banking or other regulated-industry clients
• Prior cloud exposure (GCP or similar) is a plus, though not required for this role
• Demonstrated LLMOps experience, including model versioning, inference optimization, production monitoring and evaluation, and ongoing lifecycle management of deployed models (required)

SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.