Machine Learning Systems Engineer

Machine Learning Systems EngineerLocation - Palo Alto, CA (On-site) - Five days per week in-office in the Bay Area.Compensation - $200,000 – $300,000 Base + Competitive EquityVisa - Open to Visa Transfers (OPT, H1B Transfers)Company Stage - Growth Stage – $56M FundingIndustry - Artificial Intelligence, Machine Learning, Generative AI, AI Infrastructure, Developer InfrastructureOur client is building a new generation of highly efficient AI models designed to dramatically improve the speed and economics of large language model inference.The company has pioneered diffusion-based language models that generate responses in parallel rather than relying exclusively on traditional sequential token generation. This approach enables significantly faster and more efficient AI inference while maintaining competitive model quality.The company launched one of the first commercially available diffusion-based language models in early 2025 and is now deploying large-scale AI models with Fortune 500 organizations.The team is small, highly technical, and research-driven, with engineers working directly alongside world-class researchers and founders. The organization places a strong emphasis on technical depth, experimentation, performance optimization, and production-scale AI infrastructure.As a Machine Learning Systems Engineer, you'll work on the infrastructure that enables large-scale model training and inference, contributing directly to systems that make advanced AI models faster, more efficient, and more reliable.This is an opportunity to join an elite AI team where you can work at the intersection of machine learning, distributed systems, GPU infrastructure, and high-performance model serving.What You'll DoDesign, build, and operate infrastructure supporting large-scale ML training and inference systemsDevelop high-performance systems for serving and deploying large language modelsOptimize model inference for latency, throughput, memory utilization, and cost efficiencyBuild and maintain production ML infrastructure across GPU and cloud environmentsWork with inference engines such as vLLM, TensorRT, ONNX Runtime, and SGLangDevelop and optimize GPU-accelerated ML workloads using CUDABuild scalable training and inference pipelines using PyTorch and/or TensorFlowDeploy and manage ML workloads across Kubernetes and containerized environmentsDesign distributed systems capable of supporting high-volume model inferenceImprove model serving performance across different hardware and infrastructure configurationsBuild reliable systems for model deployment, monitoring, evaluation, and production operationsWork closely with research teams to translate new model architectures into production systemsOptimize infrastructure for emerging diffusion-based language models and other generative AI architecturesDevelop tooling and automation for ML experimentation and deploymentBuild and maintain cloud infrastructure across AWS, Azure, or comparable environmentsWork with Kubeflow and other ML orchestration platformsInvestigate performance bottlenecks across compute, networking, memory, and model-serving layersDevelop systems that make model training and inference faster, more efficient, and more reliableContribute to system architecture and technical strategy across the ML infrastructure stackOperate with high ownership in a fast-moving, deeply technical AI environmentWork closely with founders and researchers on highly technical infrastructure challengesIdeal Candidate BackgroundExperience Requirements2–5 years of professional experience in ML Systems Engineering, ML Infrastructure, AI Infrastructure, or related engineering rolesExperience building and operating production ML systemsExperience working on infrastructure for model training and/or inferenceExperience deploying machine learning models into production environmentsExperience working with GPU-based computing infrastructureExperience building scalable ML or distributed systemsExperience working with modern deep learning frameworksExperience operating in technically demanding engineering environmentsExperience collaborating closely with research and engineering teamsStrong ownership mentality with demonstrated execution abilityComfortable working on complex technical problems with limited precedentStrong interest in machine learning systems and AI infrastructureAbility to operate effectively in a fast-moving, research-driven environmentTechnical RequirementsStrong Python engineering experienceStrong experience with PyTorch, TensorFlow, or comparable ML frameworksExperience with GPU computing and CUDAExperience with ML inference systems such as vLLM, TensorRT, ONNX Runtime, or SGLangStrong understanding of model serving and inference optimizationExperience with Docker and containerized ML workloadsExperience with KubernetesExperience working with AWS, Azure, or other major cloud platformsExperience building distributed systems or scalable infrastructureExperience with ML training and inference pipelinesExperience with Kubeflow or comparable ML orchestration platforms preferredStrong understanding of performance optimization and system reliabilityExperience debugging production ML infrastructureStrong understanding of computer science and software engineering fundamentalsAbility to reason about GPU utilization, memory constraints, latency, and throughputStrong debugging and performance analysis capabilitiesEducationBachelor's degree or higher in Computer Science, Computer Engineering, Electrical Engineering, Mathematics, or related technical field preferredAdvanced degree in Machine Learning, Computer Science, or related field is a plusStrong computer science, systems, and machine learning fundamentalsEquivalent practical engineering experience acceptedSoft SkillsExceptional technical ownershipStrong analytical and problem-solving abilityDeep technical curiosityComfortable working on difficult and ambiguous infrastructure problemsStrong communication skillsComfortable collaborating with researchers and highly technical engineersHigh execution velocityStrong attention to system performance and engineering qualityBias toward experimentation and continuous improvementLow-ego collaborative mentalityComfortable receiving and incorporating technical feedbackStrong ability to reason from first principlesComfortable working in a small, high-performing teamStrong interest in cutting-edge AI systemsWillingness to work five days per week in Palo AltoCompensation & BenefitsBase Salary: $200,000 – $300,000Competitive Equity PackageOpportunity to work on cutting-edge diffusion-based language modelsDirect collaboration with world-class AI researchers and foundersOpportunity to work on large-scale ML training and inference infrastructureExposure to advanced GPU optimization and AI systems engineeringSignificant technical ownership in a small, elite engineering organizationOpportunity to influence foundational ML infrastructure and model-serving architectureHigh-growth AI company environmentOpportunity to work on AI systems being deployed by Fortune 500 organizationsWhy JoinThis is an opportunity to join a highly technical AI company working on one of the most important challenges in modern machine learning: making advanced language models significantly faster and more efficient.You'll work directly on the infrastructure powering next-generation AI models, solving difficult problems across distributed systems, GPU computing, model inference, and production ML infrastructure.As part of a small and technically elite team, you'll work closely with researchers and founders and have meaningful influence over how AI systems are designed, optimized, and deployed.