ML Infrastructure Engineer

Who is Recruiting from Scratch:

Recruiting from Scratch is a premier talent firm that focuses on placing the best product managers, software, and hardware talent at innovative companies. Our team is 100% remote and we work with teams across the United States to help them hire.

ML Infrastructure Engineer
Location - San Francisco, CA - (On-site - 5 Days Per Week)
Compensation - $200,000 - $300,000 Base + Competitive Equity
Visa - Visa Sponsorship Available (New H1B, H1B Transfers, TN, Eligible Visa Types)
Company Stage - Series B ($100M+ Raised)
Industry

Artificial Intelligence, Machine Learning Infrastructure, AI Infrastructure, Developer Tools, Enterprise AI, Document AI, Enterprise SaaS
About the Company

The company is building one of the leading AI infrastructure platforms powering document intelligence for modern AI applications.

Its platform transforms complex documents-including PDFs, spreadsheets, presentations, images, and enterprise files-into structured, production-ready data for AI systems. Processing tens of millions of pages every month, the platform serves hundreds of customers ranging from AI-native startups to Fortune 10 enterprises, enabling reliable document ingestion for large language models and AI workflows.

Backed by Andreessen Horowitz, Benchmark, and First Round Capital with over $100M in funding, the company has experienced exceptional growth, increasing revenue more than 8x year-over-year while becoming one of the fastest-growing AI infrastructure companies in the market.

As an ML Infrastructure Engineer, you'll own critical training and inference infrastructure powering production machine learning systems. You'll work closely with ML researchers and product engineers to ensure models are trained, deployed, monitored, and served efficiently while continuously improving latency, reliability, scalability, and cost across production AI workloads.

This is an exceptional opportunity to join a fast-growing AI-native company where ML Infrastructure Engineers build the foundational systems enabling next-generation AI products at massive production scale.
What You'll Do

• Build and maintain production ML model serving infrastructure powering millions of document processing requests
• Design and improve training infrastructure supporting models ranging from hundreds of millions to tens of billions of parameters
• Optimize inference latency, throughput, reliability, and GPU utilization across production systems
• Develop observability, monitoring, logging, and alerting across the ML infrastructure stack
• Build internal tooling and data pipelines enabling ML researchers to rapidly deploy models into production
• Architect infrastructure that intelligently routes inference workloads across multiple cloud providers
• Optimize infrastructure for model accuracy, latency, reliability, and operational cost
• Collaborate closely with ML researchers, software engineers, and product teams to accelerate experimentation
• Improve Kubernetes-based infrastructure supporting large-scale ML workloads
• Debug complex production infrastructure issues involving GPUs, distributed systems, and model serving
• Design scalable systems supporting rapid AI model deployment and iteration
• Champion engineering excellence across ML infrastructure, automation, and production reliability

Ideal Candidate Background
Experience Requirements

• 3+ years of ML Infrastructure Engineering experience
• Experience building production ML model serving and inference infrastructure
• Experience working at AI-native startups or organizations training production ML models
• Experience deploying and maintaining large-scale production ML systems
• Experience supporting ML researchers with production infrastructure
• Strong startup ownership mentality with demonstrated engineering execution
• Experience building highly scalable infrastructure supporting AI products
• Experience collaborating closely with ML, Product, and Infrastructure teams
• Experience optimizing production inference systems
• Previous experience working with GPU-intensive workloads strongly preferred

Technical Requirements

• Strong software engineering fundamentals across distributed systems and infrastructure
• Expert-level Python programming skills
• Strong Kubernetes, Docker, Helm, and cloud infrastructure experience
• Deep understanding of GPU optimization, model serving, and inference infrastructure
• Experience with PyTorch and modern ML deployment workflows
• Experience building observability, monitoring, and logging systems for ML infrastructure
• Strong understanding of model deployment, training pipelines, and production inference
• Experience working with 1-3 node model training and single/double-node serving
• Familiarity with cloud platforms including AWS, Azure, or GCP
• Ability to rapidly debug production infrastructure and performance bottlenecks

Education

• Bachelor's degree in Computer Science, Machine Learning, Engineering, Mathematics, or another technical discipline preferred
• Strong engineering background with demonstrated infrastructure expertise
• Top technical universities, competitive programming backgrounds, or exceptional engineering accomplishments strongly preferred
• Publications at top ML conferences (NeurIPS, ICML, ICLR, CVPR) are a strong plus

Soft Skills

• Outstanding technical problem-solving
• Strong engineering ownership
• Excellent collaboration skills
• High execution mentality
• Comfortable operating with ambiguity
• Strong cross-functional communication
• Startup mindset with high execution velocity
• Structured systems thinker
• Passion for AI infrastructure and production ML systems
• Continuous learner with deep technical curiosity

Compensation & Benefits

• Base Salary: $200,000 - $300,000
• Competitive Equity Package
• Visa Sponsorship Available
• On-site work model (5 days/week in San Francisco)
• Opportunity to build core infrastructure powering one of the fastest-growing AI platforms
• Direct collaboration with ML researchers and engineering leadership
• Significant ownership across production AI infrastructure
• High-impact engineering culture with exceptional career growth
• Well-funded Series B company backed by leading venture firms
• Opportunity to build foundational infrastructure enabling next-generation AI systems

Why Join

This is an opportunity to build the infrastructure powering one of the fastest-growing AI platforms transforming how enterprises process information.

You'll work at the intersection of machine learning infrastructure, distributed systems, GPU optimization, model serving, production engineering, and AI infrastructure while solving some of the hardest scalability challenges in modern AI.

As an ML Infrastructure Engineer, you'll own foundational systems that enable production AI models to operate reliably, efficiently, and at massive scale while helping define the future of enterprise AI infrastructure.