Machine Learning Engineer, Frontier Data Products

Who is Recruiting from Scratch: 

Recruiting from Scratch is a specialized talent firm dedicated to helping companies build exceptional teams. We partner closely with our clients to deeply understand their needs, then connect them with top-tier candidates who are not only highly skilled but also the right fit for the company’s culture and vision. Our mission is simple: place the best people in the right roles to drive long-term success for both clients and candidates. 

Title of Role: Machine Learning Engineer – Frontier Data Products
Location: San Francisco, CA or New York City, NY
Company Stage of Funding: Series C (Profitable, $10B Valuation)
Office Type: Onsite (5 Days/Week)
Salary: $130,000 – $500,000 + Equity + Performance Bonus

Company Description

Our client is one of the fastest-growing AI infrastructure companies, building the human intelligence platform powering the next generation of frontier AI models. Their platform enables leading AI labs and enterprises to access expert knowledge at scale, accelerating the development of advanced AI systems through high-quality human feedback and evaluation.

As a profitable Series C company with a multi-billion-dollar valuation, they are building foundational infrastructure at the intersection of machine learning, human intelligence, and AI product development.

What You Will Do

• Design and build production machine learning systems that evaluate, validate, and improve AI-generated work


• Develop evaluation frameworks for complex tasks where labels are ambiguous, incomplete, or constantly evolving


• Build feedback loops that incorporate human review, disagreement resolution, and model corrections into continuous learning systems


• Own production model performance, including quality, latency, explainability, drift detection, and cost optimization


• Improve AI system performance using prompting, retrieval, fine-tuning, heuristics, active learning, and human-in-the-loop techniques


• Partner closely with backend engineers to integrate ML inference into scalable production workflows


• Debug and improve live production ML systems where model quality directly impacts customer outcomes


• Help define the architecture and long-term technical direction of AI evaluation infrastructure

Ideal Candidate Background

• 3+ years of experience building and deploying production machine learning systems


• Strong software engineering skills with Python and production backend development


• Experience shipping ML systems that measurably improved product or business outcomes


• Strong understanding of evaluation methodologies, error analysis, and production model behavior


• Experience with LLM applications, retrieval systems, model evaluation, or AI-powered products


• Strong engineering fundamentals across the full ML lifecycle, from data through deployment


• Comfortable operating in ambiguous, fast-moving environments with evolving requirements


• Excellent communication and cross-functional collaboration skills

Preferred

• Experience with LLM evaluation, RAG, prompt engineering, fine-tuning, or AI agents


• Experience building human-in-the-loop machine learning systems


• Experience with Temporal, AWS, PostgreSQL, LiteLLM, or distributed backend systems


• Familiarity with production monitoring, drift detection, and model observability


• Startup or high-growth engineering experience


• Strong product instincts and ability to balance ML sophistication with operational simplicity


• Experience working on AI infrastructure, developer platforms, or ML tooling

Compensation and Benefits

• Competitive base salary: $130,000 – $500,000


• Generous equity package


• Bi-annual performance bonus


• Relocation assistance


• Housing stipend for employees living near the office


• Monthly meal, wellness, and lifestyle stipends


• Comprehensive medical, dental, and vision coverage


• Opportunity to build core infrastructure used by the world's leading AI companies

This is an exceptional opportunity for machine learning engineers who enjoy building production AI systems, solving complex evaluation challenges, and shaping the infrastructure powering frontier AI development.