ML Engineer

About RustLabs We’re building the data layer for frontier AI. RustLabs is a high-throughput annotation and evaluation platform used by AI labs to produce training data, RLHF preference signals, and expert evaluations across text, image, code, and multimodal domains. We’re early, well-funded, and working directly with research teams at top labs. The role You’ll own the technical core of the platform — the pipelines that turn raw tasker output into clean, model-ready training data. This is a hands‑on IC role with significant ownership: you’ll design annotation schemas with customers, build evaluation infrastructure, write the tooling that ensures data quality at scale, and sit at the intersection of ML research and operations. What you’ll do Design and ship annotation pipelines (RLHF, SFT, eval, red‑teaming) end-to‑end — schema design, tasker UX, quality controls, aggregation, delivery to customers. Build evaluation infrastructure: automated checks, LLM-as-judge systems, calibration tooling, and inter‑annotator agreement metrics. Work directly with research teams at AI labs to understand their data needs and translate them into shippable annotation products. Build the internal tooling that lets a distributed tasker workforce produce gold‑standard data at scale — onboarding flows, qualification tests, payout logic, quality dashboards. Contribute to research artifacts (datasets, evals, papers) when relevant work ships. What we’re looking for 2+ years of professional experience as an ML engineer, research engineer, or similar role. Strong Python, comfort with modern ML frameworks (PyTorch, HuggingFace). You’ve shipped ML systems to production — you’ve debugged training pipelines, you know what “data quality” actually means in practice, you understand the failure modes of LLM evaluation. Familiarity with RLHF, SFT, or evaluation methodologies is a strong plus. You can write clean code, move fast, and talk directly to customers. Bonus: prior experience at an AI lab, data platform, or annotation company. Bonus: open-source contributions. #J-18808-Ljbffr RustLabs