ML Systems Engineer for Human Feedback & Evaluation (SF)
Design Arena
Design Arena hosts cutting-edge ML evaluation platforms and data engines. This role focuses on building systems that learn from human preference signals to evaluate frontier AI models. You will collaborate with researchers and scale data pipelines for large-scale experiments. Ideal candidates have a strong STEM background and hands-on experience with ML systems and human-in-the-loop learning. This on-site role in San Francisco offers meaningful equity and competitive compensation. #J-18808-Ljbffr Design Arena
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the ML Systems Engineer for Human Feedback & Evaluation (SF) in San Francisco, CA vacancy
- Senior ML Systems Engineer, Frameworks & Tooling at Cohere Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises... ..., xFormers, etc.). Familiarity with evaluation and serving frameworks (vLLM,...SuggestedFull timeWork at officeRemote workFlexible hours
- Sieve is seeking a Machine Learning Engineer in San Francisco to own the entire ML lifecycle for multimodal data—... ...dataset design, model improvement, evaluation systems, and production pipelines... ...improvements quickly. On-site in SF five days a week, you will drive...Suggested
- .... It is a discovery engine that transforms messy... ...We are hiring an ML Engineer, Discovery... ...workflows for compound evaluation, program prioritization... ...and improve the AI systems that orchestrate and... ...based on real user feedback Location: Beautiful SF office with a high-...SuggestedFull timeWork at office
Mithrl
San Francisco, CA16 hours ago - ...integration into real-world systems, with the... ...for a visionary Senior ML Engineer who will bridge the gap... ...fallback mechanisms, human-in-the-loop (HITL) checkpoints... ...models; utilizing evaluation frameworks to quantify... ...~ Experience using feedback loops to continuously...SuggestedFull timeShift work
Palm Venture Studios
San Francisco, CA16 hours ago - ...You will build the ML systems that carry us from... ...you ship, every feedback loop you close, and... ...Machine Learning Engineer, embedded in the Fully... ...loops that turn human underwriter... ...confidence scoring and evaluation frameworks that define... ...together too ️ SF, NYC, Dallas-Fort...SuggestedFull timeWork at office
Shepherd
San Francisco, CA16 hours ago - ...without needing a human on the other end.... ...Machine Learning Engineer to join our team and... ...exciting time to join an ML team designing a... ...provides precise feedback, and make it more... ...Expanding our ASR systems to new languages... ...such as training/evaluation datasets and labeling...Full timeLive inWork at officeWorldwide
Speak
San Francisco, CA16 hours ago - ...seeking talented MLOps Engineers with deep, hands-on... ...AI model training and evaluation work, including writing... ...data for frontier AI systems. This is a W-2 employment... ...infrastructure, and ML framework-level topics... ...clear, written technical feedback. Develop guidelines...Full timeWeekday work
- ...Mission: Give every human... ...stage startups in SF. Behind closed doors... ...OpenAI to rigorously evaluate the capabilities... ...machine learning systems that power our evaluation... ...generation data engine. You'll train... ...building ML systems. You've... ...learns from human feedback. You're excited...Full timeRelocationVisa sponsorship
- ...Join to apply for the ML Engineer - Personalization & Recommendation Systems role at krea.ai . About Krea At Krea... ...is to build tools that empower human creativity, not replace it. We believe... ...search, ranking, or aesthetic evaluation. Interest in creative tools,...Full timeH1b
krea.ai
San Francisco, CA1 day ago - ...with safety, privacy, and real-world responsibility in mind. Our ML team comes from a culture of academic research driven to... ...privacy for the sake of ML advancement. Responsibilities Own LLM evaluation processes and methods with a focus on generating benchmarks representative...Local areaShift work
- ...AI in San Francisco is seeking a Machine Learning Engineer to own the full model lifecycle—training, evaluating, fine-tuning, and deploying models for production.... .... The role emphasizes hands-on development, strong ML fundamentals, and a pragmatic approach to when to train...
$240k - $270k
...you come in. As an AI/ML Engineer, you’ll join a growing... ...Prototype and productionize AI systems that feel intuitive but... ...in all our offices in SF, NYC, London and Sydney... ...for the purposes of evaluating your candidacy for... ...to complement, not replace, human decision-making....Full timeWork at officeFlexible hoursSigma Computing
San Francisco, CA16 hours ago$100 - $150 per hour
...machine learning pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against established criteria to ensure... ...are reproducible and defensible. Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous...Contract workSummer workRemote workMercor
San Francisco, CA1 day ago- ...best web query results. The ideal candidate will possess a strong intuition for relevance and feature engineering. This position requires a rigorous approach to evaluation based on real-world outcomes. At Parallel, the work is fully in-person, fostering a collaborative...
$150.4k - $277.6k
...'re seeking research engineers to build systems for breakthrough innovations... ..., harness design, evaluation, and deployment. We... ...and value clear-eyed feedback. Lean habits: You can... .... 2+ years of ML engineering experience... ...accessibility is a fundamental human right. You\u2019ll...Relocation- ...As a Machine Learning Engineer at Advex, you will play... ...well versed with the ML life-cycle process, collect... ...Diffusion Models # Evaluation of generative models... ...are the backbone of humanity. It is our duty to improve... ...of people. Feedback is the breakfast of champions...Full time
Openreq
San Francisco, CA16 hours ago - Uber AI Solutions (UAIS) is building data and evaluation infrastructure for next-gen AI. We ship end-to-end ML and backend systems, owning a vertical from idea to product. You’ll work with product managers and engineers to turn vision into revenue-generating solutions used...
$240k - $270k
...you come in. As an AI/ML Engineer, you’ll join a growing... ...opportunitiesPrototype and productionize AI systems that feel intuitive but... ...in all our offices in SF, NYC, London and Sydney... ...for the purposes of evaluating your candidacy for... ...to complement, not replace, human decision-making.Full timeWork at office- ...About Human Archive Human Archive is a... ...sophisticated biological systems we have ever... ...Machine Learning Engineer, you’ll work on multimodal... ...downstream policy evaluation. This is a hands-... ...and research feedback What We’re... ...physical AGI Strong ML engineering...Full timeShift work
Human Archive
San Francisco, CA16 hours ago - ...ways that feel natural and human. With this vision, we're designing... ...Responsibilities: Own evaluation pipelines — design, build,... .... Proven software engineer who loves ML; comfortable writing production... ...user-facing, online ML systems—despite shifting requirements...Full timeContract workFlexible hoursShift work
Sesame, L.l.c.
San Francisco, CA16 hours ago - ...email, and enterprise systems. Born in Y... ...models that make human-like conversations... ...one of our first ML hires, shaping not... ...with product and engineering teams to integrate... ...together. We give feedback with kindness, challenge... ...for the purpose of evaluating and selecting you...Full timeWorldwideShift work
HappyRobot
San Francisco, CA16 hours ago - ...mission is to organize human intelligence to... ...of shaping the systems that are redefining... ...Machine Learning Engineer on the Marketplace... ...This is an applied ML role with direct product... ...cold start, noisy feedback, heterogeneous... ...• Develop evaluation and experimentation...Full timeWork at officeRelocation package
Mercor
San Francisco, CA16 hours ago - .... Speak is building a human-level, AI-powered tutor... ...speak, get instant feedback, and progress through... ...role We’re hiring an ML Engineer, Assessments to help... ...best-in-class assessment systems across multiple... ...Define and operationalize evaluation Implement validation...Full timeLive inImmediate start
Speak
San Francisco, CA16 hours ago $213k - $263k
...training ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge: How... ...? We are seeking visionary machine learning engineers and researchers to architect the scalable deep learning systems, novel data workflows, and eval tools that...Full timeRemote workWaymo
San Francisco, CA16 hours ago$225k - $300k
...platform that restores humanity to healthcare and... ...ROI for health systems across the country... ...Machine Learning Engineer at Ambience , you... ...failures and designing evaluations to building,... ...testing, production feedback, and human... ...years in production ML, research engineering...Full timeWork at officeImmediate startRemote workFlexible hours3 days per weekAmbience Healthcare
San Francisco, CA16 hours ago- ...culture out of our new SF Rincon Hill office. If... ...Senior Machine Learning Engineer, you will build the intelligence... .... You will own systems that turn noisy,... ...OCR pipelines, voice AI, evaluation systems, and backend production... ...logic, retrieval, human review, or hybrid systems...Full timeWork at office
Hike Medical
San Francisco, CA16 hours ago - ...researchers to create evaluations, publish benchmarks... ...Work together with engineers, scientists,... ...About Handshake AI Human data is the core infrastructure... ...scale. The ML Opportunity at... ...impact: Your ML systems directly impact millions... ...lunch/gym in our SF office Time Off...Full timeWork at officeRemote workFlexible hours
Handshake
San Francisco, CA16 hours ago - ...those documents the way a human would, and make it... ...hiring a Machine Learning Engineer to help us train and deploy... ...training, fine tuning, and evaluating ML models used in production systems Language/Skills: You’... ...person role at our office in SF. We’re an early stage...Full timeWork at officeLocal area
Reducto
San Francisco, CA16 hours ago - ...Scientist. It is a discovery engine that transforms messy... ...THE ROLE We are hiring an ML Engineer, Analysis and Simulation... ...reasons about biological systems, this is an exceptional... ...continuously based on real user feedback Location: Beautiful SF office with a high-energy,...Full timeWork at office
Mithrl
San Francisco, CA16 hours ago - ...Responsibilities As a senior Machine Learning Systems Engineer on the Search Platform team, you will... ...search. Own end-to-end delivery of ML components from experimentation through... ...systems with robust monitoring and evaluation harnesses.Agentic Search & RetrievalDesign...Work at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to ML Systems Engineer for Human Feedback & Evaluation (SF). Be the first to apply!
Related searches
- graduate machine learning engineer San Francisco, CA
- senior ml engineer San Francisco, CA
- machine learning engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- entry level machine learning engineer San Francisco, CA
- machine learning software engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- computer vision machine learning engineer San Francisco, CA
- junior machine learning research engineer San Francisco, CA
- lead system engineer San Francisco, CA