AI Evaluation Engineer - Production ML Pipelines
Distyl AI, Inc.
Distyl AI, Inc. is looking for an AI Evaluation Engineer to design and implement evaluation frameworks for AI systems deployed in customer environments. The ideal candidate will have 2+ years of software engineering experience and strong Python skills, focusing on Evaluation-Driven Development. The role involves building evaluation pipelines, maintaining test suites, and collaborating with engineers to ensure systems meet high quality standards. The position follows a hybrid work model, requiring 3+ days in the San Francisco office. #J-18808-Ljbffr Distyl AI, Inc.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluation Engineer - Production ML Pipelines in San Francisco, CA vacancy
$150k - $250k
Distyl AI is seeking an AI Evaluation Engineer to design evaluation frameworks and build AI systems. The position requires strong Python programming and 2+ years of software engineering experience. Candidates should be systems-oriented with experience in evaluation-driven...Pipeline- ...insight. Rillet is an AI-native ERP that can... ...Need As an Applied AI Engineer on Rillet's AI & ML Team, you will design and ship production AI systems that transform... ..., fine-tuning pipelines, and RAG systems at scale... ...internal tooling and evaluation frameworks that...PipelineWork at officeRemote workRelocationFlexible hours
Rillet
San Francisco, CA3 days ago - A cutting-edge AI firm in San Francisco is seeking a Research Engineer to develop evaluation systems and benchmarking pipelines for language models. Candidates should have a strong background... ..., coding skills, and familiarity with ML models. You will work collaboratively...Pipeline
$240k - $280k
A leading software monitoring company is seeking a Senior Software Engineer on its AI/ML team to build evaluation infrastructure for measuring the performance of AI systems. This role involves designing datasets, creating benchmarks, and ensuring AI features behave reliably...Pipeline$180k - $225k
About Scale AI Scale AI is the data foundation... ...deploy reliable production AI applications.... ...Deployed AI Engineer on our Enterprise... ...infrastructure, data pipelines, and business requirements... ...sources Implement evaluation frameworks to... ...data scientists, ML engineers, and...PipelineFull time$150k - $210k
AI Engineer Location: San Francisco, CA, in-office Compensation... ...intersection of AI, product development, and... ...LLMs, APIs, and data pipelines into production‑ready... ...with advancements in AI/ML, especially in agent frameworks... ..., monitoring, and evaluation frameworks to mitigate...PipelineWork at officeLocal areaImmediate startFlexible hours$150k
Tzafon is seeking a skilled engineer to enhance their machine intelligence systems... ...'ll be responsible for building evaluation infrastructure, designing data pipelines, and implementing fine-tuning... ...a solid track record of shipping ML systems, and strong opinions on evaluation...Pipeline- ...Origin is building physical AI for the built world.... ...building interiors at production quality. OG-1 is... ...flywheel: teleoperation pipelines (GELLO, SpaceMouse, VR)... ...detection. Design offline evaluation metrics that predict real... ...MS/PhD in CS, Robotics, ML, or equivalent...Pipeline
$350k
...and conversational AI systems. This team... ...hands‑on research and engineering role for someone... ...speech research into production systems that serve... ...development, data pipelines, training infrastructure, evaluation, inference optimization... ...‑oriented ML systems. Strong research...PipelineImmediate startRemote workWork visa$200k - $400k
...often pushed straight to production. Simile is changing... ...We have built the first AI simulation of society,... ...across the stack to train, evaluate, deploy, and monitor... ...tight research-to-product pipeline. This requires intense... ...Must Haves ML Proficiency: High proficiency...PipelineFlexible hours$172k - $300k
...believe meaningful AI doesn’t start with... ...performance, and production-ready systems. We... ...empower scientists, engineers, financial experts... ...machine learning (ML) techniques to... ...RAG), fine-tuning pipelines, prompt engineering... ...and comprehensive evaluation workflows to ensure...PipelineFull timeLocal area$190k - $230k
...vertically integrated AI infrastructure... ...Enterprise AI Automation Engineer to play a key role... ...deployment pipelines and lifecycle management... ...for AI agents in production environments... ...enablement programs Evaluating emerging AI... ...including 3+ years in AI/ML or AI application...PipelineTemporary work$125.5k - $230.2k
...and Decision Science – AI Native Engineering AI/Machine Learning... ...You will enhance data pipelines and storage, ensuring... ...you will monitor and evaluate learning processes to... ...and delivering AI/ML use cases relevant to... ...solutions from pilots to production while meeting...PipelineFull timeWork experience placementSummer holidayFlexible hoursErnst & Young Oman
San Francisco, CA3 days ago- Jack & Jill is seeking a Founding AI/ML Engineer in San Francisco. The role involves building Generative Engine Optimization systems for... ...Ideal candidates have strong NLP fundamentals and experience in production ML systems. You'll collaborate with world-class researchers...Pipeline
$248.4k - $310.5k
A leading AI solutions provider in San Francisco is seeking a Software Engineer for Robotics & Autonomous Systems. You will build production systems for data collection and model training, and collaborate with stakeholders. Ideal candidates have 3+ years in software engineering...Pipeline- ...leading technology company is seeking an Automation Engineer based in California to create automated scripts for AI training. This role involves building reliable... ...with engineers, and continuously improving CI/CD pipelines. The ideal candidate has expertise in Python, R,...PipelineRemote jobFlexible hours
$130k - $220k
...summary by the Joinrs AI : The selection process... ...company. They help engineers, enterprises, investors... ...described as an AI Evaluation Engineer / Technical Generalist... ...role and not a pure ML research position. It... ..., technical product work, strategic analysis...Full timeWorldwideAurora Jobs ApS
San Francisco, CA1 day ago- HopHR is building its Silicon Valley engineering team in San Francisco and seeks a skilled AI/ML Engineer. This role focuses on developing production-grade AI systems and involves... ...significant responsibilities in building LLM pipelines and optimizing RAG architectures....PipelineWork at office
- ...Francisco is seeking an experienced Senior Software Engineer to own and innovate major parts of their AI stack. The role involves designing workflows that... ...experience, strong backend skills, and familiarity with ML products. Competitive salary and comprehensive benefits...Pipeline
$200k - $250k
Job Title: Founding AI/ML Engineer Salary: $200-250K + Equity Company Description: Generalcatalyst... ...search. You will bridge research and production, transforming novel black‑box... ...will do: Design and build end‑to‑end ML pipelines including data processing, model training...Pipeline$180k - $250k
David Joseph & Company is seeking a skilled engineer in San Francisco to own the intelligence... ...core capabilities, designing fine-tuning pipelines, and ensuring reliability for enterprise... ...have strong experience in Python and ML frameworks, particularly PyTorch. This full...PipelineFull time$7.5k
...Description Job Description AI Engineer Location: San... ...the world's leading ML research labs — to... ...layer that powers the product. AI is not a feature here... ...latency: tune LLM and VLM pipelines, and classical ML... ...robust evals: build evaluation frameworks that make AI...PipelineWork at officeRelocationVisa sponsorshipRelocation package$150k - $350k
...Collate is an AI document generation... ...Our AI researchers, engineers, and designers have... ...that power Collate’s products. You’ll work at... ...standards for how we evaluate, and deploy models... .... Build pipelines and infrastructure... ...building and deploying ML/AI systems in...PipelineCollate
San Francisco, CA2 days ago- Baseten is looking for an AI Solutions Engineer in San Francisco to partner with customers in architecting and deploying production-grade AI applications. The role combines software... ...strong expertise in Python and AI/ML pipelines. Benefits include competitive pay, full...PipelineFlexible hours
- ...About the role Our client is a well-funded AI startup building production-grade ML infrastructure used by enterprise customers. They are looking for a Senior AI/ML Engineer to own model training pipelines, evaluation systems, and inference serving at scale. Full-time...PipelineFull time
- ...Blockchain solutions * AI & Machine Learning... ...* Custom software engineering We build scalable digital products for startups,... ...engineering standards * Evaluate emerging... ...Experience with AI/ML systems and LLM integrations... ...* DevOps and CI/CD pipelines **_What We Offer_...PipelineRemote workWorldwideFlexible hours
$150k - $350k
Job Title AI/ML Research Engineer Salary $150k-$350k + Equity Company Description Fast-growing enterprise... ...rapidly from advanced research to production, designing multi-agent systems and... .... Develop robust reasoning pipelines and vision-language models for understanding...Pipeline$207k - $290k
...Description About JazzX AI: Vision:... ...actually run in production, handle real complexity... ...an experienced AI Engineer with deep... ...including training pipelines, simulation environments... ...systems. Evaluation & Monitoring: Define... ...experience in AI/ML engineering, including...PipelineWorldwideFlexible hours- ...infrastructure that turns advanced AI research into real-... ...-loop processes, and production-grade reliability at... ...an AI Platform Backend Engineer , you will own and... ...enables product teams, ML engineers, and customer... ...backend services and data pipelines that power Brain Co.’s...Pipeline
$192k - $259.8k
...Location Type Hybrid Department Engineering Job Summary Drata's AI Platform team builds the production infrastructure that powers AI... ...understand, deployment pipelines that handle model upgrades without... ...years building or operating AI/ML infrastructure in production....PipelineFull timeFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Engineer - Production ML Pipelines. Be the first to apply!
Related searches
- ai research engineer San Francisco, CA
- ai developer San Francisco, CA
- ai prompt engineer San Francisco, CA
- ai engineer San Francisco, CA
- senior ai engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- machine learning ai engineer San Francisco, CA
- pipeline companies hiring San Francisco, CA
- pipeline San Francisco, CA