LLM Quality & Evaluation Engineer
Dormont Manufacturing Co
Dormont Manufacturing Co is looking for a Software Engineer to support the measurement and quality of its AI assistant. This hybrid role involves designing evaluation datasets, building pipelines for evaluating assistant performance, and collaborating with other engineers to ensure product quality. Candidates should have a minimum of 2 years in software engineering, particularly in Go and Python, and experience with machine learning systems. A comprehensive benefits package and competitive salary are offered. Join us in shaping the future of AI and make a significant impact with your engineering skills. #J-18808-Ljbffr Dormont Manufacturing Co
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the LLM Quality & Evaluation Engineer in San Francisco, CA vacancy
- ...Health, Inc. is seeking a Forward Deployed QA Engineer based in San Francisco with the... ...States. The role focuses on ensuring the quality and stability of our conversational AI product... ...background in QA Engineering, expertise with LLM behaviors, and excellent communication...SuggestedRemote job
- Dynamo AI is seeking a candidate to lead LLM evaluation and benchmarking in San Francisco, California. You will generate high-quality data and develop innovative methods for assessing the safety and helpfulness of LLMs. The role requires domain knowledge in evaluation...Suggested
$176k - $253k
...Member of Technical Staff, AI Quality, in San Francisco. Your main... ...high standards through robust evaluation processes. You'll build... ...systems, and work directly with engineers to ensure our AI systems excel... ...experience, particularly in LLM and agent evaluations. Competitive...Suggested- ...passionate about ensuring the highest quality for cutting-edge generative... ...? As a software quality engineer at WRITER, you'll play a... ...plans for our AI agents and LLM-powered applications, ensuring... ...ML concepts, including model evaluation metrics, data pipelines, and...SuggestedFull timeWork at officeLocal areaFlexible hours
Writer Corporation
San Francisco, CA3 days ago - Reflection in San Francisco is looking for a data engineer dedicated to enhancing data quality for LLM pre-training. Your role will involve collaborating with world-class researchers to establish high standards for data collection and processing. You will design automated...Suggested
$120k - $140k
...rapidly while maintaining rigorous quality standards. Ellipsis Health... ...As a Forward Deployed QA Engineer, you will occupy a critical,... ...RCA) of Large Language Model (LLM)-driven behaviors in fast-paced... ...shadow agents. Prompt Evaluation & Optimization: Apply a strong...Full timeRemote workFlexible hours- ...combination of inventive research, design, and engineering. Our organization is very flat, and our team is... ...About the Role As a Software Engineer on the Agent Quality team at Cursor, you’ll build the measurement, evaluation, and feedback-loop infrastructure that makes the...
- B Capital is seeking a data engineer to ensure high data quality for training AI models. You will own the upstream data quality for LLM post-training and design automated QA methods in a collaborative environment. Ideal candidates will have strong engineering skills, a...
- ...Senior Compliance Engineer, AI Governance True Anomaly seeks those with the talent and ambition to build the technology that... ...operationalize compliance checkpoints and governance gates within LLM pipelines, evaluate AI vendors and platforms (including OpenAI, Anthropic...Permanent employment
True Anomaly
San Francisco, CA6 days ago $113.4k - $221.75k
...product we are building from scratch, and quality engineering is a core founding-team focus. We seek... ..., and developer-friendly. Stand up evaluation harnesses for automated workflows: golden... ...designing test strategy for AI / ML / LLM-powered features —prompt regression, evaluation...Full timeTemporary workLocal areaWorldwideShift work- Anysphere is seeking a Software Engineer for the Agent Quality team in San Francisco, CA. In this role, you will design and build infrastructure to evaluate and improve ML agents. Responsibilities include creating evaluation systems, defining quality metrics, and collaborating...
- Mercor is seeking an Engineering / Manufacturing / Technical Operations Evaluator to assess AI-generated artifacts against quality standards. The role demands a strong background and fluency in English, ideally with over 5 years of experience in the relevant field. This...Remote jobWork at officeFlexible hours
$113.4k - $221.75k
...Senior Quality Engineer The video rendering team owns the high-performance rendering pipeline that powers Premiere Pro, an industry-leading... ..., playback, editing, color, and export workflows. Evaluate user impact, functionality, performance, and regression risks...Temporary workLocal areaWorldwideAdobe
San Francisco, CA3 days ago- ...Quality Engineer Remedy Robotics is a medical technology company developing robotic systems for endovascular intervention. Its proprietary... ...control definition and implementation, and residual risk evaluation Establish and maintain end-to-end requirements...
Blackbird
San Francisco, CA3 days ago - B Capital is seeking a data engineer focused on ensuring high-quality data for AI model training. You will work closely with top researchers to develop metrics and standards for data quality. The ideal candidate has solid experience in Python and data pipelines and is detail...
- ...0+ million raised to date. About the role As a Distributed LLM Inference Engineer, you will help with systems and optimizations that push the... ...Anyscale Inc. is an Equal Opportunity Employer. Candidates are evaluated without regard to age, race, color, religion, sex,...Work at office
- Neura Market is seeking a Software Quality Engineer in San Francisco to ensure the quality of generative AI applications. In this hybrid role, you will define quality strategies, develop automation frameworks, and collaborate with cross-functional teams. The ideal candidate...
$137k - $188k
...our San Francisco headquarters, and reporting to the Forensic Engineering Manager, the Senior Compliance Engineer is a key member of... ...identification, signal‑flow tracing, and multimedia framework evaluation. Analyze Android and iOS devices and applications for compliance...Full timeWork at officeLocal areaRemote workVia Licensing Corporation
San Francisco, CA4 days ago$147.4k - $272.1k
...are looking for a Software Development Engineer in Test with a strong background and experience... ...who is passionate about improving the quality of software products and collaborating... ...the world. This team designs and builds evaluation tools, automation frameworks, and...Relocation$181.1k - $318.4k
Software Reliability & Stability Quality Engineer, Siri San Francisco Bay Area, California, United States Software and Services At Apple, quality... ...-term measurements of quality. You will create and utilize LLM tools to assist in this overall effort.The ideal engineer...Temporary workRelocation- ...Location Type On-site Department Engineering Our Mission Reflection’s mission... ...ensure that the data used to train and evaluate our models meets a high bar for quality, reliability, and downstream... ...will: Own upstream data quality for LLM post‑training and evaluation by...Full timeRelocation package
- ...responsible for building our Forward Deployed Engineering function. This team plays a critical... ...‑the‑art models, orchestrating complex LLM workflows, integrating with enterprise... ...codebase for inference, fine‑tuning, and evaluation. Lead end‑to‑end deployment across hybrid...Full timeRelocation package
$180k - $270k
...dedicated to developing transformative AI tools for productivity. The role involves collaborating with machine learning researchers and engineering teams to define metrics, improve model capabilities, and ensure effective performance tracking. Candidates should bring strong...- ...looking for highly motivated QA Automation Engineers to test in a mix of technologies... ...QA Engineer will be expected to be the quality gatekeeper for the product. The position... ...maintain scalable test automation strategies Evaluate multiple approaches and technologies to...Remote jobWorldwide
- ...milestones, manage timelines, track dependencies Unblock teams and elevate risks proactively Technical Coordination Translate between engineering and business stakeholders Understand technical trade‑offs to make informed prioritization calls Own technical roadmaps and...Worldwide
- Test QA Engineer job at Manatal Tech Team Account (Production). San Francisco, CA. About the Role We are looking for a QA Tester who is passionate about delivering high-quality software products. The ideal candidate is detail-oriented, analytical, and proactive in identifying...
- ...give you a chance to work with some of the most talented people in the graphics industry. Our Software team is looking for a QA Test Engineer to help us in our mission. The QA Test Engineer role supports our Software, Database, and Systems Engineering teams by testing our...Full timeWork experience placementWork at office
- Ellipsis Health is seeking a Forward Deployed QA Engineer to ensure the quality of its conversational AI product, Sage. The role requires expertise in software QA and a solid understanding of LLM behavior. Candidates will engage with clients to perform rigorous testing...Remote jobFlexible hours
- ...Sr. Software Quality Engineer As a Sr. Software Quality Engineer , you will work with the cross-functional team to act as a champion of Design Quality Assurance and Engineering activities throughout the product development lifecycle. You hold your work and teams to the...
Echo Neurotechnologies
San Francisco, CA4 days ago - ...Interface (BCI) space, driving innovation through advanced hardware engineering and AI solutions. Our mission is to deliver cutting-edge... ...autonomy to people living with disabilities and improve their quality of life. Team Culture Join a small, dedicated team of...
Echo Neurotechnologies
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to LLM Quality & Evaluation Engineer. Be the first to apply!
Related searches
- customer quality engineer San Francisco, CA
- supplier quality engineer San Francisco, CA
- senior supplier quality engineer San Francisco, CA
- senior manager quality engineering San Francisco, CA
- quality engineer manager San Francisco, CA
- advanced quality engineer San Francisco, CA
- quality systems engineer San Francisco, CA
- principal quality engineer San Francisco, CA
- senior data quality engineer San Francisco, CA
- quality engineer remote San Francisco, CA