Qualitative Evaluation Engineer, Multimodal AI
Luma AI
Luma AI is seeking a Qualitative Evaluation Engineer to design evaluative systems that capture believability, identity retention, and scene coherence in generative models. You’ll translate fuzzy quality into concrete criteria and collaborate with researchers and technical artists to guide model development. You will lead qualitative studies, build scalable evaluation frameworks, and turn nuanced judgments into actionable insights for fine-tuning, data curation, and UX improvements. #J-18808-Ljbffr Luma AI
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Qualitative Evaluation Engineer, Multimodal AI in New York, NY vacancy
- Mercor is partnering with a leading AI research organization to engage experienced sales engineers for a project focused on evaluating how well AI systems perform real-world technical sales work. You will define what excellent work looks like by designing task-specific...Suggested
- Cohere is seeking a technical staff member focused on multimodal AI. You will design and develop systems integrating text, speech, and vision... ...representation learning. The role emphasizes hands‑on engineering, scalable systems, and potential publications; remote‑friendly...SuggestedRemote workWorldwide
- Mercor is partnering with a leading AI research organization to recruit experienced sales engineers who will define evaluation criteria for real-world technical sales work. You will score samples, provide written justifications, and help ensure reproducible, defensible...Suggested
- Obsidian is seeking contributors to evaluate frontier AI coding models through structured technical assessments. You will work on realistic ML engineering workflows, model evaluation, and production-ready deployment considerations. The role emphasizes hands-on use of frontier...Suggested
- Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models in collaboration with a leading AI lab. You will assess AI-generated music in German and English across a wide range of genres and rate it against detailed quality standards...Suggested
- ...of Medicine in New York seeks a Data Science Analyst/Engineer to build the data foundation for machine learning and... .... You will develop scalable data pipelines, train and evaluate ML models, and explore multimodal integration with imaging or -omics data to improve patient...
- United States Digital Space LLC is building a dataset to evaluate AI coding agents by creating realistic developer environments with codebases, infrastructure, tickets, docs, and a believable development history. You will design tasks from intermediate states and craft...
$216k - $270k
About Scale AIScale AI is the data foundation for AI, helping... ...problems remains one of the hardest engineering challenges.As a Senior... ...enterprise customers to design, evaluate, and deploy intelligent... ...optimization.Familiarity with multimodal AI systems and frontier foundation...Full time- ...digital partner that combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions that deliver... ...successful digital experiences and client partnerships.As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced...Immediate start
$118k - $157k
...motivated and experienced Security Sales Engineer to join our dynamic enterprise sales engineering... ...sales cycle, including during customer evaluations and proof-of-value initiatives.Develop... ...and security platform for the AI era, providing businesses with unified visibility...Work at office- ...Software Engineer, Agent Evaluation and QualityEngineering · Full-time · San Francisco; New YorkOur mission is to automate coding. The first step... ...cost.What You'll Work OnDesigning and building best-in-class AI evaluation system: curated datasets, offline replay, scorers...Full timeWork at office
Anysphere
New York, NY1 day ago - ...SUMMARY: The Senior Refrigeration Design Engineer - Controlled Environment Rooms reports to... ...technical specs, and submittal packages. Evaluate and select materials and components (e.g.... ...directly into each article, started with the help of AI. #J-18808-Ljbffr TriMark USAFull timeFor contractorsLocal areaRemote work
- Encord is hiring a Solutions Engineer to be the core technical expert for our most complex... ...on robotics, autonomous systems, and multimodal AI. You will own end-to-end technical... ...customers through security, architecture, and evaluation steps while #J-18808-Ljbffr Encord
- Artemis Arc is seeking an Evaluation Specialist Sr to design and implement end-to-end evaluation systems for the VHA AI Education Hub. You will measure adoption, impact, and learning outcomes across AI education initiatives for the Veterans Health Administration. You will...
- Mercor is seeking a domain expert to craft original Thai law questions for an AI evaluation dataset. You will draft exam-style items with ten answer options and a worked solution, remotely and on a flexible schedule. Requirements: law degree or Thai equivalent, ideally...Remote jobWeekly payFor contractorsFreelanceFlexible hours
$320k
...NVIDIA is seeking a Distinguished Engineer to serve as the founding technical leader for our AI Safety & Security Engineering team. Rooted in the firm belief that... ...engineering managers, security partners, and evaluation teams to build consensus through active listening...Full timeRemote work$115.2k - $214k
...That’s what makes us Roche.Advances in AI, data and computational sciences are transforming... ...AI for Drug Discovery, the Software Engineering team builds and operates software... ...coordinated workflows that support the design and evaluation of therapeutic molecules. In this role,...Full timeLocal areaWorldwideRelocation package3 days per week$200k - $300k
...mathematics and computer science, physics and engineering, media and tech. We’re a community of... ...to know you.Please be advised: Use of AI tools during interviews or assessments is... ...agreed upon. We employ various methods to evaluate the authenticity of candidate responses....Work at officeLocal areaImmediate start$100k - $160k
We are seeking a Lead Engineer - GenAI, Agentic AI, and Knowledge Graph Architect to design, develop,... ...that combine:Large Language Models and multimodal foundation modelsSymbolic reasoning,... ...recovery, observability, evaluation, and governanceArchitect and deploy...- Stone Alliance Group is seeking an experienced Data Engineer to join the Center for Data Analytics, Innovation, and Rigor in New York... .... You will develop data pipelines, data infrastructure, and AI evaluation frameworks using Python, SQL, Docker, Kubernetes, AWS and...Full timeWork at office
$120k - $200k
...research accelerator for frontier AI labs and a trusted partner... ..., reasoning, coding, multimodality, and STEM. For enterprises,... ...system behind Turing’s software-engineering data programs, turning complex... ...review, and rubric-based evaluations. They can require coordinating...$150k - $250k
...Distyl AI Job Opportunity Distyl is an applied AI technology company partnering with... ...At Distyl, we build AI systems using Evaluation-Driven Development —an approach where evaluation... ...behavior in production. AI Evaluation Engineers focus on designing and implementing the...Work at officeFlexible hours3 days per weekDistyl AI
New York, NY22 hours ago$190k - $290k
...Field Engineer, Data Engine Scale AI is seeking a highly skilled and motivated AI Data Engineer to join our dynamic Federal Engineering team.... ...same role. This allows us to ensure a fair and thorough evaluation of all applicants. At Scale, our mission is to develop...Full timeScale AI
New York, NY22 hours ago- ...Role Responsibilities Review and refine AI-generated prompts, responses, and code... ...language Support benchmarking efforts to evaluate and compare model capabilities Requirements Strong experience in software engineering, technical research, or educational...Part timeRemote work
Crossing Hurdles
New York, NY1 day ago $102.75k - $195.25k
...integrations and knowledge bases that other engineering teams will depend on every day? Are you... ...vector databases, using embeddings and evaluation techniques to keep the LLM Wiki and AWS... ...knowledge bases.Engineer grounded AI capabilities by developing retrieval-augmented...Work at officeLocal areaRemote workVisa sponsorship- ...AI QA EngineerWe are seeking an AI QA Engineer to ensure the quality, reliability, safety, and performance of AI-powered applications and machine learning... ...safety.Create and manage test datasets, prompts, and evaluation benchmarks.Test AI features for functionality,...
Ova Technologies
New York, NY1 day ago - Mindrift connects specialists with project-based AI opportunities for leading tech companies. This role focuses on building a dataset to evaluate AI coding agents and how well models handle real-world developer tasks. The work is project-based, not permanent employment...Permanent employmentTemporary work
$193.8k - $285k
...systems and abstractions that DoorDash Engineering depends on: reliable, efficient, secure,... ...trade-offs.You have proficiency in using AI coding tools (e.g., Claude Code, Codex,... ...works: Gem assists our recruiting team by evaluating job related qualifications and...Hourly payWork at officeLocal areaRemote workFlexible hours- ...what's possibleAs a Senior Lead Software Engineer at JPMorganChase within the Corporate & Investment... ...test new technologies and methodologies, evaluating their potential impact on the firm.Drives adoption and governance of approved AI-assisted engineering practices across...
$103.75k - $174.75k
...technologies, and a commitment to back the broader engineering community through open source, our... ...and our colleagues.At American Express, AI is reshaping the future of commerce and... ...frameworks, vector databases, agent evaluation frameworks, and AI observability tooling...Internship
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Qualitative Evaluation Engineer, Multimodal AI. Be the first to apply!
